Any tips on speeding things up while using google colab?

[juan9999]

I tried renting a premium GPU (A100 40) on colab today and while it was certainly 2X as fast, it was only 2X as fast (and 5X the cost). RAM utilization seemed to max out at about 10-20%

Is this because all the files are read from google drive vs SSD? Can this be sped up? i.e. is there a way to install the models etc to the SSD of the Virtual machine and only copy generated models/ generated images at the beginning and end of the notebook?

[TheLastBen]

A100 isn't well suited for Dreambooth or SD, T4 is the best for that, and if you're using dreambooth for providing services, consider the V100

[juan9999]

Thanks for the tip. I haven't had a V100 assigned to me yet to test out! How do I go about getting one?

[TheLastBen]

The V100 in case you're running google cloud or any other platform, but for google colab, you're stuck with the overpriced A100 which is a waste of money if you're using dreambooth. I suggest using the standard T4.

[juan9999]

Gotcha.

Thanks. Any idea how a T4 compares to a 3090?

Runbot have 3090's for $.44/h (so 2X cost of T4 on google but theoretically 4X faster).

I don't mind maybe using the T4 to generate models since I'm not waiting but when I'm playing with image generation, I'd prefer something zippier!

[TheLastBen]

3090 isn't very far from the T4, my advice is stick with the T4, it's more economical.

[juan9999]

I must be doing something wrong then or maybe having stuff on google colab is very slow. I'm using the free 1.5 member servers on https://lukium.ai/ which run using a 3090 and image generation is at least 5X as much as I'm getting on colab using a T4 with extra (on $10 plan). Only got 2it/s with the A100 on image generation

Is it because colab uses google drive (which is very convenient)

[TheLastBen]

The storage has no effect on the speed, only the GPU, what resolution does take 2it/s ?

[juan9999]

I was doing 756*756 no highres fix

[TheLastBen]

You will have to experiment with different GPUs to determine which one has the best performance/price ratio.