RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling cublasGemmEx

[mesmerizinq]

When using the GPU in wsl2, I get the following error:

It occurs during training in the very first iteration.

The pixel drawer worked fine before I set up the GPU, since it used the CPU for training and had compiled diffvg for CPU, but now that my GPU is set up, I can't even replicate using the CPU without purging the nvidia driver (compiling diffvg for CPU and setting torch.cuda.is_available = lambda : False just leads to another error).

[ajmeese7]

Hmmmm, this is an interesting issue. I am not available to look into it at the moment but I will see if I can find some time tomorrow to dig deeper into it.

[mesmerizinq]

Thanks so much. It's not the end of the world if there's no easily found fix, I just very much enjoy this project for the straight-forward code and the interesting art. Sadly, setting the device to 'cpu' and just using the pixeldrawer still generates an error (under wsl2), albeit one that may be much easier for you to understand.

The error occurs while encoding ViT-B/32, even though everything looks just the same as on my linux laptop which doesn't have a nvidia GPU and is thereby "forced" to use the cpu as its device, where everything works fine.

Any help is greatly appreciated, but I can fully understand if these errors are too obscure, since there's a good chance they can be attributed to the use of wsl2 in some way.

[ajmeese7]

This thread recommends upgrading your CUDA version, I'm not sure what version you are running now but this solution may solve your issue. This GitHub thread may also offer some insight into your issue, I'm not sure if their proposed solutions will do the trick for you.

If those don't work, I will not be able to help you troubleshoot much further unfortunately, because I no longer have any Windows boxes to test on to see if I can reproduce the issue. Please let me know.