Using Tensor Cores instead of CUDA

[TimmekHW]

Let's use Tensor Cores instead of using the CUDA cores of the video card. This will speed up audio processing speed by 7 times!

[Eruore1997]

ok have you been able to do it?

[Thespacemanfil]

Do you have any proof of the potential performance gain?

[TimmekHW]

Only experience from SD 1.5 and SDXL

https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT

[Kuuko-fokkusugaru]

As far as I know, the software already uses the Tensor cores when available. But this is not a matter of using the Tensor cores = no CUDA cores being used. It's a bit more complex than that. It does use the Tensor cores through CUDA. In fact, with enough CUDA and Tensor cores, the software doesn't make much usage of the GPU and it's extremely fast. The "delay" that happens with the software is not a lack of performance, it's the normal delay that you need before processing the audio. The longer the chunk size, the more accurate the conversion will be. So if you add a 2 seconds chunk size, the software is not taking 2 seconds because it's slow, but because you are feeding it 2 seconds of audio before converting it. For me, with 0.3 seconds chunk size and increasing the extra value, I get the conversion with very high quality only taking the 0.3 seconds delay that I am feeding the software with.