4k performance improvable ?

[grobalt]

https://www.svp-team.com/forum/viewtopic.php?pid=81483#p81483 integrated this as a test -> in 1080p transcoding i reached around 115fps with RTX4090 and 22 fps for 2160p .. any idea to improve 4k performance ? :)

[WolframRhodium]

If tunings like increasing num_threads, setting use_cuda_graph are not useful, it is likely that the performance is limited by PCIe bandwidth rather than GPU compute. RIFE 4.6 (2x) alone should run at ~53.2 fps in 4k with 4090.

This problem is specific to rife in vs-mlrt and I will try to solve it. Thanks for the information!

Anyway, for advanced usage, it is possible to build a single trt model for multiple video resolutions by setting parameters opt_shapes, max_shapes and static_shape in Backend.TRT, but the performance is usually slower.

[grobalt]

Need to push this to SVP team as i dont know where to check the parameters num_threads etc. But i can confirm, i run the 4090 on a pcie4 x16 slot and another user confirms my framerate

[WolframRhodium]

We need additional information to determine why the performance degrades. It would be better if you could provide profiling information like nsight system data. The installer can be downloaded without registration from the cuda toolkit.

[grobalt]

thanks, i think i found the file that set the parameter: vsmlrt.py
which includes:

` @DataClass(frozen=False)
class TRT:
""" backend for nvidia gpus

    basic performance tuning:
    set fp16 = True (on RTX GPUs)
    increase num_streams
    increase workspace
    set use_cuda_graph = True
    """

    max_shapes: typing.Optional[typing.Tuple[int, int]] = None
    opt_shapes: typing.Optional[typing.Tuple[int, int]] = None
    fp16: bool = False
    device_id: int = 0
    workspace: int = 128
    verbose: bool = False
    use_cuda_graph: bool = False
    num_streams: int = 1
    use_cublas: bool = False # cuBLAS + cuBLASLt
    static_shape: bool = True
    tf32: bool = True
    log: bool = True

    # as of TensorRT 8.4, it can be turned off without performance penalty in most cases
    use_cudnn: bool = True
    use_edge_mask_convolutions: bool = True
    use_jit_convolutions: bool = True

    heuristic: bool = False # only supported on Ampere+ with TensorRT 8.5+

    # internal backend attributes
    supports_onnx_serialization: bool = False

`

but playing with streams 2 and cuda graph True did not change any performance .. maybe i am looking in the wrong place. currently downloading nsight system

[AkarinVS]

2160p video has 4x more data than 1080p. If your system can do 115fps at 1080p, then the max performance it can get for 2160p is 115/4 fps, which is around 28.75 fps.

You also need to consider the data movement cost, at 4k, each 22fps, the GPU requires 7.74 GB data from system DRAM and outputs 2.11GB. How much DRAM bandwidth do you have in your system? Even if it's not limited by PCIe bandwidth, it will very likely hit DRAM bandwidth ceiling. Remember that RIFE is not the only thing that uses the DRAM bandwidth, the rest of the VS script and SVP also consume DRAM bandwidth.

[grobalt]

Standard Ryzen system with 32 GB memory, dual rank 2 dimms.
The benchmarks https://github.com/AmusementClub/vs-mlrt/wiki/NVIDIA-GeForce-RTX-4090 show way more than i have, trying to figure out why (as my numbers are confirmed by other SVP users)

[AkarinVS]

I think it's just overhead from the rest of the VS script and SVP itself.

We have just benchmarked RIFE v4.6 on 2160p input with RTX 4090 (num_streams=2), and got 53 fps (The wiki shows result for RIFE v4.4). But note our benchmark only test the speed of RIFE alone, no source filter or any downstream filters. As I've mentioned, even a single filter will require 2GB/s DRAM bandwidth at 22 fps.

[grobalt]

So there is still enough bandwidth available. I think i struggle in setting up the parameters correctly .. .

how can i do the benchmark you did ? sorry .. i don't have developer skills

[WolframRhodium]

Launch a new svp process by specifying path to svp executable in nsight system, and then transcode for a short period of time. Collection of GPU metrics should be enabled.

[WolframRhodium]

Thanks for the report. What's the value of threads of svp?

Ideally, memory copies should overlap computation, and the performance would be even higher. However it seems that the GPU is idle most of the time.

[WolframRhodium]

Is it possible to grab the vapoursynth script for isolated analysis? Probably mpv has some overheads.

[grobalt]

Thanks for the report. What's the value of threads of svp?

Ideally, memory copies should overlap computation, and the performance would be even higher. However it seems that the GPU is idle most of the time.

the GPU thread setting in SVP is 1,2 or 4 -> all behave the same. Most likely no influence to the script.

[grobalt]

Is it possible to grab the vapoursynth script for isolated analysis? Probably mpv has some overheads.

i think it uses the file C:\Program Files (x86)\SVP 4\script\base.py
base.zip

[AkarinVS]

As clearly shown in the profile, the issue is not that RIFE is not fast enough, but mpv/SVP simply can't feed frame requests to the filter fast enough (see the next post for a profile where the GPU is fully saturated). In fact, the GPU sits idle for at least half the time.... In this particular case, no amount of tuning of vs-mlrt parameters can speed things up. One must find the real bottleneck elsewhere.

[WolframRhodium]

Here's the tuning target, in which computations overlap memory copies and the PCIe bandwidth is saturated. (num_streams=5, use_cuda_graph=True on A100, 108 fps at 4k)

report2.zip

[grobalt]

Here new report with a new script
32258583.zip

https://drive.google.com/drive/folders/1EMsewBfnbVhFmmE9hgCvWGQAFk9FzHWj?usp=share_link
report4

[WolframRhodium]

The .py script is not complete and some constants are missing. After these params are filled, the script should be runnable by the command vspipe.exe xxxx.py . and this command should be profiled.

[WolframRhodium]

Anyway, the optimal value of num_streams depends on hardware configuration, video resolution and the model itself, but higher values should saturate the system more. Note that the value of num_streams should not be greater than core.num_threads in .py, otherwise resources are wasted.

[hooke007]

Based on my personal experience, mpv's vapoursynth filter is not fast enough. Even though the simpliest script would make 4K-video's playback laggy on my 10875H i7 CPU.

video_in.set_output()

[aloola18]

I'm using 5900x+3070ti and got
30FPS with 3840x2176@30 > 60 with SVP transcoding.
~27fps with 3840x2176@25 > 50 with SVP transcoding.

so it's clearly OP's setup has problems.
maybe PSU is not strong enough? which PSU are you using?

[grobalt]

i dont think you use rife with these numbers :) my numbers are the same as for other SVP users with 4090 in the SVP forum ...

[WolframRhodium]

Could you please run a baseline benchmark?

test.vpy (run with vspipe -p test.vpy .):

import vapoursynth as vs
from vsmlrt import RIFE, Backend, RIFEModel
core = vs.core


num_streams = 2
use_cuda_graph = True
peak_performance = True


if peak_performance:
    src = core.std.BlankClip(width=3840, height=2176, length=2000, format=vs.RGBS, color=[0.5]*3)
else:
    src = core.std.BlankClip(width=3840, height=2160, length=2000, format=vs.YUV420P8, color=[128]*3)
    src = core.misc.SCDetect(src)
    src = core.resize.Bicubic(src, format=vs.RGBS, matrix_in_s="709")
    src = core.std.AddBorders(src, 0, 0, 0, 16)

backend = Backend.TRT(fp16=True, num_streams=num_streams, use_cuda_graph=use_cuda_graph)
flt = RIFE(src, multi=2, model=RIFEModel.v4_6, backend=backend)

if not peak_performance:
    flt = core.std.Crop(flt, 0, 0, 0, 16)

flt.set_output()

peak_performance=True is an ideal case.

It seems that use_cuda_graph=True is more useful for TCC driver mode (for professional cards) than WDDM.

[chainikdn]

rife_v4.6.onnx.1920x1088_fp16_workspace256_trt-8502_cudnn_I-fp16_O-fp32_NVIDIA-GeForce-RTX-2060_3dcbe72f.engine.cache.lock
"I-fp16_O-fp32" part looks suspicious

File 'C:\Program Files (x86)\SVP 4\rife\vsmlrt.py', line 948, in RIFE
return core.std.Interleave([clip, output])
Interleave: clips are mismatched in format starting at clip #1, passed Undefined[1920x1088] and RGBS[1920x1088]

[WolframRhodium]

Thanks, please update files again.

[hooke007]

Have no idea what happened but I got the impressing result. ref #19 (reply in thread)

in&out fp32
Output 4000 frames in 72.82 seconds (54.93 fps)

in&out fp16
Output 4000 frames in 52.98 seconds (75.50 fps)

edit: I checked it out. input fp32 with output fp16 would seriously make speed much slow (Output 4000 frames in 107.41 seconds (37.24 fps))

[chainikdn]

can't say anything other than "it works" cause I'm GPU-capped and there's no perf. difference in this case

[WolframRhodium]

don't know why input fp32 with output fp16 is slow

[hooke007]

Graet work!
Now RTX4080 could make 24fp→48fps(3840X2160yuv444p12→p16) smooth in mpv.

[grobalt]

for me hdr is working as well .. but somehow ... an LG test clip is not working and i dont understand why not

https://4kmedia.org/lg-new-york-hdr-uhd-4k-demo/

[hooke007]

Smooth here. Could be another issue on your side.

[grobalt]

2 rife threads ? 10875H i7 CPU ?

[hooke007]

Nope, that's my obsolete device (10875H & 2070s). I use 7950x & 4080 now.

[grobalt]

Ok .. 7950 with good memory has nearly double memory bandwidth than my Ryzen 5600x with 3800Mhz mem and 2 CCD as well

[aloola18]

I just want to report that turning OFF Hardware-accelerated GPU greatly increases FPS on Windows 11.

ON
https://cdn.discordapp.com/attachments/290709370600423424/1066759015536799855/image.png

OFF
https://cdn.discordapp.com/attachments/290709370600423424/1066757184819576882/image.png

[hooke007]

use x264/265 instead, unless you are sure that "Hardware-accelerated" doesn't affect nvenc.

[aloola18]

use x264/265 instead, unless you are sure that "Hardware-accelerated" doesn't affect nvenc.

sorry I mean Hardware-accelerated GPU scheduling windows 11.
turn it off in System > display > graphics > default graphics settings >

[hooke007]

I know what you were talking about.
I mean you'd better to use vspipe only to test. Because you don't know if "Hardware-accelerated" would affect other processing stages.

[aloola18]

The old script I saw above:
OFF: Output 4000 frames in 104.49 seconds (38.28 fps)
ON: Output 4000 frames in 108.95 seconds (36.71 fps)

edited the script output with FP16-FP16
OFF: Output 4000 frames in 75.87 seconds (52.72 fps)
ON: Output 4000 frames in 85.03 seconds (47.04 fps)

[WolframRhodium]

We observe similar improvements here. It seems nv's hardware scheduling implementation reduces the number of concurrent tasks for reduced latency, but throughout is sacrificed.

[dtlnor]

here is some result I tested with vsmlrt v13.1

I use the script above and use vspipe to benchmark. It gives very weird result. Idk if this is problem from my end.

vsmlrt v13.1

• AMD Ryzen™ 9 7950X
• DDR5 2x32 6000mhz
• Nvidia 4090 (wddm, ecc disabled, rBar enabled)
• GPU processor clock @ 2490 MHz
• GPU memory clock @ 1469 MHz
• driver 528.49
• CUDA driver: 12.0.147
• Windows 10 22H2
• Hardware accelerated GPU scheduling OFF
• VapourSynth R60 (py 3.10 64bit)
• backend:
  fp16 = True, use_cuda_graph = True ( using the script above)

model		TRT 1 stream	TRT 2 streams
rife io-fp16 peak static (3840x2176, model=46)		64.06 / 1706 (report 14, ~68% usage)	123.22 / 2720 (report 13, ~100% usage)
rife io-fp16 overhead static (3840x2176, model=46)		56.41 / 1680 (report 15, ~60% usage)	68.37 / 2721 (report 16, ~80% usage)
rife io-fp32 peak static (3840x2176, model=46)		31.99 / 1974 (report 19, ~50% usage)	49.73 / 3327 (report 18 , ~70% usage)
rife io-fp32 overhead static (3840x2176, model=46)		37.01 / 1983 (report 22, ~60% usage)	53.92 / 3381 (report 21 , ~80% usage)

some reports if anyone interested: https://drive.google.com/drive/folders/1zhigwILaQm13FgY4O5w7oZ18iABY5Lhw?usp=share_link

only rife io-fp16 peak performance has reach 100% cuda usage and gives > 120fps result. Others just not able to.
Also there is a weird case is for io-fp32 peak, I had reach 74.59fps once, with 100% cuda usage(I didn't do profiling with that run).
I have never reach 70fps again since that one.

note: I didn't flip the result of io-fp32 peak and overhead, the peak performance one is slower than the mpv simulate one. I have no idea why.

[WolframRhodium]

RIFE performance on multi-chiplet processors may be affected by inter-chiplet bandwidth. Please consider to lower the value of core.num_threads to 8, for example, or other thread pinning techniques.

[MysteryDove]

We've monitored huge performance hit when running on a 5950x cpu with no thread limit, please consider bind the workload to single ccd by lowering core.num_threads or other software

[dtlnor]

@MysteryDove @WolframRhodium Ah, I miss that statement. Thank you for your helps. The value between 4~16 are stable for me (~73fps), not setting the core.num_threads or over 28 are likely to taking ~49fps

[dtlnor]

Here is my new result with core.num_threads = 16. Its OKey new. You guys are awesome.

model		TRT 1 stream	TRT 2 streams
rife io-fp16 peak static (3840x2176, model=46)		63.34 / 1687	121.22 / 2708
rife io-fp16 overhead static (3840x2176, model=46)		55.65 / 1695	69.14 / 2706
rife io-fp32 peak static (3840x2176, model=46)		39.06 / 2110	74.39 / 3331
rife io-fp32 overhead static (3840x2176, model=46)		36.73 / 1999	52.09 / 3312