RIFE

RIFE is a Real-time Intermediate Flow Estimation algorithm for video frame interpolation. It supports arbitrary-timestep interpolation between a pair of inputs.

Link:

• (stable) https://github.com/AmusementClub/vs-mlrt/releases/download/model-20220923/rife_v8.7z

Includes these models:

• v4.0, v4.2, v4.3, v4.4, v4.5, v4.6, v4.7, v4.8, v4.9, v4.10: RGB frame interpolation

Requirements & Parameters

1. block_w and block_h (tile size) must be multiples of 32 / scale.

vsmlrt.py wrapper Usage

In order to simplify usage, we provided a Python wrapper module vsmlrt:

from vsmlrt import RIFE, RIFEModel, Backend

src = core.std.BlankClip(format=vs.RGBS)

# backend could be:
#  - CPU Backend.OV_CPU(): the recommended CPU backend; generally faster than ORT-CPU.
#  - CPU Backend.ORT_CPU(num_streams=1, verbosity=2): vs-ort cpu backend.
#  - GPU Backend.ORT_CUDA(device_id=0, cudnn_benchmark=True, num_streams=1, verbosity=2)
#     - use device_id to select device
#     - set cudnn_benchmark=False to reduce script reload latency when debugging, but with slight throughput performance penalty.
#  - GPU Backend.TRT(fp16=True, device_id=0, num_streams=1): TensorRT runtime, the fastest NV GPU runtime.
flt = RIFE(src, model=RIFEModel.v4_4, backend=Backend.ORT_CUDA())

Raw Model Usage

not recommended

Benchmarking

Measurements: FPS / Device Memory (MB)

Device memory:

• GPU: device memory including context

RTX 3090

Software: VapourSynth R60, Windows Server 2019, Graphics Driver 526.47.

Input size: 1920x1088

Backends

1. vs-mlrt v12

Performance

FP16

Model	[1] ort-cuda	[1] trt	[1] ort-cuda (2 streams)	[1] trt (2 streams)
v4.4	53.62 / 1771	71.30 / 626	83.34 / 2926	107.3 / 962