Iterative VideoDecoder calls

[danikiyasseh]

📚 The doc issue

I am in the process of extracting features of video clips (~30s) from a long video (>10mins).

Is it preferable to:

1. Instantiate VideoDecoder once (with the long video) and simply retrieve clips based on indices
2. Instantiate VideoDecoder in each iteration of the loop (for each video clip) and retrieve a specific clip

Based on my experience with other decoder libraries (e.g., decord), there tends to be a memory leak in certain scenarios.

Thanks!

[NicolasHug]

If you know the timestamps / indices of all the frames of all the clips you want to sample, then calling get_frames_at / get_frames_played_at is probably more efficient than creating multiple instances of VideoDecoder.

I guess only benchmarks will give you a definite answer, but the guideline here would be to avoid backwards seeks. If you decode a frame at index i and then a frame at index j < i, on the same VideoDecoder object, that's potentially going to be slow.

As long as you call the samplers API, or something like get_frames_at / get_frames_played_at, then you would be avoiding backwards seeks (these APIs sort the timestamps internally).