Packager fails to remove old media segments if both HLS and DASH is being used for LIVE segmentation

[tamasgp]

System info

Operating System: Ubuntu 22.04.2 LTS
Shaka Packager Version: 5ee2b7f-release

Issue and steps to reproduce the problem

Packager Command:
packager-linux-x64 in=udp://230.2.2.222:1111?interface=172.22.238.253&reuse=1,stream=audio,drm_label=AUDIO,init_segment=audio_init.mp4,segment_template=audio_$Number$.m4s,playlist_name=/mnt/life/audio.m3u8,hls_group_id=audio in=udp://230.2.2.222:1111?interface=172.22.238.253&reuse=1,stream=video,init_segment=p00_init.mp4,segment_template=p00_$Number$.m4s,playlist_name=p00.m3u8 in=udp://230.2.2.222:1112?interface=172.22.238.253&reuse=1,stream=video,init_segment=p01_init.mp4,segment_template=p01_$Number$.m4s,playlist_name=p01.m3u8 in=udp://230.2.2.222:1113?interface=172.22.238.253&reuse=1,stream=video,init_segment=p02_init.mp4,segment_template=p02_$Number$.m4s,playlist_name=p02.m3u8 --hls_master_playlist_output /mnt/life/life.m3u8 --hls_playlist_type LIVE --segment_duration 6.4 --preserved_segments_outside_live_window 2 --time_shift_buffer_depth 32 --mpd_output /mnt/life/life.mpd --suggested_presentation_delay 25

What is the expected result?

Old (outside the live window) media segments should be removed without an error.

What happens instead?

In case of using both HLS and DASH manifest generation, both formats' "RemoveOldSegment" function is being triggered. The media segments are removed by the first format's invocation, but the other format shows a warning message and keeps the segment filenames in a list (to try to remove them later). This "segments_to_be_removed_ list" will grow forever, which can lead to OOM / memory leak.

Media segments should be removed only once in a common code section, or if the file removal is being failed with a "not found" error, then the filename should be popd from the removal queue.

[acris5]

This can easily be fixed with a patch:

diff --git a/packager/file/local_file.cc b/packager/file/local_file.cc
index 5ddc304e..37144e2d 100644
--- a/packager/file/local_file.cc
+++ b/packager/file/local_file.cc
@@ -139,7 +139,9 @@ bool LocalFile::Delete(const char* file_name) {
   auto file_path = std::filesystem::u8path(file_name);
   std::error_code ec;
   // On error (ec truthy), remove() will return false anyway.
-  return std::filesystem::remove(file_path, ec);
+  if (std::filesystem::exists(file_path))
+    return std::filesystem::remove(file_path, ec);
+  else return true;
 }
 
 }  // namespace shaka

[joeyparrish]

@acris5, thanks for the patch. Would you be willing to make a PR?

Also, would it work to just ignore the return value of remove() instead of checking exists()?