Player panics when large gaps in playback stream

[yaruno]

Hi,

Love your project and got a couple of questions related to managing media streams that are missing parts of it.

I'm testing out delivering h264/h265 mpeg-ts stream over webrtc to the player. Everything works flawlessly in good networking conditions (i.e. no packet loss) but when I introduce gaps or packet loss to the stream the player starts to panic and starts giving out [MSEController] > Failed to execute 'appendBuffer' on 'SourceBuffer': The HTMLMediaElement.error attribute is not null errors.

To my understanding this comes from the deeper browser side as it's unable to 'bridge the gaps' between segments that it is attempting to play. Now naturally I'd like to solve this issue in some smart way. In your opinion what would the best approach to handle or managing the missing information or gaps on video & audio streams? Is there a way to quickly reset the media player or should this be handled at remuxer e.g. by inserting last received video frames again when gap is detected or maybe h264payload parser level? At the moment I'm resorting on reloading the player when it panics but it's not a great way going forwards.

[xqq]

@yaruno

The player could only provide a log like Failed to execute 'appendBuffer' on 'SourceBuffer': The HTMLMediaElement.error attribute is not null errors. since the JavaScript side doesn't provide any details about the reason for the playback error.

Firstly you'd better check the browser's internal media logs like chrome://media-internals, or the Media Tab in the DevTools for more detailed error information.

[yaruno]

Hi xqq,

Yeah the JS unfortunately isn't very verbose about the errors. For reference, here's what chrome://media-internals says when the player panics.

What I've been testing out has been sending a h264 video feed to a player and then introducing gaps into it by omitting e.g. 10 % of video packets. I'm curious on how the player handles gaps in mpeg-ts video feed but so far it's not very happy about them. In contrast to audio only streams over mpeg-ts which seems to be more robust as when gaps appear as it seems like silent frames are added in the remuxing phase to the audio stream.

I'm curious what's your opinion on handling potential gaps or missing frame data on the video stream. What would be the best approach to tackle them?

[HCLAC]

Hi, yaruno, I use the data channel of WebRTC to transmit H265 format videos to the web. I would like to consult with you on how to play videos using Mpets.js?