Subtitle without a DEFAULT=YES, AUTOSELECT=YES

[aminvakil]

We use androidx/media (former exoplayer) to play m3u8s created by nginx-vod-module, and it crashes when there is an empty subtitle track by default.
So I was wondering what is the standard for this and if there MUST be a AUTOSELECT=YES in every m3u8, and I've reached this:
https://datatracker.ietf.org/doc/html/rfc8216:

      DEFAULT

      The value is an enumerated-string; valid strings are YES and NO.
      If the value is YES, then the client SHOULD play this Rendition of
      the content in the absence of information from the user indicating
      a different choice.  This attribute is OPTIONAL.  Its absence
      indicates an implicit value of NO.

      AUTOSELECT

      The value is an enumerated-string; valid strings are YES and NO.
      This attribute is OPTIONAL.  Its absence indicates an implicit
      value of NO.  If the value is YES, then the client MAY choose to
      play this Rendition in the absence of explicit user preference
      because it matches the current playback environment, such as
      chosen system language.

      If the AUTOSELECT attribute is present, its value MUST be YES if
      the value of the DEFAULT attribute is YES.

      FORCED

      The value is an enumerated-string; valid strings are YES and NO.
      This attribute is OPTIONAL.  Its absence indicates an implicit
      value of NO.  The FORCED attribute MUST NOT be present unless the
      TYPE is SUBTITLES.

      A value of YES indicates that the Rendition contains content that
      is considered essential to play.  When selecting a FORCED
      Rendition, a client SHOULD choose the one that best matches the
      current playback environment (e.g., language).

      A value of NO indicates that the Rendition contains content that
      is intended to be played in response to explicit user request.

which from what I understand both autoselect and default are optional, and it has not mentioned that there SHOULD be a DEFAULT=YES, AUTOSELECT=YES in master.m3u8.

So right now we cannot include subtitles in m3u8 with an empty default, I was wondering how can we have subtitles in m3u8 which none of them are default, and it can be manipulated in json, if having a default subtitle is desirable.

Something like this:

#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs0",NAME="فارسی",LANGUAGE="fa",AUTOSELECT=NO,DEFAULT=NO,URI="index-f6.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs0",NAME="انگلیسی",LANGUAGE="en",AUTOSELECT=NO,DEFAULT=NO,URI="index-f7.m3u8"

and

#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs0",NAME="فارسی",LANGUAGE="fa",AUTOSELECT=YES,DEFAULT=YES,URI="index-f6.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs0",NAME="انگلیسی",LANGUAGE="en",AUTOSELECT=NO,DEFAULT=NO,URI="index-f7.m3u8"

P.S.
I totally understand if you say this is the way currently is and you don't have time / energy to change this, but I would appreciate if you could point me to the correct directions of how to change this, therefore I can whip up a PR for this, so everyone benefits from this new feature, thank you!

[maximk-1]

Set up nginx-proxy in front of kaltura with sub_filter/subs_filter/js_body_filter.

[aminvakil]

@maximk-1 Yeah, this has crossed our mind too, but we thought if there was a straight way to do this instead of hacking our way into it.

[aminvakil]

Also we do not want to disable autoselect in all playlists, some playlists may be needed to have a default autoselected subtitle.

[erankor]

Hi, can you explain what do you mean by 'empty subtitle track'? because my interpretation of 'empty' is a subtitle track without any cues, and in that case it sounds very weird. It is possible, for example, that the first subtitle cue arrives only after a minute (e.g. a video that starts by playing some music) - the client has no way of knowing whether the subtitle track is empty or not, before it downloads all the subtitle segments. Also, even when the input file is empty, the module serves valid WebVTT segments to the client - that contain only the WEBVTT header.

[aminvakil]

Hi, can you explain what do you mean by 'empty subtitle track'? because my interpretation of 'empty' is a subtitle track without any cues, and in that case it sounds very weird. It is possible, for example, that the first subtitle cue arrives only after a minute (e.g. a video that starts by playing some music) - the client has no way of knowing whether the subtitle track is empty or not, before it downloads all the subtitle segments.

Sorry, I should've been more informative, by empty I mean something like this:

{
    "sequences": [
        {
            "clips": [
                {
                    "path": "/usr/share/nginx/html/multim3u8/once.mp4",
                    "type": "source"
                }
            ],
            "language": "en",
            "label": "English",
            "description": "English audio"
        },
        {
            "clips": [
                {
                    "path": "empty",
                    "type": "source"
                }
            ],
            "label": "Without subtitle"
        },
        {
            "clips": [
                {
                    "path": "/usr/share/nginx/html/multim3u8/once.srt",
                    "type": "source"
                }
            ],
            "language": "en",
            "label": "English"
        }
    ]
}

Adding a long duration subtitle without any real subtitle fixed the issue for us, something like this:

WEBVTT

00:00:00.000 --> 20:00:05.000

 

Also, even when the input file is empty, the module serves valid WebVTT segments to the client - that contain only the WEBVTT header.

https://www.w3.org/TR/webvtt1/#file-structure

An optional U+FEFF BYTE ORDER MARK (BOM) character.
The string "WEBVTT".
Optionally, either a U+0020 SPACE character or a U+0009 CHARACTER TABULATION (tab) character followed by any number of characters that are not U+000A LINE FEED (LF) or U+000D CARRIAGE RETURN (CR) characters.
Two or more [WebVTT line terminators](https://www.w3.org/TR/webvtt1/#webvtt-line-terminator) to terminate the line with the file magic and separate it from the rest of the body.
Zero or more [WebVTT region definition blocks](https://www.w3.org/TR/webvtt1/#webvtt-region-definition-block), [WebVTT style blocks](https://www.w3.org/TR/webvtt1/#webvtt-style-block) and [WebVTT comment blocks](https://www.w3.org/TR/webvtt1/#webvtt-comment-block) separated from each other by one or more [WebVTT line terminators](https://www.w3.org/TR/webvtt1/#webvtt-line-terminator).
Zero or more [WebVTT line terminators](https://www.w3.org/TR/webvtt1/#webvtt-line-terminator).
Zero or more [WebVTT cue blocks](https://www.w3.org/TR/webvtt1/#webvtt-cue-block) and [WebVTT comment blocks](https://www.w3.org/TR/webvtt1/#webvtt-comment-block) separated from each other by one or more [WebVTT line terminators](https://www.w3.org/TR/webvtt1/#webvtt-line-terminator).
Zero or more [WebVTT line terminators](https://www.w3.org/TR/webvtt1/#webvtt-line-terminator).

Please correct me, but the json which I've just posted provides this in master.m3u8:

#EXTM3U

#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs0",NAME="Without subtitle",AUTOSELECT=YES,DEFAULT=YES,URI="index-f2.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs0",NAME="English",LANGUAGE="en",AUTOSELECT=NO,DEFAULT=NO,URI="index-f3.m3u8"

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2335290,RESOLUTION=1920x800,FRAME-RATE=23.974,CODECS="avc1.640029,mp4a.40.2",SUBTITLES="subs0"
index-f1-v1-a1.m3u8

#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=166759,RESOLUTION=1920x800,CODECS="avc1.640029",URI="iframes-f1-v1-a1.m3u8"

and index-f2.m3u8 (path: empty) returns:

#EXTM3U
#EXT-X-TARGETDURATION:10
#EXT-X-ALLOW-CACHE:YES
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:1
#EXT-X-ENDLIST

So there is no WebVTT file, whether it has valid WebVTT headers or not.

[erankor]

You are using empty subtitles with a single MP4 video, what's the point?
This feature exists for cases in which you stitch multiple videos, that have different subtitle languages. For example, if you stitch a logo before the main video, the logo may not have subtitles at all. The module requires a complete matrix of sequences x clips, so the gaps have to be filled with something. For example, if the main video has EN subtitles, the logo video would use empty for the EN subtitles.
What happens in your case is that you don't define the video duration in the JSON (it is required when stitching multiple videos, but optional when using a single video), and when the module gets a request for the subtitles, it "sees" a zero duration (the module doesn't read the video file when getting a subtitles request). So, you get this empty m3u8, which is indeed invalid.

[aminvakil]

You are using empty subtitles with a single MP4 video, what's the point?

To not show subtitle by default, how should we do that?

[aminvakil]

If module does not always picks up the first subtitle and appends DEFAULT=YES, AUTOSELECT=YES to it, so there would be no default subtitle, AND there was an option which you can have a default subtitle, it would be fine.

[erankor]

Got it, makes sense. I agree the correct solution is to make the default/autoselect configurable, as you initially suggested.
A temporary workaround may be to use empty, but specify the video duration in the JSON, that should fix the empty m3u8.

[aminvakil]

@erankor We do not have exact duration of all sequences currently, so we cannot add duration right now.

I would appreciate if you could point me to where to look to make it configurable, so I can come up with a PR improving this.

[erankor]

Hi, had some free time... opened #1506 for this, please give it a try (I didn't have time to test it myself, but seems pretty simple)

[erankor]

Thanks, merged

[aminvakil]

Thank you!