From aae96753e0e2dac9bf2cb3658136485be9b4245e Mon Sep 17 00:00:00 2001 From: Philip de Nier Date: Wed, 28 Aug 2024 15:33:18 +0100 Subject: [PATCH 1/9] adr: add flow bit rate property definitions ADR --- docs/README.md | 2 + docs/adr/0022-flow-bit-rate-properties.md | 162 ++++++++++++++++++++++ 2 files changed, 164 insertions(+) create mode 100644 docs/adr/0022-flow-bit-rate-properties.md diff --git a/docs/README.md b/docs/README.md index bf1a08a..fdb032a 100644 --- a/docs/README.md +++ b/docs/README.md @@ -30,6 +30,8 @@ For more information on how we use ADRs, see [here](./adr/README.md). | [0018](./adr/0018-restrict-direct-source-modification.md) | Restrict direct Source modification | | [0019](./adr/0019-consolidate-modified-updated-terms.md) | Rename `modified_by` properties in Source and Flow schemas to `updated_by` | | [0020](./adr/0020-version-signalling.md) | Improving the signalling of the supported API version in implementations | +| [0021](./adr/0021-storage-label-format.md) | Options for `get_urls` labels on Flow Segments | +| [0022](./adr/0022-flow-bit-rate-properties) | Definitions for Flow bit rate properties and additional properties | | [0025](./adr/0025-flow-property-updates.md) | Options for updating Flow properties | \* Note: ADR 0004a was the unintended result of a number clash in the early development of TAMS which wasn't caught before publication diff --git a/docs/adr/0022-flow-bit-rate-properties.md b/docs/adr/0022-flow-bit-rate-properties.md new file mode 100644 index 0000000..0945025 --- /dev/null +++ b/docs/adr/0022-flow-bit-rate-properties.md @@ -0,0 +1,162 @@ +--- +status: "proposed" +--- +# Flow Bit Rate Properties + +## Context and Problem Statement + +The current specification of a Flow includes the `avg_bit_rate` and `max_bit_rate` properties. +The specification does not define in detail what these values represent. +The purpose of this ADR is to provide some clarity to what these properties are, providing improved interoperability by avoiding differing and incompatible interpretations. + +A question that needs answering is whether the bit rates are for the essence data, the segments or both. + +The TAMS generally stores media in segments. +A question is whether the properties in HLS and DASH are applicable to TAMS where the segments may not have the same restrictions to support delivery to end-user clients. +In the most extreme case, a Flow could be stored as a single segment like a regular media file. + +### Essence Bit Rates + +One interpretation of these properties is that they measure media essence (e.g. H.264 and AAC) bit rates. +These properties relate to the essence only but don't account for the container overhead or network conditions. + +Essence bit rate information can be carried in the coded bitstream, such as the `hrd_parameters` structure in the H.264 bitstream that specifies the maximum bit rate into the CPB (Coded Picture Buffer) in the HRD (Hypothetical Reference Decoder). + +Essence bit rate information can also be found in the container. +The MXF AVC (H.264) sub-descriptor defined in SMPTE ST 381-3 includes the `AVC Maximum Bitrate` and `AVC Average Bitrate` properties. +MP4 / Quicktime has a `btrt` box that includes `Max bit rate` and `Average bit rate` properties. +The MPEG-4 Systems (ISO/IEC 14496-1) defines a descriptor containing `maxBitRate`, `avgBitRate` and `bufferSizeDB` properties; this descriptor can be included in the MP4 / Quicktime `esds` box. + +### Segment Bit Rates + +Another interpretation of these properties is that they measure media segment (e.g. MPEG-TS and fMP4) bit rates. +These properties relate to the segment files that contain the media essence but don't account for network conditions. + +The HLS (HTTP Live Streaming) specification includes a required `BANDWIDTH` Variant Stream property that specifies the largest sum of `peak segment bit rate` values. +The `AVERAGE-BANDWIDTH` Variant Stream property specifies the largest sum of `average segment bit rate` values. +HLS also defines a required `EXT-X-TARGETDURATION` tag that specifies the target maximum segment duration. + +The DASH (Dynamic Adaptive Streaming over HTTP) specification includes a `bandwidth` Representation attribute and `minBufferTime` MPD attribute that together specify the sufficient conditions that allow continuous playback after buffering `minBufferTime` with a constant bit rate channel with bit rate equal to `bandwidth`. +DASH also specifies a `maxSegmentDuration` MPD attribute that specifies the maximum segment duration in any media Representation. + +The SCTE 214-1 (Society of Cable Telecommunications Engineers) "MPEG DASH for IP-Based Cable Services, Part 1: MPD Constraints and Extensions", section 10.3, is a useful resource to understand the DASH and HLS properties and how they can be used to calculate receiver buffer size or buffer times for continuous playback (under ideal network conditions). +The SCTE 214-1 specification adds a DASH extension `maxSegmentRate` (note that `Rate` is bit rate) Representation attribute to DASH that comparable to the HLS `BANDWIDTH` property (but applies to a single Representation). + +## Considered Options + +* Option 1: Leave the definition of bit rates to be provided externally. +* Option 2: Define essence bit rates. +* Option 3a: Define segment bit rates. +* Option 3b: Define segment bit rates with additional properties. +* Option 4: Define both essence and segment bit rate properties. + +## Decision Outcome + +tbd + +### Implementation + +tbd + +## Pros and Cons of the Options + +### Option 1: Leave the definition of bit rates to be provided externally + +This is the current state +The definition is left to the applications using TAMS. + +* Good, provides flexibility for applications to decide what bit rate definition is most relevant and useful +* Bad, reduces interoperability because the property definition in use may be unknown, which leads to values being misinterpreted or ignored +* Bad, multiple definitions need to be supported for different applications, increasing the implementation burden + +### Option 2: Define essence bit rates + +The `avg_bit_rate` is calculated as `int(total_essence_bit_size / (total_essence_duration_sec * 1000))` (kbit/sec) of essence data in the Flow. + +The `avg_bit_rate` could be calculated from the essence data that is present in the TAMS. +The value could be an expectation of what could be present in the TAMS (e.g. derived from an encoder setting for a live ingest). +The value could be an approximation. +The value could change over time as essence data becomes available or is removed from TAMS. + +The `max_bit_rate` is calculated as `max(max_bit_rate, int(window_essence_bit_size / 1000))` for any 1 second time window of essence data. + +This definition is similar to the MPEG-4 Systems (ISO/IEC 14496-1) `maxBitRate`, which supports different essence types when compared with something like `AVC Maximum Bitrate` in MXF that is specific to H.264 and the HRD. + +The `max_bit_rate` can be interpreted as the bit rate for a constant bit rate network channel that along with a 1 second buffer is sufficient to support continuous playback. + +The `max_bit_rate` could be calculated from the essence data that is present in the TAMS. +The value could be an expectation of what could be present in the TAMS (e.g. derived from an encoder setting for a live ingest). +The value could be an approximation, ideally an upper bound (e.g. the bit rate from the H.264 `hrd_parameters`). +The value could change over time as essence data becomes available or is removed from TAMS. + +* Good, provides bit rate information that is independent of the container, allowing it to be used in streaming contexts where no or a different media container is used +* Bad, it doesn't directly provide bit rate information for users of TAMS where the segments are the means for accessing essence + +### Option 3a: Define segment bit rates + +The `avg_bit_rate` is calculated as `int(total_segment_bit_size / (total_segment_duration_sec * 1000))` (kbit/sec) for the segments of the Flow. + +The `avg_bit_rate` could be calculated from the segments that are present in the TAMS. +The value could be an expectation of what could be present in the TAMS (e.g. derived from an encoder / muxer setting for a live ingest). +The value could be an approximation. +The value could change over time as segments becomes available or are removed from TAMS. + +The `max_bit_rate` is calculated as `int(peak_segment_bit_rate / 1000)`. +The `peak_segment_bit_rate` corresponds to the definition in HLS, where it is defined as the maximum bit rate for a contiguous sequence of segments that has a duration between 0.5 and 1.5 the target segment duration. +The HLS definition is extended to include single segments above 1.5 the target segment duration to handle Flows that don't follow the HLS retrictions for segment durations, e.g. segmentation optimised for best quality, with variable and potentially large GOP sizes etc.. + +The `max_bit_rate` could be calculated from the segments that are present in the TAMS. +The value could be an expectation of what could be present in the TAMS (e.g. derived from an encoder / muxer setting for a live ingest). +The value could be an approximation, ideally an upper bound (e.g. the bit rate from the H.264 `hrd_parameters` plus some container overhead approximation). +The value could change over time as segments become available or are removed from TAMS. + +* Good, provides bit rate information for TAMS users that access essence via segments +* Bad, the information provided by the Flow is incomplete and doesn't allow the bit rate to be used to decide how much segment data needs to be buffered in the receiver for example +* Bad, it doesn't directly provide essence bit rate information for users of TAMS where the essence in other contexts is not transmitted in the same segments as TAMS + +### Option 3b: Define segment bit rates and additional properties + +In [Option 3a](#option-3a-define-segment-bit-rates) the `max_bit_rate` on it's own doesn't provide sufficient information to calculate the size or duration of the receiver buffer needed to allow for continuous playback (under ideal network conditions). +The descriptions and calculations from SCTE 214-1 section 10.3 provides insights to what information could be provided. + +A [_tams_segmentation_rate](../appnotes/0003-tag-names#_tams_segmentation_rate) Flow tag is currently used in some applications to provide a target segment rate for the Flow. +This segment rate provides the target segment duration (1 / segment rate) that is used to calculate the `max_bit_rate`. +The proposal is to add a `segment_duration` (rational, seconds) property that specifies the target segment duration for the Flow. +A duration is used rather than a rate to avoid confusion with bit and frame rates and to match the descriptions and properties in the HLS and DASH specifications. +A rational type allows more precise values to be set (e.g. to match frames rates), avoiding potential rounding issues when doing calculations over long periods of time. + +The description in SCTE 214-1 section 10.3 includes a formula for calculating the buffer size required for continuous playback. +The buffer size is defined as `1.1 * MSR[R] * SDmax`. +`MSR[R]` in the TAMS context is `max_bit_rate`, `SDmax` is the maximum segment duration and `1.1` adds a 10% overhead for additional event data. +The proposal is to add a `max_segment_duration` (floating point, seconds) property that provides the maximum segment duration (`SDmax`) for the Flow. +The `SDmax` is approximately `min(segment_duration * 1.5, max_segment_duration)` and the minimum buffer size in bytes is then approximately `max_bit_rate * 1000 * SDmax * 8`. +The calculations break down once segments exceed the 1.5 times the `segment_duration` as a segment won't fit into the buffer. +SCTE 214-1 puts limits on the segment size so that they don't exceed the buffer size. + +Would a property equivalent to `minBufferTime` in DASH be useful as well? + +The TAMS doesn't require media to be segmented in a way that is compatible with the restrictions applied to HLS and DASH segments. +Some applications my optimise segments for throughput rather than taking latency into account. +E.g. a 10 MB segment size may be the optimal segment size for an object store for maximum throughput. +The proposal is to add `segment_size` (integer, bytes) and `max_segment_size` (integer, bytes) properties for applications that segment based on size rather than duration. + +In summary, this option extends [Option 3a](#option-3a-define-segment-bit-rates) with these properties: + +* `segment_duration` (rational, seconds): the target segment duration for segmenting the Flow media +* `max_segment_duration` (floating point, seconds): the maximum segment duration in TAMS for the Flow +* `segment_size` (integer, bytes): the target segment size for segmenting the Flow media +* `max_segment_size` (integer, bytes): the maximim segment size in TAMS for the Flow + +* Good, provides information about the segments that allows the bit rate information to be used to estimate receiver buffer sizes required for continuous playback (not taking network conditions into account) +* Bad, it doesn't directly provide essence bit rate information for users of TAMS where the essence is transmitted by other means + +### Option 4: Define both essence and segment bit rate properties + +TAMS should support a wide range of applications as possible. +This option combines [Option 2](#option-2-define-essence-bit-rates) with [Option 3b](#option-3b-define-segment-bit-rates-and-additional-properties) to allow bit rate information to be provided for both essence data and segments. + +The `avg_bit_rate` property is split into `avg_essence_bit_rate` and `avg_segment_bit_rate` properties. +The `max_bit_rate` property is split into `max_essence_bit_rate` and `max_segment_bit_rate` properties. + +Should there be a recommendation to always set `max_segment_bit_rate`? +This corresponds to the required `BANDWIDTH` variant stream property in HLS and required `bandwidth` Representation property in DASH. From 376ee1606e08bd834b5b7225c1b4ec2f4b6ca56d Mon Sep 17 00:00:00 2001 From: Philip de Nier Date: Mon, 11 Nov 2024 12:41:50 +0000 Subject: [PATCH 2/9] Split properties from option 3b to other options This change follows a discussion which decided that segment duration should be added and a maximum 1.5 variance assumed. It was decided that the maximum segment duration was not required. The need to have segment size and maximum segment size requires further input. --- docs/adr/0022-flow-bit-rate-properties.md | 48 +++++++++++++++++------ 1 file changed, 36 insertions(+), 12 deletions(-) diff --git a/docs/adr/0022-flow-bit-rate-properties.md b/docs/adr/0022-flow-bit-rate-properties.md index 0945025..740762e 100644 --- a/docs/adr/0022-flow-bit-rate-properties.md +++ b/docs/adr/0022-flow-bit-rate-properties.md @@ -47,12 +47,15 @@ The SCTE 214-1 specification adds a DASH extension `maxSegmentRate` (note that ` * Option 1: Leave the definition of bit rates to be provided externally. * Option 2: Define essence bit rates. * Option 3a: Define segment bit rates. -* Option 3b: Define segment bit rates with additional properties. +* Option 3b: Define segment bit rates with target segment duration. +* Option 3c: Define segment bit rates with segment size. +* Option 3d: Define segment bit rates with max segment duration. * Option 4: Define both essence and segment bit rate properties. ## Decision Outcome -tbd +Currently favouring "Option 3b", because it allows a receiver to estimate the buffer size using the maximum bit rate and target segment duration. +Option 3c may be chosen if segment size is a requirement. ### Implementation @@ -114,7 +117,7 @@ The value could change over time as segments become available or are removed fro * Bad, the information provided by the Flow is incomplete and doesn't allow the bit rate to be used to decide how much segment data needs to be buffered in the receiver for example * Bad, it doesn't directly provide essence bit rate information for users of TAMS where the essence in other contexts is not transmitted in the same segments as TAMS -### Option 3b: Define segment bit rates and additional properties +### Option 3b: Define segment bit rates with target segment duration In [Option 3a](#option-3a-define-segment-bit-rates) the `max_bit_rate` on it's own doesn't provide sufficient information to calculate the size or duration of the receiver buffer needed to allow for continuous playback (under ideal network conditions). The descriptions and calculations from SCTE 214-1 section 10.3 provides insights to what information could be provided. @@ -129,31 +132,52 @@ The description in SCTE 214-1 section 10.3 includes a formula for calculating th The buffer size is defined as `1.1 * MSR[R] * SDmax`. `MSR[R]` in the TAMS context is `max_bit_rate`, `SDmax` is the maximum segment duration and `1.1` adds a 10% overhead for additional event data. The proposal is to add a `max_segment_duration` (floating point, seconds) property that provides the maximum segment duration (`SDmax`) for the Flow. -The `SDmax` is approximately `min(segment_duration * 1.5, max_segment_duration)` and the minimum buffer size in bytes is then approximately `max_bit_rate * 1000 * SDmax * 8`. -The calculations break down once segments exceed the 1.5 times the `segment_duration` as a segment won't fit into the buffer. -SCTE 214-1 puts limits on the segment size so that they don't exceed the buffer size. +The variation in segment duration is assumed to be between 0.5 and 1.5 the segment_duration. +The `SDmax` is approximately `segment_duration * 1.5` and the minimum buffer size in bytes is then approximately `max_bit_rate * 1000 * SDmax * 8`. Would a property equivalent to `minBufferTime` in DASH be useful as well? +In summary, this option extends [Option 3a](#option-3a-define-segment-bit-rates) with this property: + +* `segment_duration` (rational, seconds): the target segment duration for segmenting the Flow media + +* Good, provides information about the segments that allows the bit rate information to be used to estimate receiver buffer sizes required for continuous playback (not taking network conditions into account) +* Bad, it doesn't directly provide essence bit rate information for users of TAMS where the essence is transmitted by other means + +### Option 3c: Define segment bit rates with segment size + +SCTE 214-1 puts limits on the segment size so that they don't exceed the buffer size. The TAMS doesn't require media to be segmented in a way that is compatible with the restrictions applied to HLS and DASH segments. Some applications my optimise segments for throughput rather than taking latency into account. E.g. a 10 MB segment size may be the optimal segment size for an object store for maximum throughput. The proposal is to add `segment_size` (integer, bytes) and `max_segment_size` (integer, bytes) properties for applications that segment based on size rather than duration. -In summary, this option extends [Option 3a](#option-3a-define-segment-bit-rates) with these properties: +In summary, this option extends [Option 3b](#option-3b-define-segment-bit-rates-with-target-segment-duration) with these properties: -* `segment_duration` (rational, seconds): the target segment duration for segmenting the Flow media -* `max_segment_duration` (floating point, seconds): the maximum segment duration in TAMS for the Flow * `segment_size` (integer, bytes): the target segment size for segmenting the Flow media * `max_segment_size` (integer, bytes): the maximim segment size in TAMS for the Flow -* Good, provides information about the segments that allows the bit rate information to be used to estimate receiver buffer sizes required for continuous playback (not taking network conditions into account) -* Bad, it doesn't directly provide essence bit rate information for users of TAMS where the essence is transmitted by other means +* Good, provides information about the segments optimised for size rather than duration +* Good, provides information about the minimum buffer size required to accommodate a single segment +* Neutral, the properties may not be used + +### Option 3d: Define segment bit rates with max segment duration + +In this option the maximum segment duration is provided explicitly by the `max_segment_duration` property rather than assuming the maximum size variation being `segment_duration * 1.5`. +The `SDmax` is then approximately `min(segment_duration * 1.5, max_segment_duration)` + +In summary, this option extends [Option 3b](#option-3b-define-segment-bit-rates-with-target-segment-duration) with this property: + +* `max_segment_duration` (floating point, seconds): the maximum segment duration in TAMS for the Flow + +* Good, provides information about the segments that allows a more accurate bit rate estimate +* Good, the calculations would not break down once segments exceed the 1.5 times the `segment_duration`. +* Neutral, the property may not be used ### Option 4: Define both essence and segment bit rate properties TAMS should support a wide range of applications as possible. -This option combines [Option 2](#option-2-define-essence-bit-rates) with [Option 3b](#option-3b-define-segment-bit-rates-and-additional-properties) to allow bit rate information to be provided for both essence data and segments. +This option combines [Option 2](#option-2-define-essence-bit-rates) with [Option 3b](#option-3b-define-segment-bit-rates-with-target-segment-duration) to allow bit rate information to be provided for both essence data and segments. The `avg_bit_rate` property is split into `avg_essence_bit_rate` and `avg_segment_bit_rate` properties. The `max_bit_rate` property is split into `max_essence_bit_rate` and `max_segment_bit_rate` properties. From 315e4ed6b975e242a3c9faa9afed247eb56e23ab Mon Sep 17 00:00:00 2001 From: Philip de Nier Date: Fri, 3 Jan 2025 12:22:22 +0000 Subject: [PATCH 3/9] review: add links to referenced container and codec specifications --- docs/adr/0022-flow-bit-rate-properties.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/adr/0022-flow-bit-rate-properties.md b/docs/adr/0022-flow-bit-rate-properties.md index 740762e..52cd1f2 100644 --- a/docs/adr/0022-flow-bit-rate-properties.md +++ b/docs/adr/0022-flow-bit-rate-properties.md @@ -20,26 +20,26 @@ In the most extreme case, a Flow could be stored as a single segment like a regu One interpretation of these properties is that they measure media essence (e.g. H.264 and AAC) bit rates. These properties relate to the essence only but don't account for the container overhead or network conditions. -Essence bit rate information can be carried in the coded bitstream, such as the `hrd_parameters` structure in the H.264 bitstream that specifies the maximum bit rate into the CPB (Coded Picture Buffer) in the HRD (Hypothetical Reference Decoder). +Essence bit rate information can be carried in the coded bitstream, such as the `hrd_parameters` structure in the [H.264](https://www.itu.int/rec/T-REC-H.264) bitstream that specifies the maximum bit rate into the CPB (Coded Picture Buffer) in the HRD (Hypothetical Reference Decoder). Essence bit rate information can also be found in the container. -The MXF AVC (H.264) sub-descriptor defined in SMPTE ST 381-3 includes the `AVC Maximum Bitrate` and `AVC Average Bitrate` properties. -MP4 / Quicktime has a `btrt` box that includes `Max bit rate` and `Average bit rate` properties. -The MPEG-4 Systems (ISO/IEC 14496-1) defines a descriptor containing `maxBitRate`, `avgBitRate` and `bufferSizeDB` properties; this descriptor can be included in the MP4 / Quicktime `esds` box. +The MXF AVC (H.264) sub-descriptor defined in [SMPTE ST 381-3](https://pub.smpte.org/doc/st381-3/) includes the `AVC Maximum Bitrate` and `AVC Average Bitrate` properties. +[MP4](https://www.iso.org/standard/79110.html) / [Quicktime](https://developer.apple.com/documentation/quicktime-file-format) has a [`btrt`](https://developer.apple.com/documentation/quicktime-file-format/bit_rate_atom) box that includes `Max bit rate` and `Average bit rate` properties. +The MPEG-4 Systems ([ISO/IEC 14496-1 Information technology — Coding of audio-visual objects, Part 1: Systems](https://www.iso.org/standard/55688.html)) defines a descriptor containing `maxBitRate`, `avgBitRate` and `bufferSizeDB` properties; this descriptor can be included in the MP4 / [Quicktime `esds`](https://developer.apple.com/documentation/quicktime-file-format/mpeg-4_elementary_stream_descriptor_atom) box. ### Segment Bit Rates Another interpretation of these properties is that they measure media segment (e.g. MPEG-TS and fMP4) bit rates. These properties relate to the segment files that contain the media essence but don't account for network conditions. -The HLS (HTTP Live Streaming) specification includes a required `BANDWIDTH` Variant Stream property that specifies the largest sum of `peak segment bit rate` values. +The HLS ([HTTP Live Streaming](https://datatracker.ietf.org/doc/html/rfc8216)) specification includes a required `BANDWIDTH` Variant Stream property that specifies the largest sum of `peak segment bit rate` values. The `AVERAGE-BANDWIDTH` Variant Stream property specifies the largest sum of `average segment bit rate` values. HLS also defines a required `EXT-X-TARGETDURATION` tag that specifies the target maximum segment duration. -The DASH (Dynamic Adaptive Streaming over HTTP) specification includes a `bandwidth` Representation attribute and `minBufferTime` MPD attribute that together specify the sufficient conditions that allow continuous playback after buffering `minBufferTime` with a constant bit rate channel with bit rate equal to `bandwidth`. +The DASH ([Dynamic adaptive streaming over HTTP](https://www.iso.org/standard/83314.html)) specification includes a `bandwidth` Representation attribute and `minBufferTime` MPD attribute that together specify the sufficient conditions that allow continuous playback after buffering `minBufferTime` with a constant bit rate channel with bit rate equal to `bandwidth`. DASH also specifies a `maxSegmentDuration` MPD attribute that specifies the maximum segment duration in any media Representation. -The SCTE 214-1 (Society of Cable Telecommunications Engineers) "MPEG DASH for IP-Based Cable Services, Part 1: MPD Constraints and Extensions", section 10.3, is a useful resource to understand the DASH and HLS properties and how they can be used to calculate receiver buffer size or buffer times for continuous playback (under ideal network conditions). +The SCTE 214-1 ([Society of Cable Telecommunications Engineers, MPEG DASH for IP-Based Cable Services, Part 1: MPD Constraints and Extensions](https://account.scte.org/standards/library/catalog/scte-214-1-mpeg-dash-for-ip-based-cable-services-part1-mpd-constraints-and-extensions/)), section 10.3, is a useful resource to understand the DASH and HLS properties and how they can be used to calculate receiver buffer size or buffer times for continuous playback (under ideal network conditions). The SCTE 214-1 specification adds a DASH extension `maxSegmentRate` (note that `Rate` is bit rate) Representation attribute to DASH that comparable to the HLS `BANDWIDTH` property (but applies to a single Representation). ## Considered Options From fb95ddae288d094c3ddfc254e39381e3be1b570f Mon Sep 17 00:00:00 2001 From: Philip de Nier Date: Fri, 3 Jan 2025 14:12:45 +0000 Subject: [PATCH 4/9] Add AppNote for setting flow bit rate properties --- docs/README.md | 1 + .../0013-setting-flow-bit-rate-properties.md | 44 +++++++++++++++++++ 2 files changed, 45 insertions(+) create mode 100644 docs/appnotes/0013-setting-flow-bit-rate-properties.md diff --git a/docs/README.md b/docs/README.md index fdb032a..d70f085 100644 --- a/docs/README.md +++ b/docs/README.md @@ -55,3 +55,4 @@ For more information on how we use application notes, see [here](./appnotes/READ | [0010](./appnotes/0010-long-running-sources-and-flows.md) | Long-running Sources and Flows | | [0011](./appnotes/0011-c2pa.md) | C2PA provenance across related Sources and Flows | | [0012](./appnotes/0012-using-flow-segment-timeranges.md) | Using Flow Segment timeranges | +| [0013](./appnotes/0013-setting-flow-bit-rate-properties.md) | Setting Flow bit rate properties | diff --git a/docs/appnotes/0013-setting-flow-bit-rate-properties.md b/docs/appnotes/0013-setting-flow-bit-rate-properties.md new file mode 100644 index 0000000..93d2cb5 --- /dev/null +++ b/docs/appnotes/0013-setting-flow-bit-rate-properties.md @@ -0,0 +1,44 @@ +# Setting Flow Bit Rate Properties + +It was decided in the [Flow Bit Rate Properties](../adr/0022-flow-bit-rate-properties.md#) ADR to follow [option 3b](../adr/0022-flow-bit-rate-properties.md#option-3b-define-segment-bit-rates-with-target-segment-duration). +Option 3b defines the `max_bit_rate` and `avg_bit_rate` Flow properties to be segment bit rates (rather than essence bit rates). +Option 3b adds a `segment_duration` Flow property that is the target duration for the segments. +The `segment_duration` and `max_bit_rate` can be used for calculating a minimum receiver buffer size. + +This AppNote defines how clients can calculate the bit rates if they are not known from the process that is creating them (e.g. an encoder). + +Further background on the definition and calculations can be found in the [Flow Bit Rate Properties](../adr/0022-flow-bit-rate-properties.md#) ADR. + +## Setting Average Bit Rate + +The `avg_bit_rate` is calculated as `int(total_segment_bit_size / (total_segment_duration_sec * 1000))` (kbit/sec) for the segments of the Flow. + +The `avg_bit_rate` could be calculated from the segments that are present in the TAMS. +The value could be an expectation of what could be present in the TAMS (e.g. derived from an encoder / muxer setting for a live ingest). +The value could be an approximation. +The value could change over time as segments becomes available or are removed from TAMS. + +## Setting Maximum Bit Rate + +The `max_bit_rate` is calculated as `int(peak_segment_bit_rate / 1000)`. +The `peak_segment_bit_rate` corresponds to the definition in HLS ([HTTP Live Streaming](https://datatracker.ietf.org/doc/html/rfc8216)), where it is defined as the maximum bit rate for a contiguous sequence of segments that has a duration between 0.5 and 1.5 the target segment duration. +The HLS definition is extended to include single segments above 1.5 the target segment duration to handle Flows that don't follow the HLS retrictions for segment durations, e.g. segmentation optimised for best quality, with variable and potentially large GOP sizes etc.. + +The `max_bit_rate` could be calculated from the segments that are present in the TAMS. +The value could be an expectation of what could be present in the TAMS (e.g. derived from an encoder / muxer setting for a live ingest). +The value could be an approximation, ideally an upper bound (e.g. the bit rate from the H.264 `hrd_parameters` plus some container overhead approximation). +The value could change over time as segments become available or are removed from TAMS. + +## Setting Segment Duration + +The `segment_duration` is the media segment duration that the media segmenter (e.g. encoder) is targetting. + +The segment durations may vary, e.g. to ensure that video segments with variables size GOPs always start with a key frame. + +## Calculating Buffer Size for Continuous Playback + +The description in SCTE 214-1 ([Society of Cable Telecommunications Engineers, MPEG DASH for IP-Based Cable Services, Part 1: MPD Constraints and Extensions](https://account.scte.org/standards/library/catalog/scte-214-1-mpeg-dash-for-ip-based-cable-services-part1-mpd-constraints-and-extensions/)), section 10.3 includes a formula for calculating the buffer size required for continuous playback. +The buffer size is defined as `1.1 * MSR[R] * SDmax`. +`MSR[R]` in the TAMS context is `max_bit_rate`, `SDmax` is the maximum segment duration and `1.1` adds a 10% overhead for additional event data. +The duration of segments is assumed to vary between 0.5 and 1.5 times the `segment_duration`. +The `SDmax` is therefore approximately `segment_duration * 1.5` and the minimum buffer size in bytes is then approximately `max_bit_rate * 1000 * SDmax * 8`. From 90275bd5966e4b584a63f9e7d5d0b56046fc4a9e Mon Sep 17 00:00:00 2001 From: Philip de Nier Date: Fri, 3 Jan 2025 14:37:56 +0000 Subject: [PATCH 5/9] api: Implement flow bit rate property changes --- api/schemas/flow-core.json | 24 +++++++++++++++++++++-- docs/adr/0022-flow-bit-rate-properties.md | 8 +++++--- 2 files changed, 27 insertions(+), 5 deletions(-) diff --git a/api/schemas/flow-core.json b/api/schemas/flow-core.json index 4efc2e4..d5b73af 100644 --- a/api/schemas/flow-core.json +++ b/api/schemas/flow-core.json @@ -75,15 +75,35 @@ "pattern": "^[^\\s\/]+/[^\\s\/]+$" }, "avg_bit_rate": { - "description": "The average bit rate of the flow content in 1000 bits/second. A precise definition of how the bit rate is calculated is not provided here.", + "description": "The average bit rate of the flow segments in 1000 bits/second. A precise definition can be found in the [Setting Flow Bit Rate Properties](https://github.com/bbc/tams/blob/main/docs/appnotes/0013-setting-flow-bit-rate-properties.md) AppNote.", "type": "integer", "minimum": 0 }, "max_bit_rate": { - "description": "The maximum bit rate of the flow content in 1000 bits/second. A precise definition of how the bit rate is calculated is not provided here.", + "description": "The maximum bit rate of the flow segments in 1000 bits/second. A precise definition can be found in the [Setting Flow Bit Rate Properties](https://github.com/bbc/tams/blob/main/docs/appnotes/0013-setting-flow-bit-rate-properties.md) AppNote.", "type": "integer", "minimum": 0 }, + "segment_duration": { + "description": "The target flow segment duration in seconds. The duration for each segment may vary around this target value. See also the [Setting Flow Bit Rate Properties](https://github.com/bbc/tams/blob/main/docs/appnotes/0013-setting-flow-bit-rate-properties.md) AppNote for how this property can be used to calculate buffer sizes.", + "type": "object", + "required": [ + "numerator" + ], + "properties": { + "numerator": { + "description": "numerator", + "type": "integer", + "exclusiveMinimum": 0 + }, + "denominator": { + "description": "denominator", + "type": "integer", + "default": 1, + "exclusiveMinimum": 0 + } + } + }, "timerange": { "description": "The timerange of samples available in the flow, as described by the [TimeRange](../schemas/timerange#top) type", "$ref": "timerange.json" diff --git a/docs/adr/0022-flow-bit-rate-properties.md b/docs/adr/0022-flow-bit-rate-properties.md index 52cd1f2..e7597ab 100644 --- a/docs/adr/0022-flow-bit-rate-properties.md +++ b/docs/adr/0022-flow-bit-rate-properties.md @@ -54,12 +54,14 @@ The SCTE 214-1 specification adds a DASH extension `maxSegmentRate` (note that ` ## Decision Outcome -Currently favouring "Option 3b", because it allows a receiver to estimate the buffer size using the maximum bit rate and target segment duration. -Option 3c may be chosen if segment size is a requirement. +It was decided to select "Option 3b", because it allows a receiver to estimate the buffer size using the maximum bit rate and target segment duration. +Option 3c may be chosen in future if segment size is a requirement. ### Implementation -tbd +Implemented in PR [#86](https://github.com/bbc/tams/pull/86). +The implementation includes a more precise definition for the `max_bit_rate` and `avg_bit_rate` properties and addition of the `segment_duration` property. +The [Setting Flow Bit Rate Properties](../appnotes/0013-setting-flow-bit-rate-properties.md) AppNote defines how to set the properties. ## Pros and Cons of the Options From 003ecaf33708369dec146350f6036a486b40206d Mon Sep 17 00:00:00 2001 From: Philip de Nier Date: Fri, 3 Jan 2025 14:54:16 +0000 Subject: [PATCH 6/9] review: change float to rational for max_segment_duration --- docs/adr/0022-flow-bit-rate-properties.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/adr/0022-flow-bit-rate-properties.md b/docs/adr/0022-flow-bit-rate-properties.md index e7597ab..8d7160f 100644 --- a/docs/adr/0022-flow-bit-rate-properties.md +++ b/docs/adr/0022-flow-bit-rate-properties.md @@ -133,7 +133,7 @@ A rational type allows more precise values to be set (e.g. to match frames rates The description in SCTE 214-1 section 10.3 includes a formula for calculating the buffer size required for continuous playback. The buffer size is defined as `1.1 * MSR[R] * SDmax`. `MSR[R]` in the TAMS context is `max_bit_rate`, `SDmax` is the maximum segment duration and `1.1` adds a 10% overhead for additional event data. -The proposal is to add a `max_segment_duration` (floating point, seconds) property that provides the maximum segment duration (`SDmax`) for the Flow. +The proposal is to add a `max_segment_duration` (rational, seconds) property that provides the maximum segment duration (`SDmax`) for the Flow. The variation in segment duration is assumed to be between 0.5 and 1.5 the segment_duration. The `SDmax` is approximately `segment_duration * 1.5` and the minimum buffer size in bytes is then approximately `max_bit_rate * 1000 * SDmax * 8`. @@ -170,7 +170,7 @@ The `SDmax` is then approximately `min(segment_duration * 1.5, max_segment_durat In summary, this option extends [Option 3b](#option-3b-define-segment-bit-rates-with-target-segment-duration) with this property: -* `max_segment_duration` (floating point, seconds): the maximum segment duration in TAMS for the Flow +* `max_segment_duration` (rational, seconds): the maximum segment duration in TAMS for the Flow * Good, provides information about the segments that allows a more accurate bit rate estimate * Good, the calculations would not break down once segments exceed the 1.5 times the `segment_duration`. From de07161ac247553cbfbd56a2b23e4ac7f09e3e53 Mon Sep 17 00:00:00 2001 From: Philip de Nier Date: Fri, 3 Jan 2025 15:01:36 +0000 Subject: [PATCH 7/9] review: add Good/Bad to option 4 --- docs/adr/0022-flow-bit-rate-properties.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/adr/0022-flow-bit-rate-properties.md b/docs/adr/0022-flow-bit-rate-properties.md index 8d7160f..51c0e07 100644 --- a/docs/adr/0022-flow-bit-rate-properties.md +++ b/docs/adr/0022-flow-bit-rate-properties.md @@ -186,3 +186,6 @@ The `max_bit_rate` property is split into `max_essence_bit_rate` and `max_segmen Should there be a recommendation to always set `max_segment_bit_rate`? This corresponds to the required `BANDWIDTH` variant stream property in HLS and required `bandwidth` Representation property in DASH. + +* Good, supports applications that require either essence or segment bit rate types +* Bad, the additional choice of a bit rate type may harm interoperability or require additional logic in implementations to support all bit rate types From 6e43b9b7723fa8a51af23c11576fad6bd785ff2c Mon Sep 17 00:00:00 2001 From: Philip de Nier Date: Fri, 3 Jan 2025 15:50:03 +0000 Subject: [PATCH 8/9] review: add units to max bit rate formula description in app note --- docs/appnotes/0013-setting-flow-bit-rate-properties.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/appnotes/0013-setting-flow-bit-rate-properties.md b/docs/appnotes/0013-setting-flow-bit-rate-properties.md index 93d2cb5..9d90d04 100644 --- a/docs/appnotes/0013-setting-flow-bit-rate-properties.md +++ b/docs/appnotes/0013-setting-flow-bit-rate-properties.md @@ -20,7 +20,7 @@ The value could change over time as segments becomes available or are removed fr ## Setting Maximum Bit Rate -The `max_bit_rate` is calculated as `int(peak_segment_bit_rate / 1000)`. +The `max_bit_rate` is calculated as `int(peak_segment_bit_rate / 1000)` (kbit/sec) for the segments of the Flow. The `peak_segment_bit_rate` corresponds to the definition in HLS ([HTTP Live Streaming](https://datatracker.ietf.org/doc/html/rfc8216)), where it is defined as the maximum bit rate for a contiguous sequence of segments that has a duration between 0.5 and 1.5 the target segment duration. The HLS definition is extended to include single segments above 1.5 the target segment duration to handle Flows that don't follow the HLS retrictions for segment durations, e.g. segmentation optimised for best quality, with variable and potentially large GOP sizes etc.. From 69479f088ffad5f367a0a0f2be75a87e1a0f0efc Mon Sep 17 00:00:00 2001 From: Philip de Nier Date: Fri, 3 Jan 2025 15:53:33 +0000 Subject: [PATCH 9/9] adr: accepted adr 0022 --- docs/adr/0022-flow-bit-rate-properties.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/adr/0022-flow-bit-rate-properties.md b/docs/adr/0022-flow-bit-rate-properties.md index 51c0e07..61f92d6 100644 --- a/docs/adr/0022-flow-bit-rate-properties.md +++ b/docs/adr/0022-flow-bit-rate-properties.md @@ -1,5 +1,5 @@ --- -status: "proposed" +status: "accepted" --- # Flow Bit Rate Properties