Understanding broadpeak.io behavior

What does broadpeak.io actually do?

broadpeak.io is a platform built for developers with the purpose to build better, faster and more resilient media applications within the spectrum of content contextualization and personalization.

What is media content contextualization?

A whole set of applications and use-cases can be answered with content contextualization such as Blackout, Simsub, Channel assembly, Ad Insertion, etc., the objective being to adapt the content delivered to given Audiences, (group of viewers), based on contextual information (location, devices, time, etc.). The platform takes live and on-demand feeds and adapts them by replacing or stitching the streams with contextualized content. Contextualization is defined through rules which you can define and have full control over.

Reference timing & synchronization

๐Ÿ“˜

Time synchronization matters!

It is very important that you take time synchronization into account when implementing your application with broadpeak.io, as a de-synchronization of couple of seconds between your programming system, your origin packager and broadpeak.io might not lead to the expected outcome.

broadpeak.io allows external systems such as a scheduling system to create what we call a slot, which is basically a period of time which starts at a known point of time, and which finishes at another known point of time. A slot can also contain additional information such as Audience information (the people that this slot addresses), and some Action information (What it is expected to happen during this period). A slot is also called a MediaPoint under the ESNI specifications.

When scheduling systems define ESNI MediaPoints to create Content Replacement slots, they need to provide a matchTime value - as in the example below. The value of this attribute reflects the time when the slot should start. See below for example with a case of Blackout.

<Media href="MyBlackoutService">
  <MediaPoint id="MyMediaPoint" matchTime="2021-08-18T16:00:00Z" expectedDuration="PT0H1M">
   .....
  </MediaPoint>
</Media>

Note: The xml above is not a fully functional xml body. Parts that are not relevant for the example have been omitted.

This time information is then taken into account by the platform to perform the switch to the indicated alternate channel (if specified) for the right Audience (if specified), at the right time. Indeed, the platform compares the MediaPoint matchTime value with the internal clock to trigger the switch from the original source to the replacement source. It is only when the internal clock has reached the matchTime value, that broadpeak.io initiates the manifest contextualization process.

Note: the platformโ€™s internal clock is synchronized through Amazon Time Sync Service. More information can be found on AWS website.

Period of consideration for Source & Service creation

When creating or updating a Source or a Service, there is a necessary delay for the platform to take into account the information, propagate it, and use it as replacement content. This delay is maximum 15 seconds.
To translate this into the implementation of the scheduling logic, this means that the Source should be created at least 15 seconds before you want to see the program scheduled onto the stream.

Example:

  • If we want to schedule a program to start at 2022-12-20T16:20:00.000Z which relies on Source X, the Source should be created before 2022-12-20T16:05:00.000Z.

Contextualisation process

The system looks for the right place in the original manifest to insert the reference to the replacement content using the actual content PDT that it retrieves or computes directly from the manifest.

๐Ÿšง

Important

Your origin packager and programming system must be synchronized in time with broadpeak.io. A manifest which presents a time that is late in comparison to broadpeak.io internal clock can lead to missed slots, or delayed slot start time.

In HLS, broadpeak.io uses PDT (Program Date Time) as reference timing for the manifest.
In MPEG-DASH, broadpeak.io uses the timing information of the segment timeline specified inside the manifest as reference timing.

Virtual Channel and Content Replacement Applications

Slot precision

This section gives more details on how broadpeak.io handles slot timing precision. The behavior is different based on the type of Application implemented on the platform.

The slot precision is 1 second.

With a 1s precision, this means that if the programming system defines a matchTime with a value in milliseconds, the platform will automatically round the time to the nearest second.

Sometimes, the matchTime value indicated is not exactly between the end time of a media segment and the start time of the next segment. The following sections explain how this scenario is being handled in HLS and MPEG-DASH:

Management of slot start time between two media segments in HLS

HLS Media playlists directly reflect the media segments through the numbering syntax. Each media segment has a duration, which in most cases, is the same for all media segments.
In the event that the matchTime of the MediaPoint is not exactly between the end time of one media segment and the beginning of the next media segment, broadpeak.io identifies the start time of the last segment before the matchTime value, and starts inserting the reference to the media segments of the alternate content there.

Example:
Assuming we have an original content whose media manifest reflects a 4-second media segment. This media manifest shows that the last available video segment starts at 12:00:00 and ends at 12:00:04. Here is how the original manifest would look like:

#EXTM3U                                                                                                              
#EXT-X-VERSION:5
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-MEDIA-SEQUENCE:1
#EXT-X-TARGETDURATION:4
#EXT-X-PROGRAM-DATE-TIME:2022-11-10T11:59:40.000000+00:00
#EXTINF:4, no desc
audio=129117-video=633990-01.ts
#EXTINF:4, no desc
audio=129117-video=633990-02.ts
#EXTINF:4, no desc
audio=129117-video=633990-03.ts
#EXTINF:4, no desc
audio=129117-video=633990-04.ts
#EXTINF:4, no desc
audio=129117-video=633990-05.ts
#EXTINF:4, no desc
audio=129117-video=633990-06.ts

Assuming we also have a programming system which creates a mediapoint with a matchTime at 2022-11-10T12:00:02:456Z.

  1. As the precision of the platform is 1s, the recorded matchTime becomes actually 2022-11-10T12:00:02:000Z.
  2. Also as '2022-11-10T12:00:02:000Z' does not exactly fall at the end of an existing media segment, broadpeak.io then starts inserting reference to media segments of the alternate source at 2022-11-10T12:00:00:000Z. In this case, the latest segment of the original content audio=129117-video=633990-06.ts is replaced by the latest segment available on the manifest of the replacement source.
#EXTM3U                                                                                                              
#EXT-X-VERSION:5
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-MEDIA-SEQUENCE:1
#EXT-X-TARGETDURATION:4
#EXT-X-PROGRAM-DATE-TIME:2022-11-10T11:59:40.000000+00:00
#EXTINF:4, no desc
audio=129117-video=633990-01.ts
#EXTINF:4, no desc
audio=129117-video=633990-02.ts
#EXTINF:4, no desc
audio=129117-video=633990-03.ts
#EXTINF:4, no desc
audio=129117-video=633990-04.ts
#EXTINF:4, no desc
audio=129117-video=633990-05.ts
#EXT-X-DISCONTINUITY
#EXT-X-PROGRAM-DATE-TIME:2022-11-10T12:00:00.000000+00:00
#EXTINF:4, no desc
../../../../replacement_content/hls/audio=129117-video=633990-190.ts

The example above explains how it works with a single variant, but obviously this mechanism is applied to all the existing variants requested by the player.

Management of slot's start time between two media segments in MPEG-DASH

In MPEG-DASH, the slot precision of 1s also applies.

The handling of these scenarios in MPEG-DASH is slightly different, as the protocol allows more flexibility with the multi-period syntax. Indeed, in MPEG-DASH it is possible to represent several periods on which media segments can overlap. In the event that the matchTime of the MediaPoint is not exactly between the end time of one media segment and the beginning of the next media segment, broadpeak.io sends out a MPD which reflects overlapping segments across two different periods.

We are reusing the previous example where the programming system creates a mediapoint with a matchTime at 2022-11-10T12:00:02:456Z.

  1. As for HLS, the precision of the platform is 1s, the recorded matchTime becomes actually 2022-11-10T12:00:02:000Z.
  2. Even though '2022-11-10T12:00:02:000Z' timing does not exactly fall at the end of an existing media segment, broadpeak.io represents the alternate content on an another period at 2022-11-10T12:00:02:000Z, which automatically translates to the end of the previous period at the very same timing.
<MPD xmlns="urn:mpeg:dash:schema:mpd:2011" type="dynamic"> 
  <Period id="1" start="PT1668081000.00S"> <!-- Period 1 -->
    <BaseURL>../../../dash/originalcontent/</BaseURL> <!-- path points to original content -->
    ...
    <AdaptationSet ...>
      <SegmentTemplate timescale="60000" presentationTimeOffset="100084860000000">
        <SegmentTimeline>
          <S t="100084860000000" d="120000" r="299" />
        </SegmentTimeline>
      </SegmentTemplate>
      <Representation id="video=991000" ... />
      <Representation id="video=2971000" ... />
    </AdaptationSet>
  </Period> 
  <Period id="2" start="PT1668081602.00S">  <!-- Period 2 -->
    <BaseURL>../../../dash/replacementcontent</BaseURL> <!-- path points to replacement content -->
    ...
     <AdaptationSet ...>
      <SegmentTemplate timescale="60000" presentationTimeOffset="100084896120000">
        <SegmentTimeline>
          <S t="100084896120000" d="120000" r="5" />
        </SegmentTimeline>
      </SegmentTemplate>
      <Representation id="video=991000" ... />
      <Representation id="video=2971000" ... />
    </AdaptationSet>
  </Period>
</MPD>

Note: The manifest above is not a fully functional MPD. Parts of the MPD structure that are not relevant for the example have been omitted.

In the end, it is up to the client to present the latest media segment of the first period fully or only partially.
As MPDs are not always easy to interpret, here is a graphical representation of the example.

10081008

For more information we invite you to check the following section of the DASH-IF implementation guidelines

Precision of program duration

The duration of Asset Sources is not always an integer number of seconds, but rather a float that is a combination of seconds and milliseconds. In the case of a content-replacement slot, the duration specified in the API call must be rounded down to the second below.

Example: The duration of a program is 600s and 80ms. In this case, the time to be used as the duration value in the Slot is 600 seconds.

Consequences of slot duration inaccuracy

Below is a description of what would happen if the duration defined in the replacement slot did not exactly match the duration of the replacement asset, or live event.

VOD as a replacement source

If the specified duration is shorter than the total duration of the VOD, the VOD will only be inserted on the specified duration. The end of the VOD will then be truncated.

If the specified duration is longer than the total duration of the VOD, the VOD will only be inserted on the specified duration. The VOD will be looped once it reaches the end, and will start again from the beginning until it reaches the duration specified in the slot.

Live as a replacement source

If the specified duration is shorter than the total duration of the Live event, the replacement Live will only be inserted on the specified duration. The end of the event will then be truncated. Using broadpeak.io it is possible to extend the duration of an existing scheduled event by updating the existing slot.

If the specified duration is longer than the total duration of the Live event, the replacement Live will be inserted on the specified duration. This means that the content on the Live source will be inserted, even after the end of the actual program. Using broadpeak.io it is possible to reduce the duration of an existing scheduled event by updating the existing slot.

Scheduling a program in the Past

Creating a replacement slot with an end in the past is now accepted by the API, and will have no effect.
Any replacement slot scheduled in the past with an end in the future will be scheduled and will take effect immediately after the consideration period. The manifest will reference the program where it should be at the time of the streaming.

Ex: A VOD asset of 60 minutes is scheduled at 22/06/2022, 16:00 UTC. The API call is sent at 22/06/2022, 16:30 UTC. When retrieving the manifest at 16:30, the manifest will reference the content of the VOD, 30 minutes, in the VOD content.

Slots consideration and reflection period

When creating or updating a slot, there is a necessary delay for the platform to take into account the information, propagate it and reflect into the manifest. This delay is 15 seconds.
To translate this into the implementation of the scheduling logic, this means that sending API calls to create or update a slot must be done at least 15 seconds before you want to see the change reflected on the stream.

Example:

  • If we want to schedule a program to start at 2022-12-20T16:20:00.000Z, the API call to create the slot must be sent before 2022-12-20T16:05:00.000Z
  • If we want to extend the duration of a program which initially is supposed to end at 2022-12-20T16:47:30.000Z , the API call to update the slot must be sent at least 15 seconds before the end of the initial scheduled event, so at 2022-12-20T16:47:15.000Z. The duration value in this call must be extended to the actual new duration.

Dynamic Ad Insertion Application

Ad Insertion Precision

The precision of the platform when using the Dynamic Ad Insertion Application is 1 ms. This means, that when SCTE-35 markers are being received in Manifests, broadpeak.io will be accurate at the millisecond.

Ad Insertion logic

  1. Without Transcoding involved
    When no transcoding Service is associated to the DAI Service, it is expected that the creatives are transcoded and packaged outside of broadpeak.io. It is also expected that the VAST response will contain m3u8 or mpd Mediafiles.
    In the case where the VAST response does not contain m3u8 or mpd Mediafiles, no insertion will be performed.
  2. With Transcoding involved
    As soon as a transcoding Service is associated to a DAI Service, it is expected that the flow will go through the transcoding and preparation of the creatives through BROADPEAK.IO so they can be delivered in HLS and MPEG-DASH format.

GAP Filler's behavior for live ad replacement

By default gap fillers can only be enabled on Live ad-replacement. It usually makes sense to use them to fill the gap between the total duration of all Ads inserted and the break duration in the events where durations do not match. They tremendously increase the quality of experience by avoiding falling back into the original Ad once back to the original stream.

Below are the expected behaviors when associating a gap-filler to an Ad insertion services.

  1. Default behavior with Gap-filler
    In the case the VAST returned by the ADS contains valid Mediafiles, the gap filler will be used only if the total duration of all creatives is lesser than the Ad break duration.
    In the case the VAST returned by the ADS does not contain valid Mediafiles, the gap-filler will be used to fill the total duration of the break.
  2. Not using a Gap-filler
    In the case the VAST returned by the ADS contains valid Mediafiles, and the the total duration of all creatives is lesser than the Ad break duration, the break will not be completely filled.
    In the case the VAST returned by the ADS does not contain valid Mediafiles, the insertion will not be performed.

๐Ÿ“˜

In HLS, the segment's size of the slate directly impacts its capabilities to fill the break.

Ad Insertion with Subtitles in HLS

When implementing Ad Insertion applications with Sources that contains subtitles or captions, often compatibility issues are met because Ad creative do not contain subtitles tracks. In HLS, if a stream starts with a subtitling variant, and this variant becomes unavailable for the period where the Ad is inserted, most likely it will create issues at the player level.

To avoid this problem, we have implemented a logic with reference to dummy subtitles in broadpeak.io.

broadpeak.io analyses the original VOD stream, and if a subtitle track is available, the Ad inserted in the Manifest will contain reference to dummy segments as per below.

Example of Live Ad replacement:

#EXTM3U 
#EXT-X-VERSION:5 
#EXT-X-INDEPENDENT-SEGMENTS 
#EXT-X-MEDIA-SEQUENCE:0 
#EXT-X-TARGETDURATION:6 
#EXT-X-PROGRAM-DATE-TIME:2011-01-01T00:00:01.000000+00:00 
#EXTINF:4.0001, no desc 
originalstream_12451841_deu=8000-391772911.webvtt 
#EXTINF:4.0001, no desc 
originalstream_12451841_deu=8000-391772912.webvtt 
#EXTINF:4.0001, no desc
originalstream_12451841_deu=8000-391772913.webvtt 
#EXTINF:4.0001, no desc
originalstream_12451841_deu=8000-391772914.webvtt 
#EXT-X-DISCONTINUITY
#EXT-X-PROGRAM-DATE-TIME:2011-01-01T00:00:19.016000+00:00  
#EXTINF:4.0001, no desc
/empty.webvtt
#EXTINF:4.0001, no desc 
/empty.webvtt
#EXTINF:4.0001, no desc 
/empty.webvtt
#EXT-X-DISCONTINUITY 
#EXT-X-PROGRAM-DATE-TIME:2011-01-01T00:00:31.016300+00:00  
#EXTINF:4.0001, no desc 
originalstream_12451841_deu=8000-391772920.webvtt 
#EXTINF:4.0001, no desc 
originalstream_12451841_deu=8000-391772921.webvtt 
#EXTINF:4.0001, no desc 
originalstream_12451841_deu=8000-391772922.webvtt 
#EXTINF:4.0001, no desc 
originalstream_12451841_deu=8000-391772923.webvtt 
#EXTINF:4.0001, no desc 
originalstream_12451841_deu=8000-391772924.webvtt 
#EXTINF:4.0001, no desc 
originalstream_12451841_deu=8000-391772925.webvtt 

๐Ÿšง

Limitations

  • This only applies to webvtt subtitling format. Other subtitling formats are not supported.
  • In Live, the original content must be packaged so the subtitling variant is aligned on media variants at the beginning and the end of the Ad break.
  • In VOD, we often see some systems which generate a single webvtt file which all the subtitling data. This is not a supported use case.
    In the case where webvtt media segment is not aligned with the rest of the media variants, broadpeak.io will not be able to insert dummy webvtt segments.

Matching HLS audio and video variants

๐Ÿ“˜

Matching rules for HLS video tracks

  1. Codecs. broadpeak.io looks at variant CODECS compatibility between original and replacement content. No codec compatibility = no contextualisation. Transcoding profiles are also taken into account in the comparison.
  2. Languages. broadpeak.io looks at LANGUAGE compatibility between original and replacement content. If languages are different, variant with no correspondence in the replacement content are matched with the DEFAULT=YES compatible variant.
  3. Bandwidth. If there is codec compatibility, broadpeak.io looks at the BANDWITDH value, and matches a variant with the variant that has the closest bandwidth value on the replacement content.

HLS players are very sensitive to general stream changes and more specifically to codec changes within the same stream. For this reason, broadpeak.io only performs manifest contextualization if the content sources and the original source are prepared in the same way.

The platform is designed to maintain the best QoE and attempts to match each variant of the original content with the best compatible variant of the replacement content. There is a set of rules for compatibility checks that is described in the sections below.

Matching HLS variants on multiplexed audio and video manifests

  1. Codecs compatibility of multiplexed audio and video variants.

When it comes to performing content replacement, broadpeak.io looks for compatibility between the variants of the original and replacement content. The solution parses all codecs in the existing video variants of the original manifest, and attempts to find a compatible match with the codec in the replacement content manifest.

Codecs value is retrieved from the attribute CODECS= on #EXT-X-STREAM-INF tag as per below:

#EXT-X-STREAM-INF:BANDWIDTH=833000,AVERAGE-BANDWIDTH=758000,CODECS="mp4a.40.2,avc1.4D401F",RESOLUTION=426x240,FRAME-RATE=25,AUDIO="audio-aacl-63",CLOSED-CAPTIONS=NONE
replacementcontent-audio_63340_eng=63200-video=643600.m3u8

Since audio and video are multiplexed, the codec attribute contains two values separated by a coma, which describe the audio and the video codec of the track. The compatibility check is performed on both audio and video codecs, so if a track is compatible with the video codec but not with the audio codec - or vice-versa - it is considered as not compatible.

Any video track that does not have a compatible match leads to a global incompatibility that results in a contextualization error. In this case, the original manifest is returned.
If all video tracks have a compatible match, the next compatibility rule applies.

  1. Language compatibility of multiplexed audio and video variants

In Content Replacement applications, it is common to perform content switches from national or international content to regional content. Therefore, original and replacement content often contain different languages and different number of audio tracks. Here as well, compatibility rules need to be applied.

broadpeak.io looks at the language of the codec-compatible remaining variant, and uses the audio rendition Id that is described in the variant to identify which languages are available for each variant, in case of multiple audios. In the example below, the audio rendition Id is audio-aacl-63.

# AUDIO groups
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio-aacl-63",NAME="English",LANGUAGE="eng",AUTOSELECT=YES,DEFAULT=YES,CHANNELS="2"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio-aacl-63",NAME="Spanish",LANGUAGE="spa",AUTOSELECT=YES,CHANNELS="2",URI="replacementcontent-audio_63350_spa=63200.m3u8"

#EXT-X-STREAM-INF:BANDWIDTH=833000,AVERAGE-BANDWIDTH=758000,CODECS="mp4a.40.2,avc1.4D401F",RESOLUTION=426x240,FRAME-RATE=25,AUDIO="audio-aacl-63",CLOSED-CAPTIONS=NONE
replacementcontent-audio_63340_eng=63200-video=643600.m3u8

The audio rendition describes the language of each Audio variants available in the group with the attribute LANGUAGE; this is the value that broadpeak.io uses to retrieve the language available in the replacement content.

๐Ÿ“˜

Note

broadpeak.io does not maintain an ISO-639-1 to ISO-639-2 mapping table. Please make sure both original and replacement content use the same language code syntax or else it will be considered incompatible.

If there is no exact language match, broadpeak.io attempts to match the audio tracks from the original content to the audio variant which has the DEFAULT=YES attribute within the audio group. Indeed, it is better for a player to have an audio track that is not the same language, than no audio at all - which in most cases would result in an error during playout.

More audio variants on the original content than in the replacement content
Assuming you have a single audio rendition group with two languages in your original content, English (en) and Spanish (sp), and a single audio rendition group with a single audio track in English (en) in your replacement content. Let's assume for the sake of the example that both audio rendition groups are codecs compatible. Graphically, this would look like below:

982982

As there is no Spanish audio track equivalent in the replacement content, broadpeak.io's contextualization mechanism looks for a codec compatible track that has the attribute DEFAULT=YES - in our case it is the English track - and considers it as a match.

Once the media manifest contextualized, the variant with the Spanish audio track would reference English media segment of the replacement content.

982982

Less audio variants on the original content than in the replacement content

When the replacement content has more audio tracks than the original content, broadpeak.io only matches the tracks that are present in the content source manifest. In the event that there is no language match, broadpeak.io uses a codec-compatible audio track with the "DEFAULT=YES" attribute as a match.

If we take a new example, where the original content contains a single audio rendition group with a single language English (en) and the replacement content contains a single audio rendition group with two audio tracks, English (en) and Spanish (sp). Let's assume for the sake of the example that both audio rendition groups are codec-compatible. Graphically, this would look like below:

982982

In this example, since there is no Spanish audio track in the master manifest of the original content, broadpeak.io only matches English audio tracks together. The Spanish track cannot be referenced in the master manifest.

491491
  1. Bandwidth compatibility of multiplexed audio and video variants

Once codec and language compatibility is successfully passed, the list of remaining possibilities for a match should be reduced. broadpeak.io now compares the value of the "bandwidth" attribute of each variant in both manifests.

๐Ÿ“˜

Preserving the best quality of experience

In the process of preserving the best quality of experience, the system looks for the best variant of the replacement content to match variants of the original content. It uses bandwidth which is directly linked to the encoding parameters, and therefore to the resolution. Content encoded with the same parameters and resolution tends to have the same weights over the network.

The variant of the replacement content - which is codec-compatible, which has the smaller difference in bandwidth with the variant of the original content is selected as a match.

Variant bandwidth is retrieved from the attribute BANDWITH= on #EXT-X-STREAM-INF tag as per below:

#EXT-X-STREAM-INF:BANDWIDTH=833000,AVERAGE-BANDWIDTH=758000,CODECS="mp4a.40.2,avc1.4D401F",RESOLUTION=426x240,FRAME-RATE=25,AUDIO="audio-aacl-63",CLOSED-CAPTIONS=NONE
replacementcontent-audio_63340_eng=63200-video=643600.m3u8

Use-case with more variants in the replacement content

In general, it is best to avoid a difference in the number of variants available between the original content manifest and the replacement content manifest. In the event that this happens, for instance replacing an SD channel with a HD channel, the scenario remains viable.
The expected behavior is that broadpeak.io's contextualization mechanism matches all compatible layers of the original content with the compatible replacement layers based on bandwidth, ignoring additional layers of the replacement content that do not have equivalent in the original content. In fact, the variant(s) from the replacement content with the bigger difference in bandwidth is dropped.

If we take the following example where we have two contents, SD as original content and a HD as replacement content . They both shares 3 layers with the same encoding profile.

SD:

  1. Resolution: 448x252 - Codecs: mp4a.40.2,avc1.4D401E - Bandwidth: 688000
  2. Resolution: 768x432 - Codecs: mp4a.40.2,avc1.4D401E - Bandwidth: 1427000
  3. Resolution: 1280x720 - Codecs: mp4a.40.2,avc1.640020 - Bandwidth: 2962000

HD:

  1. Resolution: 448x252 - Codecs: mp4a.40.2,avc1.4D401E - Bandwidth: 688000
  2. Resolution: 768x432 - Codecs: mp4a.40.2,avc1.4D401E - Bandwidth: 1427000
  3. Resolution: 1280x720 - Codecs: mp4a.40.2,avc1.640020 - Bandwidth: 2962000
  4. Resolution: 1280x720 - Codecs: mp4a.40.2,avc1.640020 - Bandwidth: 4884000

The 3 lowest variant of the SD content are matched to the 3 lower variants of the HD content. The highest variant of the replacement content is dropped.

902902

Use-case with less variants in the replacement content

Here as well, it is recommended to avoid this scenario and adapt the original and replacement streams to preserve quality of experience. If this is not possible, here is how broadpeak.io handles it:

broadpeak.io's contextualization mechanism tries to match all existing variants of the original content to the codec compatible variants of the replacement content which have the smaller bandwidth difference. This way, a variant on the replacement content can become a match for several variants of the original content.

If we take the following example where we have two contents, an HD content as original content and an SD content as replacement content. They both shares 3 layers with the same encoding profile.

SD:

  1. Resolution: 448x252 - Codecs: mp4a.40.2,avc1.4D401E - Bandwidth: 688000
  2. Resolution: 768x432 - Codecs: mp4a.40.2,avc1.4D401E - Bandwidth: 1427000
  3. Resolution: 1280x720 - Codecs: mp4a.40.2,avc1.640020 - Bandwidth: 2962000

HD:

  1. Resolution: 448x252 - Codecs: mp4a.40.2,avc1.4D401E - Bandwidth: 688000
  2. Resolution: 768x432 - Codecs: mp4a.40.2,avc1.4D401E - Bandwidth: 1427000
  3. Resolution: 1280x720 - Codecs: mp4a.40.2,avc1.640020 - Bandwidth: 2962000
  4. Resolution: 1280x720 - Codecs: mp4a.40.2,avc1.640020 - Bandwidth: 4884000

With a switch from the HD content to the SD content, the highest variant of the replacement content (720p) becomes a match for both 1080p and 720p variant of the original content.

904904

Now that we have seen the compatibility logic between original content and replacement content of variants that have the audio and the video multiplexed, we can have a look at the use case where audio and video are in separated tracks.

Matching HLS variants on separate audio and video tracks

๐Ÿ“˜

Matching rules for HLS separate audio and video tracks

1.a Video Codecs. broadpeak.io looks at video CODECS compatibility between the video variant of the original and those of the replacement content. No codec compatibility = no contextualization
1.b Video Bandwidth. If there is codec compatibility, broadpeak.io looks at the BANDWIDTH value, and matches a video variant of the original content with the video variant of the replacement content that has the closest bandwidth value.
2.a Audio Codecs. broadpeak.io looks at the audio CODECS compatibility between the original and replacement content. No codec compatibility = no contextualization
2.b Audio Languages. broadpeak.io looks at LANGUAGE compatibility between audio variants of the original content and those of the replacement content. If the languages are different, variants with no correspondence in the replacement content are matched with the DEFAULT=YES compatible variant.

When working with separated audio and video tracks, things are more or less the same as with multiplexed audio and video variants, except that the compatibility check is applied on audio and video separately.

1.a Video Codecs compatibility

broadpeak.io looks for compatibility between the variants of the original and replacement content. The solution parses codecs in the existing video variants of the original manifest, and attempts to find a compatible match with the codec in the replacement content manifest.

Codecs value is retrieved from the attribute CODECS= on #EXT-X-STREAM-INF tag as per below:

#EXT-X-STREAM-INF:BANDWIDTH=833000,AVERAGE-BANDWIDTH=758000,CODECS="mp4a.40.2,avc1.4D401F",RESOLUTION=426x240,FRAME-RATE=25,AUDIO="audio-aacl-63",CLOSED-CAPTIONS=NONE
replacementcontent-video=643600.m3u8

Even when audio and video are separated, the codec attribute often contains two values separated by a coma, which describes the audio and the video codec of the track. The compatibility check is performed on video codecs at this stage.

Any video track that does not have a compatible match leads to a global incompatibility that results in a contextualization error. If all video tracks have a compatible match, the next compatibility rule applies.

1.b Video Bandwidth correspondence

In the process of preserving the quality of experience, the system looks for the best video variant of the content replacement content to match video variants of the original content. It uses bandwidth which is directly linked to the encoding parameters, and therefore to the resolution. Content encoded with the same parameters and resolution tends to have similar weights over the network.

The codec-compatible variant of the replacement content which has the smallest difference in bandwidth with the variant of the original content is selected as a match.

2.a Audio Codecs compatibility

When audio tracks are separated from the video, each audio variant of the original content must have a match on the replacement content. Similarly to the video, the audio codec is retrieved from the video variant #EXT-X-STREAM-INF.

In the example below, the first value before the coma CODECS=\"mp4a.40.2,avc1.4D401F\ is the audio codec, which is valid for all audio tracks contained in the audio rendition group: AUDIO="audio-aacl-63".

#EXT-X-STREAM-INF:BANDWIDTH=833000,AVERAGE-BANDWIDTH=758000,CODECS="mp4a.40.2,avc1.4D401F",RESOLUTION=426x240,FRAME-RATE=25,AUDIO="audio-aacl-63",CLOSED-CAPTIONS=NONE
replacementcontent-video=643600.m3u8

2.b Audio Language compatibility

Audio language compatibility process for separated video and audio tracks is the same that for multiplexed audio and video variants. We invite you to consult the following section Understanding broadpeak.io behavior

If there is no exact language match, broadpeak.io attempts to match the audio tracks from the original content to the audio variant which has the DEFAULT=YES attribute within the audio group.

๐Ÿ“˜

Note

broadpeak.io does not maintain an ISO-639-1 to ISO-639-2 mapping table. Please make sure both original and replacement content use the same language code syntax or else it will be considered incompatible.

Matching HLS Subtitle variants

Similarly to what we have with audio languages, subtitles languages also sometimes differs between original content and replacement in Content Replacement applications. broadpeak.io performs compatibility checks in order to make sure formats are compatible and languages are properly matched.

  1. Format compatibility
    broadpeak.io does not perform any format checks. It is assumed that the subtitles used are webvtt on the original and replacement content.

  2. Language compatibility
    broadpeak.io looks for the language of the subtitle variant(s) directly in the subtitle rendition group available for the audio variant. In the example below, the subtitle rendition id is textstream.

# AUDIO groups
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio-aacl-63",NAME="English",LANGUAGE="eng",AUTOSELECT=YES,DEFAULT=YES,CHANNELS="2"

# SUBTITLES groups
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="textstream",NAME="English",LANGUAGE="eng",AUTOSELECT=YES,DEFAULT=YES,URI="replacementcontent-textstream_7208961_eng=8000.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="textstream",NAME="Spanish",LANGUAGE="spa",AUTOSELECT=YES,URI="replacementcontent-textstream_12451841_spa=8000.m3u8"

# variants
#EXT-X-STREAM-INF:BANDWIDTH=833000,AVERAGE-BANDWIDTH=758000,CODECS="mp4a.40.2,avc1.4D401F",RESOLUTION=426x240,FRAME-RATE=25,AUDIO="audio-aacl-63",SUBTITLES="textstream",CLOSED-CAPTIONS=NONE
ARTE-audio_63340_eng=63200-video=643600.m3u8

# variants
#EXT-X-STREAM-INF:BANDWIDTH=83000,AVERAGE-BANDWIDTH=75000,CODECS="mp4a.40.2",AUDIO="audio-aacl-63",SUBTITLES="textstream"
ARTE-audio_63340_eng=63200.m3u8

The subtitle rendition group describes the encoding parameters of each subtitles variant available in the rendition group, including the language stated in the LANGUAGE attribute . The language value retrieved in the original content is compared to the language value retrieved in the replacement content to find a match for each subtitle variant.

If there is no match found, broadpeak.io's contextualization mechanism looks for the DEFAULT=YES subtitle track to perform a match. Therefore, languages might not be the same, but the playout is preserved.

Matching HLS Key Frames

The same logic as for Video variant compatibility check is applied when working with keyframes. Since they do not have audio, languages compatibility checks are being bypassed.

    1. Codecs need to be compatible between the keyframe variants of the original and those of the replacement content.
    1. Keyframe variants are matched based on the smallest bandwidth difference.

Referencing contextualised content

When all video, audio, subtitles and keyframes variants have been associated with a match, broadpeak.io can now manipulate manifests to reference the contextualised content(s). Original media references inside the master manifest are not modified - though some query parameters are added in the variant path.
Content replacement is being referenced directly into the media manifest of each variant, according to timing information defined by the slots.

Example of media manifest with a multiplexed audio and video variant

#EXTM3U                                                                                                              
#EXT-X-VERSION:5
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-MEDIA-SEQUENCE:1
#EXT-X-TARGETDURATION:4
#EXT-X-PROGRAM-DATE-TIME:2022-11-10T11:59:40.000000+00:00
#EXTINF:4, no desc
originalcontent-audio=129117-video=633990-01.ts
#EXTINF:4, no desc
originalcontent-audio=129117-video=633990-02.ts
#EXTINF:4, no desc
originalcontent-audio=129117-video=633990-03.ts
#EXTINF:4, no desc
originalcontent-audio=129117-video=633990-04.ts
#EXTINF:4, no desc
audio=129117-video=633990-05.ts
#EXT-X-DISCONTINUITY
#EXT-X-PROGRAM-DATE-TIME:2022-11-10T12:00:00.000000+00:00
#EXTINF:4, no desc
../../../../replacement_content/hls/replacementcontent-audio=129117-video=633990-190.ts

Example of a media manifest with a separate audio

# English Audio track

#EXTM3U                                                                                                              
#EXT-X-VERSION:5
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-MEDIA-SEQUENCE:1
#EXT-X-TARGETDURATION:4
#EXT-X-PROGRAM-DATE-TIME:2022-11-10T11:59:40.000000+00:00
#EXTINF:4, no desc
originalcontent-audio_eng=93375-01.aac
#EXTINF:4, no desc
originalcontent-audio_eng=93375_02.aac
#EXTINF:4, no desc
originalcontent-audio_eng=93375_03.aac
#EXTINF:4, no desc
originalcontent-audio_eng=93375_04.aac
#EXTINF:4, no desc
originalcontent-audio_eng=93375_05.aac
#EXT-X-DISCONTINUITY
#EXT-X-PROGRAM-DATE-TIME:2022-11-10T12:00:00.000000+00:00
#EXTINF:4, no desc
../../../../replacement_content/hls/replacementcontent-audio_eng=93375_190.aac

Example of a media manifest with a separate video

#EXTM3U                                                                                                              
#EXT-X-VERSION:5
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-MEDIA-SEQUENCE:1
#EXT-X-TARGETDURATION:4
#EXT-X-PROGRAM-DATE-TIME:2022-11-10T11:59:40.000000+00:00
#EXTINF:4, no desc
originalcontent-video=633990-01.ts
#EXTINF:4, no desc
originalcontent-video=633990-02.ts
#EXTINF:4, no desc
originalcontent-video=633990-03.ts
#EXTINF:4, no desc
originalcontent-video=633990-04.ts
#EXTINF:4, no desc
originalcontent-video=633990-05.ts
#EXT-X-DISCONTINUITY
#EXT-X-PROGRAM-DATE-TIME:2022-11-10T12:00:00.000000+00:00
#EXTINF:4, no desc
../../../../replacement_content/hls/originalcontent-video=633990-190.ts

Example of a media manifest with a separate audio which matched a DEFAULT=YES english track

# Spanish Audio track

#EXTM3U                                                                                                              
#EXT-X-VERSION:5
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-MEDIA-SEQUENCE:1
#EXT-X-TARGETDURATION:4
#EXT-X-PROGRAM-DATE-TIME:2022-11-10T11:59:40.000000+00:00
#EXTINF:4, no desc
originalcontent-audio_spa=93375-01.aac
#EXTINF:4, no desc
originalcontent-audio_spa=93375_02.aac
#EXTINF:4, no desc
originalcontent-audio_spa=93375_03.aac
#EXTINF:4, no desc
originalcontent-audio_spa=93375_04.aac
#EXTINF:4, no desc
originalcontent-audio_spa=93375_05.aac
#EXT-X-DISCONTINUITY
#EXT-X-PROGRAM-DATE-TIME:2022-11-10T12:00:00.000000+00:00
#EXTINF:4, no desc
../../../../replacement_content/hls/replacementcontent-audio_eng=93375_190.aac

Contextualisation failure

By default, incompatible original content and replacement content prevent the contextualisation process to succeed, resulting in the content in the original manifest not being replaced.

Below is a list of event which can lead to a contextualisation failure:

An incompatibility is found between the original and replacement content. This means that a video, audio, keyframe or subtitle variant of the original content does not have a compatible match in the replacement content. This may be due to codec incompatibilities or a languages incompatibilities between the variant(s) of the two manifests:

  • There is a codec incompatibility on either the video or audio between the manifests of the original and replacement content. In the event of multiplexed audio and audio variant, both audio and audio codecs of the variant must match in original and replacement manifests. When using separate audio and audio variants, codecs of the video variant must match between manifests, and audio codecs of the audio group rendition attached to the video variant must also match.
  • There is a language incompatibility on audio between the manifests of the original and replacement content. The language must match in both the original and replacement master manifests, or the first codec-compatible default track set in the replacement content manifest.
  • The original and replacement sources do not use the same audio or subtitle language code (ISO-639-1 vs ISO-639-2)
  • The VAST response does not contain compatible creative. If you are using a transcoding Service, Mediafiles shall be MP4 or Mezz type. Without transcoding Service, it is expected that the Mediafiles shall reference m3u8 or mpd files.

If you have any issue that persists and the suggestions above do not work, we invite you to validate the stream requirements in the related section: Input Streaming Formats and to check the troubleshooting section Troubleshooting .

Contact us at [email protected] if the issue persists and the documentation does not help.

Working with Ad server Tags

When defining an Ad Server as Source in broadpeak.io, it is necessary to set an Ad Tag which will allow the system to interrogate the Ad Server . The Ad tag contains what we call Macros which are placeholders that broadpeak.io will replace with real values when requesting the Ad server.

The Tag and the necessary Macros shall be provided by the Ad Technology partner that acts as Ad server or SSP.

Using Macros

When defined an Ad Server as source, you are given the possibility to enter the Ad tag coming from the Ad Technology provider of your choice. You can enter an URL and some query parameters also called Macros.

Part of the concept of targeting require the Ad Technology provider to retrieve dynamic values so personalisation can be best achieved. This is done through these Macros.

System values

Some values can be retrieved from the system directly:
- Cachebuster value: $_MMVAR_CORRELATOR
- Live break Duration in seconds: ${_MMVAR_LIVEAR_SLOTDURATION}
- Live break Duration in ms: ${_MMVAR_LIVEAR_SLOTDURATION}000
- Signal ID:

Example:

?max_pod_duration=${_MMVAR_LIVEAR_SLOTDURATION}&cb=$_MMVAR_CORRELATOR

Or

?pmxd=${_MMVAR_LIVEAR_SLOTDURATION}000&cb=$_MMVAR_CORRELATOR

Values from the Client Request

Some values can be directly retrieved from the client/CDN request query parameters or headers.

Headers values

By default, the following headers are forwarded from the Client/CDN request to the Ad Server:

  • X-Forwarded-For: The client Ip can be retrieved from here if the CDN is configured to pass it over.
  • User-Agent:

Header value mapping to Query parameters

When defining the ADS configuration (Source->ADS), it is possible to map the information received in the Client/CDN request as header to Macro values into the defined ADS tag.

To retrieve "value" of the header "header" sent to broadpeak.io in the client/CDN request, you need to use the following variable scheme: $http_header

Example: we need to retrieve the value of the header "user-agent" in the client/CDN request and pass it to the Ad server under the Macro "user-agent". The ADS Tag configuration should contain the following:

?user-agent=$http_User-Agent

Query parameters

When defining the Ad Server configuration, it is possible to map the information received in the Client/CDN request as query param to Macro values into the defined ADS tag.

Query parameter mapping to Query parameters

To retrieve "value" of the parameter "key" sent to broadpeak.io in the client/CDN request, you need to use the following variable scheme:
$arg_key

Example: we need to retrieve the value of the query param "user" in the client/CDN request and pass it to the Ad server under the query param "user_id". The ADS Tag configuration should contain the following macro:

user_id=$arg_user

Ad Reporting

Monetisation only happens when Impression are tracked.

Client Side tracking flow

Tracking on broadpeak.io is supported through a proprietary library which enable unified tracking for HLS and DASH on the client. The library is developed in numerous languages for different platform and allow:

  • Impression tracking (start, impression, 25%, 50%, 75%, complete)
  • provide ad-skipping policies if defined at the campaign level
  • provide the number of Ad per break
  • provide Ad duration

This Library must be implemented by the client application and is already integrated with major players including native players.

Server Side tracking flow

Server-side tracking is not available for now. Stay tuned for a new release soon!

Transcoding logic

Ad creative transcoding is activated by default in broadpeak.io when a transcoding Service is associated to the Ad Insertion Service.

If the VAST response contains a MediaFile of type "mezz" or "mp4" the transcoding flow will be triggered. If the VAST response does contain other MediaFile, the transcoding flow will not be triggered.

The insertion will not be performed in the case where the VAST answer does not contain any compatible MediaFile.

Creating a Transcoding Service in broadpeak.io

Transcoding Services definition

It is not possible to define a transcoding Service on your own at this stage. We know it is not ideal but you have to get in touch with us to define the Service. Please reach out to us via you direct CSM contact or via the live chat.

Transcoding consideration period

The first time a creative is returned in the VAST response under a compatible MediaFile, a transcoded job is created to prepare the creative for HLS or DASH delivery. Depending on the transcoding profiles expected and the duration of the Creative, this process may take more or less time.

The creative will only be inserted into a Manifest when the transcoded jobs is over, and ready to be packaged for HLS or DASH delivery through JITP.

Transcoded Ad retention

By default, transcoded creative are stored 1 month in broadpeak.io.
Origin HTTP Cache-Control header is set to max-age 3600 seconds.

Supported Containers & Codecs

Video Codecs

  • H.264 / AVC1 (ISO/IEC 14496-10)
  • H.265 / HEVC (ISO/IEC 23008-2)

Video Containers

  • MP4 (fmp4) (DASH)
  • TS (HLS)

Audio Codecs

  • AAC / MPEG 4-AAC (LC, HE) (ISO/IEC 14496-3)

Context management

In Live streaming applications, we usually observe that players request new manifests every three media segments. When the packager updates the manifest, it must keep consistency with the previous manifest so that the player can keep track of the media that is added.

Performing manifest manipulation on Live streams requires keeping the same level of consistency, and therefore being able to manage contextual information for each streaming session started by a player.

Session ID

To identify a session, broadpeak.io relies on a unique parameter called the sessionid. This is a unique value generated by the platform on the first request coming from the player, and which identifies a unique session. The system keeps contextual information related to this sessionid, so all the following manifest requests from the player must contain the generated sessionid.

A manifest request without sessionid will be interpreted as a new session with no previous context, and will be assigned a sessionid value.

Query Param

By default, the system will generate a sessionid value for all manifest requests that come without the sessionid query parameter or query parameter value. It will then respond with a HTTP 307 with a value in the location header being the concatenation of the manifest path of the initial request, plus the sessionid query parameters, the generated value and some other system compulsory query params.

curl -i -L https://stream.broadpeak.io/d3d9446802a/myStream/myOutput/index.mpd?zipcode=25267
HTTP/2 307
location: /myStream/myOutput/index.mpd?zipcode=25267&serviceid=d3d9446802a&sessionid=10d0e55a246-f8c09588-c798-4890-a4b7-54155d02b742

The figure below shows a sequence diagram of session creation between the client (player), the CDN and broadpeak.io. To simplify, we are using building blocks, but a CDN might be built with several nodes.

๐Ÿ“˜

Note

In addition to the sessionid query param, the system also adds a serviceid query param which takes the value which is immediately after the FQDN of the request. That value is also removed from the path.

HTTP GET https:/stream.broadpeak.io/d3d9446802a/myStream/myOutput/index.mpd?zipcode=25267
< HTTP 307 Location: /myStream/myOutput/index.mpd?zipcode=25267&serviceid=d3d9446802a&sessionid=x

Header

In the event you use Broadpeak on-premise CDN, it is possible to rely on the CDN functionality which generates the sessionid itself, and pass it to broadpeak.io in the request via specific headers. In this scenario, broadpeak.io will not generate the sessionid value and will respond with a an HTTP 200 and the contextualized manifest directly in the body. This scenario is not enabled by default.

Contact us directly or your Broadpeak representative for more information.


Did this page help you?