Ad insertion with a CDN

How to configure your CDN for Ad Insertion / Ad Replacement scenarios

The following sections show how to use broadpeak.io with your own content distribution network (CDN) in the context of Dynamic Ad Insertion. For clarity, we use generic terms that may not always apply to the CDN you are operating, but the general principles should remain the same.

📘

Summary Table

The typical rules are as follows, in the recommended order:

Rule forURL patternTarget originOrigin policyCache policySee sections
Manifests*.mpd, *.m3u8stream.broadpeak.ioForward query params & headers

Expose Location header
No caching1
Q.a, Q.b, Q.c, H, R
C.1
SSAT*/bpk-sst/*stream.broadpeak.ioForward query params & headers No caching2.c
Q.a, H, R
C.4
Ad Segments
(if managed by broadpeak.io). Also applies to gap fillers prepared by broadpeak.io
/bpkio-jitt/*stream.broadpeak.ioForward query params & headersCached2.a & 3.a
Q.a, H
C.3
Ad Segments
(if pre-packaged)
defined by your ad originyour ad origindefined by your ad originCached2.b
Q.c
C.2
Gap filler segments
(if not managed by broadpeak.io)
defined by your gap filler originyour gap filler origindefined by your gap filler originCached3.b
Q.c
C.2
Empty subtitles/empty.webvttstream.broadpeak.ioForward query paramsCached4
Q.a
C.3
Media segments-your content origindefined by your content originCached5
Q.c
C.2

Routing rules

You will need to configure your CDN to send various requests (coming from the client player) to the appropriate Origin. Typically you will need to configure the following:

  • Routing rules for manifest requests
  • Routing rule for media segment requests for the ads
  • Routing rule for media segment requests for gap fillers
  • Routing rules for media segment requests for the original content

Note that the order in which these rules are evaluated by the CDN for any incoming request is important.

1. Routing rules for manifest requests

Manifest requests need to be directed to the broadpeak.io platform. To do so, you need to create routing rules (sometimes called behaviors), typically one for each streaming format.

MPEG-DASH

  • Path to match: *.mpd
  • Origin FQDN: stream.broadpeak.io
  • Protocol: HTTPS
  • Caching: disabled (see cache policy C.1 below)
  • Origin policies: query parameters, headers, redirections and locations (see origin policies Q.a, Q.b, Q.c, H and R below)

HLS

  • Path to match: *.m3u8
  • Origin FQDN: stream.broadpeak.io
  • Protocol: HTTPS
  • Caching: disabled (see cache policy C.1 below)
  • Origin policies: query parameters, headers, redirections and locations (see origin policies Q.a, Q.b, Q.c, H and R below)

The rules above assume that your manifest URLs end with the typical filename extension for DASH and HLS manifests. You may need to adjust these if your URLs are different, or have paths that don't end in this way.

You may also want to be more specific in the (partial) path, if you want to avoid using (or cannot use) wildcards.

⚠️

Service hash in the service path

The URL endpoint generated by broadpeak.io always contains a hash string (the "service hash") at the start of the path.
E.g.: https://stream.broadpeak.io/d3d9446802a/myStream/myOutput/index.mpd.

On the first request, broadpeak.io returns a redirection (as an HTTP 307 or 200) in which the location header sends the player to a new path in which the service hash is moved to into a bpkio_serviceid query parameter.
E.g.: HTTP 307 with Location: /myStream/myOutput/index.mpd?**bpkio_serviceid=d3d9446802a**&bpkio_sessionid=xxxxx

When defining your CDN routing rules, it is best practice not to add the hashed value to be matched against, or you would potentially have to create two rules, one for the first request with the hashed value, and a second for the redirection request without the hashed value.

For example, in the example above the path to match would be "/myStream/myOutput/index.mpd".

2. Routing rules for ad segment requests

The rules are different if you the ad creatives are prepared (ie. transcoded and packaged) and originated by broadpeak.io or come already prepared.

2.a. Ads prepared and originated by broadpeak.io

This step is relevant only in the context where a transcoding service has been associated to the DAI service.

In the output manifests generated by broadpeak.io, references to segments of ad creatives transcoded and packaged will be inserted in the appropriate places (either inserted or replacing original segments).
Those segments are retrieved as sub-paths to https://stream.broadpeak.io/bpkio-jitt/*.

Creatives are prepared and originated by broadpeak.io - Programmatic or Direct Sales scenario

You will therefore need to create a routing rule sending all requests with a path starting with /bpkio-jitt/ towards broadpeak.io's streaming FQDN.

  • Path to match: /bpkio-jitt/*
  • Origin FQDN: stream.broadpeak.io
  • Protocol: HTTPS
  • Caching: enabled (see cache policy C.3 below)
  • Origin policies: broadpeak query parameters, headers (see origin policies Q.a and H below)

2.b. Ads prepared and originated by an external system

This step is relevant only in the context where a transcoding service has NOT been associated with the DAI service, or your service uses a mix of creatives already prepared, and some that need transcoding.

Creatives are prepared by an external system - Direct Sales scenario

You need to create a routing rule that sends all ad media segment requests towards the Ad Origin. This is easier if all those ad segments can be identified by a common path pattern:

  • Path to match: any unique and identifiable pattern
  • Origin FQDN: your Ad Origin's FQDN
  • Protocol: HTTPS and/or HTTP (as defined by your Ad Origin)
  • Caching: enabled (see cache policy C.2 below)
  • Origin policies: as defined by your Ad Origin

2.c. Server-side ad tracking (SSAT)

If you have activated the SSAT option in your DAI service configuration, the behaviour of broadpeak.io when it comes to ad segments is slightly different. In the contextualised manifest, the ad segment paths are suffixed with a string that starts with /bpk-sst/. When the client player requests this segment, broadpeak.io returns a redirection towards the actual ad segment. This applies whether the ad creatives are prepared by broadpeak.io or by a 3rd party (as explained in the previous section)


Therefore the CDN should be configured to route the requests to broadpeak.io when this pattern is identified. These requests are answered with an HTTP 307 redirection, which must be returned to the Client by the CDN.

Path(s) to match: */bpk-sst/*

  • Origin FQDN: stream.broadpeak.io
  • Protocol: HTTPS and/or HTTP
  • Caching: disabled (see cache policy C.4 below)
  • Origin policies: broadpeak query parameters, headers, redirections (see origin policies Q.a, H and R below)

Note that it is important that this routing rule is applied before the one that redirects ad media segments to the ad origin (2.a. and/or 2.b. before).

3. Routing rules for gap filler media segment requests

In Live DAI, it is possible to use a gap-filler to pad any remaining duration between the total duration of all ads inserted, and the ad break duration. You can either rely on your own gap-filler or use one that gets prepared (ie. transcoded, packaged and originated) by the platform.

3.a. Gap-filler prepared and originated by broadpeak.io

If the gap filler is a slate (image or video) prepared by broadpeak.io, the gap filler segments will be similar to ad segments, and will be handled with the same rule as 2.a. above.

3.b. Gap-filler prepared and originated by an external system

If you use a gap filler from an asset prepared in the appropriate format by a 3rd party system (such as your own packager), you will need a routing rule to handle those gap filler segments.

If the gap-filler origin is the same as the original content origin, the rule 5 below will be sufficient.

Otherwise, and if the gap-filler origin is a different origin, you must create a new policy:

  • Path(s) to match: any unique and identifiable pattern for the gap-filler
  • Origin FQDN: your gap-filler origin's FQDN
  • Protocol: HTTPS and/or HTTP
  • Caching: enabled (see cache policy C.2 below)

4. Routing rules for dummy subtitle segments

If your HLS source content uses WebVTT subtitles, broadpeak.io will insert references to a dummy WebVTT segment in the subtitle playlists for the duration of the ads, with path /empty.webvtt.

Requests for this segment must be directed to broadpeak.io's streaming origin.

  • Path(s) to match: /empty.webvtt
  • Origin FQDN: stream.broadpeak.io
  • Protocol: HTTPS
  • Caching: enabled (see cache policy C.3 below)
  • Origin policies: broadpeak query parameters, headers (see origin policy Q.a below)

5. Routing rules for media segment requests

Media segment requests need to be directed to the component that originates the content, typically the origin or packager within your ecosystem.

Typically, and in particular if you use a single origin, this rule will be considered as the default rule: if the client player request has been matched against any of the previous rules, this one is applied.

  • Path(s) to match: *
  • Origin FQDN: your own Origin FQDN.
  • Protocol: HTTPS and/or HTTP, according to your Origin configuration.
  • Caching: enabled (or in accordance with your own Origin's rules, see cache policy C.2 below)
  • Origin policies: as defined by your Origin

You may want to be more specific in terms of path pattern to match. This may for example be done by specifying the segment extensions, eg. *.ts, *.fmp4, *.vtt, etc. Or you can be more specific by specifying a more specific (full or partial) path to the expected segment paths. If so, do not forget to add routing policies for all media types: video, audio, subtitles, images, etc.

Origin policies

Q. Query parameters

For broadpeak.io to operate successfully, in all routing rules that send the client requests to stream.broadpeak.io, it is essential that the query parameters are forwarded, and carried all the way through the session.

Q.a. General system query parameters

⚠️ These query parameters must be forwarded on all requests made to stream.broadpeak.io, regardless of the routing rule (rules 1, 2.a, 2.c, 3.a and 4. above)

  • bpkio_serviceid
  • bpkio_sessionid

📘

See Session management section for more information on the role of these query parameters

📘

Note that those query parameters will also appear on all segment requests, including segments sent to your content origin for original segments (See rule 5 above). If your origin is not able to receive (and ignore) those parameters, you may want to configure your CDN to remove them before sending the request to the origin.

Q.b. Query parameters specific to manifest requests

In addition to the previous rule, specific query parameters may be used on manifest requests, which need to be forwarded in the corresponding rules (see rule 1. above)

For all services

  • bpkio_format

For HLS services

  • mm_sp
  • mm_index

For sessions orchestrated by SmartLib

If you use the Broadpeak SmartLib SDK on your client (such as for Client-Side Ad Tracking), additional query parameters will be sent and must be forwarded

  • response
  • bk-ml
  • bk-teardown

Query parameters for ad targeting

In most cases when implementing DAI application, it is required to pass some information from the client to the Ad server through Ad Macros, so targeting can be best achieved. This can be done through the help of query parameters that are being sent together with the streaming request.

These parameters will also all need to be forwarded by the CDN on all requests to manifests.

👍

Recommendation

For simplicity, we would recommend that all incoming query parameters on manifest requests are forwarded to broadpeak.io. Any unrecognised parameter will simply be ignored.

Q.c. Query parameters to forward to the Origin

If your content origin (and / or ad origin) expects certain parameters on manifest or segment requests, those parameters will also need to appear on the request made by the client player, and therefore must also be forwarded by the CDN on the request to broadpeak.io.

H. Headers

As for query parameters, some information carried in the HTTP headers of the client request are passed to the ad server for the purpose of targeting. Mandatory headers are the following:

  • User-Agent
  • X-Forwarded-For, which must contain the client's IP address.

Depending on the specific configuration, additional headers may be expected by the ad server, and should also be forwarded

R. Redirections

As indicated in rules 1. and 2.c above, broadpeak.io responds to the some requests with redirections:

  • Request to the streaming manifest at the beginning of the session (HTTP 307 or HTTP 200 if using the SmartLib SDK)
  • SSAT requests (HTTP 307)

In both cases, the response by broadpeak.io contains a Location header that the player needs to request next. The response must be forwarded to the client by the CDN

Expose Location

If using the SmartLib SDK, you will also need to ensure that the device can access that Location header, which is done by ensuring that there is a Access-Control-Expose-Headers header in the response from the CDN, and that it contains location in its value

Cache policies

C.1. Cache policy for manifest requests

Because content is contextualized down to the individual streaming session, manifests delivered by broadpeak.io to different users will always be different. It is therefore mandatory to disable caching on all manifest requests for your application to properly operate.

C.2. Cache policy for content segments

Media segments for the original content are not originated by broadpeak.io, but by your own origins. Define your own caching policies based on your preferences for all content segment requests.

The same applies to ad segments for ads and/or gap fillers originated from an external system.

C.3. Cache policy for ad and gap filler segments prepared by broadpeak.io

If the ad creatives are prepared by broadpeak.io, you will want to enable caching. Limiting the number of requests to broadpeak.io is important to reduce the egress costs we charge.

It is also recommended to set an expiry time which suits the expected lifetime of Ads.

Cache key

The origin policy O. above stipulates that the request sent to the broadpeak.io streaming origin must carry the bpkio_serviceid and bpkio_sessionid query parameters. However as the latter will always be different, you must ensure that the cache key for those requests includes neither of those query parameters.

For example, let's assume that an HLS manifest includes the following URI to an ad segment:

#EXTINF:6, no desc 
../../../bpkio-jitt/path/to/ad/segment.ts?bpkio_serviceid=abcdef&bpkio_sessionid=12345

📘

Gap fillers

This cache policy also applies to gap filler segments

C.4. Cache policy for ad segments with SSAT

If you have configured SSAT in your DAI service, the URL for ad segments in the manifest contain instructions that trigger the appropriate impressions and trackers in broadpeak.io. It is therefore mandatory that those requests are always sent to broadpeak.io, and are not cached.

You must therefore disable caching in the SSAT rule in your CDN configuration (see rule 2.c. above).


What’s Next

Here are some playbooks on how to do all this with specific CDN products