Ad insertion with a CDN
How to configure your CDN for Ad Insertion / Ad Replacement scenarios
The following sections show how to use broadpeak.io with your own content distribution network (CDN) in the context of Dynamic Ad Insertion. For clarity, we use generic terms that may not always apply to the CDN you are operating, but the general principles should remain the same.
Summary Table
The typical rules are as follows, in the recommended order:
Rule for URL pattern Target origin Origin policy Cache policy See sections Manifests *.mpd, *.m3u8 stream.broadpeak.io Forward query params & headers
ExposeLocation
headerNo caching 1
Q.a, Q.b, Q.c, H, R
C.1SSAT */bpk-sst/* stream.broadpeak.io Forward query params & headers No caching 2.c
Q.a, H, R
C.4Ad Segments
(if managed by broadpeak.io). Also applies to gap fillers prepared by broadpeak.io/bpkio-jitt/* stream.broadpeak.io Forward query params & headers Cached 2.a & 3.a
Q.a, H
C.3Ad Segments
(if pre-packaged)defined by your ad origin your ad origin defined by your ad origin Cached 2.b
Q.c
C.2Gap filler segments
(if not managed by broadpeak.io)defined by your gap filler origin your gap filler origin defined by your gap filler origin Cached 3.b
Q.c
C.2Empty subtitles /empty.webvtt stream.broadpeak.io Forward query params Cached 4
Q.a
C.3Media segments - your content origin defined by your content origin Cached 5
Q.c
C.2
Routing rules
You will need to configure your CDN to send various requests (coming from the client player) to the appropriate Origin. Typically you will need to configure the following:
- Routing rules for manifest requests
- Routing rule for media segment requests for the ads
- Routing rule for media segment requests for gap fillers
- Routing rules for media segment requests for the original content
Note that the order in which these rules are evaluated by the CDN for any incoming request is important.
1. Routing rules for manifest requests
Manifest requests need to be directed to the broadpeak.io platform. To do so, you need to create routing rules (sometimes called behaviors), typically one for each streaming format.
MPEG-DASH
- Path to match:
*.mpd
- Origin FQDN:
stream.broadpeak.io
- Protocol: HTTPS
- Caching: disabled (see cache policy C.1 below)
- Origin policies: query parameters, headers, redirections and locations (see origin policies Q.a, Q.b, Q.c, H and R below)
HLS
- Path to match:
*.m3u8
- Origin FQDN:
stream.broadpeak.io
- Protocol: HTTPS
- Caching: disabled (see cache policy C.1 below)
- Origin policies: query parameters, headers, redirections and locations (see origin policies Q.a, Q.b, Q.c, H and R below)
The rules above assume that your manifest URLs end with the typical filename extension for DASH and HLS manifests. You may need to adjust these if your URLs are different, or have paths that don't end in this way.
You may also want to be more specific in the (partial) path, if you want to avoid using (or cannot use) wildcards.
Service hash in the service path
The URL endpoint generated by broadpeak.io always contains a hash string (the "service hash") at the start of the path.
E.g.: https://stream.broadpeak.io/d3d9446802a/myStream/myOutput/index.mpd.On the first request, broadpeak.io returns a redirection (as an HTTP 307 or 200) in which the location header sends the player to a new path in which the service hash is moved to into a
bpkio_serviceid
query parameter.
E.g.: HTTP 307 withLocation: /myStream/myOutput/index.mpd?**bpkio_serviceid=d3d9446802a**&bpkio_sessionid=xxxxx
When defining your CDN routing rules, it is best practice not to add the hashed value to be matched against, or you would potentially have to create two rules, one for the first request with the hashed value, and a second for the redirection request without the hashed value.
For example, in the example above the path to match would be "/myStream/myOutput/index.mpd".
2. Routing rules for ad segment requests
The rules are different if you the ad creatives are prepared (ie. transcoded and packaged) and originated by broadpeak.io or come already prepared.
2.a. Ads prepared and originated by broadpeak.io
This step is relevant only in the context where a transcoding service has been associated to the DAI service.
In the output manifests generated by broadpeak.io, references to segments of ad creatives transcoded and packaged will be inserted in the appropriate places (either inserted or replacing original segments).
Those segments are retrieved as sub-paths to https://stream.broadpeak.io/bpkio-jitt/*.
You will therefore need to create a routing rule sending all requests with a path starting with /bpkio-jitt/
towards broadpeak.io's streaming FQDN.
- Path to match:
/bpkio-jitt/*
- Origin FQDN:
stream.broadpeak.io
- Protocol: HTTPS
- Caching: enabled (see cache policy C.3 below)
- Origin policies: broadpeak query parameters, headers (see origin policies Q.a and H below)
2.b. Ads prepared and originated by an external system
This step is relevant only in the context where a transcoding service has NOT been associated with the DAI service, or your service uses a mix of creatives already prepared, and some that need transcoding.
You need to create a routing rule that sends all ad media segment requests towards the Ad Origin. This is easier if all those ad segments can be identified by a common path pattern:
- Path to match: any unique and identifiable pattern
- Origin FQDN: your Ad Origin's FQDN
- Protocol: HTTPS and/or HTTP (as defined by your Ad Origin)
- Caching: enabled (see cache policy C.2 below)
- Origin policies: as defined by your Ad Origin
2.c. Server-side ad tracking (SSAT)
If you have activated the SSAT option in your DAI service configuration, the behaviour of broadpeak.io when it comes to ad segments is slightly different. In the contextualised manifest, the ad segment paths are suffixed with a string that starts with /bpk-sst/
. When the client player requests this segment, broadpeak.io returns a redirection towards the actual ad segment. This applies whether the ad creatives are prepared by broadpeak.io or by a 3rd party (as explained in the previous section)
Therefore the CDN should be configured to route the requests to broadpeak.io when this pattern is identified. These requests are answered with an HTTP 307 redirection, which must be returned to the Client by the CDN.
Path(s) to match: */bpk-sst/*
- Origin FQDN:
stream.broadpeak.io
- Protocol: HTTPS and/or HTTP
- Caching: disabled (see cache policy C.4 below)
- Origin policies: broadpeak query parameters, headers, redirections (see origin policies Q.a, H and R below)
Note that it is important that this routing rule is applied before the one that redirects ad media segments to the ad origin (2.a. and/or 2.b. before).
3. Routing rules for gap filler media segment requests
In Live DAI, it is possible to use a gap-filler to pad any remaining duration between the total duration of all ads inserted, and the ad break duration. You can either rely on your own gap-filler or use one that gets prepared (ie. transcoded, packaged and originated) by the platform.
3.a. Gap-filler prepared and originated by broadpeak.io
If the gap filler is a slate (image or video) prepared by broadpeak.io, the gap filler segments will be similar to ad segments, and will be handled with the same rule as 2.a. above.
3.b. Gap-filler prepared and originated by an external system
If you use a gap filler from an asset prepared in the appropriate format by a 3rd party system (such as your own packager), you will need a routing rule to handle those gap filler segments.
If the gap-filler origin is the same as the original content origin, the rule 5 below will be sufficient.
Otherwise, and if the gap-filler origin is a different origin, you must create a new policy:
- Path(s) to match: any unique and identifiable pattern for the gap-filler
- Origin FQDN: your gap-filler origin's FQDN
- Protocol: HTTPS and/or HTTP
- Caching: enabled (see cache policy C.2 below)
4. Routing rules for dummy subtitle segments
If your HLS source content uses WebVTT subtitles, broadpeak.io will insert references to a dummy WebVTT segment in the subtitle playlists for the duration of the ads, with path /empty.webvtt
.
Requests for this segment must be directed to broadpeak.io's streaming origin.
- Path(s) to match:
/empty.webvtt
- Origin FQDN:
stream.broadpeak.io
- Protocol: HTTPS
- Caching: enabled (see cache policy C.3 below)
- Origin policies: broadpeak query parameters, headers (see origin policy Q.a below)
5. Routing rules for media segment requests
Media segment requests need to be directed to the component that originates the content, typically the origin or packager within your ecosystem.
Typically, and in particular if you use a single origin, this rule will be considered as the default rule: if the client player request has been matched against any of the previous rules, this one is applied.
- Path(s) to match:
*
- Origin FQDN: your own Origin FQDN.
- Protocol: HTTPS and/or HTTP, according to your Origin configuration.
- Caching: enabled (or in accordance with your own Origin's rules, see cache policy C.2 below)
- Origin policies: as defined by your Origin
You may want to be more specific in terms of path pattern to match. This may for example be done by specifying the segment extensions, eg. *.ts
, *.fmp4
, *.vtt
, etc. Or you can be more specific by specifying a more specific (full or partial) path to the expected segment paths. If so, do not forget to add routing policies for all media types: video, audio, subtitles, images, etc.
Origin policies
Q. Query parameters
For broadpeak.io to operate successfully, in all routing rules that send the client requests to stream.broadpeak.io
, it is essential that the query parameters are forwarded, and carried all the way through the session.
Q.a. General system query parameters
⚠️ These query parameters must be forwarded on all requests made to stream.broadpeak.io
, regardless of the routing rule (rules 1, 2.a, 2.c, 3.a and 4. above)
bpkio_serviceid
bpkio_sessionid
See Session management section for more information on the role of these query parameters
Note that those query parameters will also appear on all segment requests, including segments sent to your content origin for original segments (See rule 5 above). If your origin is not able to receive (and ignore) those parameters, you may want to configure your CDN to remove them before sending the request to the origin.
Q.b. Query parameters specific to manifest requests
In addition to the previous rule, specific query parameters may be used on manifest requests, which need to be forwarded in the corresponding rules (see rule 1. above)
For all services
bpkio_format
For HLS services
mm_sp
mm_index
For sessions orchestrated by SmartLib
If you use the Broadpeak SmartLib SDK on your client (such as for Client-Side Ad Tracking), additional query parameters will be sent and must be forwarded
response
bk-ml
bk-teardown
Query parameters for ad targeting
In most cases when implementing DAI application, it is required to pass some information from the client to the Ad server through Ad Macros, so targeting can be best achieved. This can be done through the help of query parameters that are being sent together with the streaming request.
These parameters will also all need to be forwarded by the CDN on all requests to manifests.
Recommendation
For simplicity, we would recommend that all incoming query parameters on manifest requests are forwarded to broadpeak.io. Any unrecognised parameter will simply be ignored.
Q.c. Query parameters to forward to the Origin
If your content origin (and / or ad origin) expects certain parameters on manifest or segment requests, those parameters will also need to appear on the request made by the client player, and therefore must also be forwarded by the CDN on the request to broadpeak.io.
H. Headers
As for query parameters, some information carried in the HTTP headers of the client request are passed to the ad server for the purpose of targeting. Mandatory headers are the following:
User-Agent
X-Forwarded-For
, which must contain the client's IP address.
Depending on the specific configuration, additional headers may be expected by the ad server, and should also be forwarded
R. Redirections
As indicated in rules 1. and 2.c above, broadpeak.io responds to the some requests with redirections:
- Request to the streaming manifest at the beginning of the session (HTTP 307 or HTTP 200 if using the SmartLib SDK)
- SSAT requests (HTTP 307)
In both cases, the response by broadpeak.io contains a Location
header that the player needs to request next. The response must be forwarded to the client by the CDN
Expose Location
If using the SmartLib SDK, you will also need to ensure that the device can access that Location
header, which is done by ensuring that there is a Access-Control-Expose-Headers
header in the response from the CDN, and that it contains location
in its value
Cache policies
C.1. Cache policy for manifest requests
Because content is contextualized down to the individual streaming session, manifests delivered by broadpeak.io to different users will always be different. It is therefore mandatory to disable caching on all manifest requests for your application to properly operate.
C.2. Cache policy for content segments
Media segments for the original content are not originated by broadpeak.io, but by your own origins. Define your own caching policies based on your preferences for all content segment requests.
The same applies to ad segments for ads and/or gap fillers originated from an external system.
C.3. Cache policy for ad and gap filler segments prepared by broadpeak.io
If the ad creatives are prepared by broadpeak.io, you will want to enable caching. Limiting the number of requests to broadpeak.io is important to reduce the egress costs we charge.
It is also recommended to set an expiry time which suits the expected lifetime of Ads.
Cache key
The origin policy O. above stipulates that the request sent to the broadpeak.io streaming origin must carry the bpkio_serviceid
and bpkio_sessionid
query parameters. However as the latter will always be different, you must ensure that the cache key for those requests includes neither of those query parameters.
For example, let's assume that an HLS manifest includes the following URI to an ad segment:
#EXTINF:6, no desc
../../../bpkio-jitt/path/to/ad/segment.ts?bpkio_serviceid=abcdef&bpkio_sessionid=12345
- The player will therefore make a call to https://your-cdn-fqdn.com/bpkio-jitt/path/to/ad/segment.ts?bpkio_serviceid=abcdef&bpkio_sessionid=12345
- The CDN needs to use the path only as its cache key: /bpkio-jitt/path/to/ad/segment.ts
- If there is a "hit" (ie. this segment has been cached before), it serves the content of the cache
- Otherwise, it must make the call to broadpeak.io with all query parameters: https://stream.broadpeak.io/bpkio-jitt/path/to/ad/segment.ts?bpkio_serviceid=abcdef&bpkio_sessionid=12345
Gap fillers
This cache policy also applies to gap filler segments
C.4. Cache policy for ad segments with SSAT
If you have configured SSAT in your DAI service, the URL for ad segments in the manifest contain instructions that trigger the appropriate impressions and trackers in broadpeak.io. It is therefore mandatory that those requests are always sent to broadpeak.io, and are not cached.
You must therefore disable caching in the SSAT rule in your CDN configuration (see rule 2.c. above).
Updated 16 days ago
Here are some playbooks on how to do all this with specific CDN products