Configure closed captioning

Closed captioning (CC) allows the content provider to display text overlayed on the video content thus providing additional information about the video content to the consumers. Closed captioning is primarily used as an accessibilty tool to show a transcription of the audio portion of the video content as it occurs. The term “closed” in closed captioning indicates that the captions are not made part of the video by default. Hence, the captions have to be decoded/extracted by an external tool. Adobe Media Server (AMS) supports injecting closed caption data into the video stream supporting the following streaming techniques:

• HDS (Live and VOD)

• HLS (Live and VOD)

• RTMP

Closed captioning in AMS is implemented by injecting/embedding caption data within the H264 SEI NALU as specified in ANSI/SCTE 128 2010 Section 8. Adobe Media Server supports industry standard 608/708 closed captioning for video streaming to the following output channels/devices:

• Adobe Flash Player/AIR (through the OSMF framework)

• iOS devices (through HLS)

This document defines the input and output workflows, use cases, and configurations for supporting the 608/708 closed caption data in AMS. The closed caption data will be embedded inside the VOD stream and can be viewed through HDS, HLS, or RTMP based subscribers.

There are many techniques for implementing the caption data within the video stream. Few of these techniques are:

• Embedding the caption data in the video content. For instance, media stream or storage format can hold 608/708 caption data embedded inside the H264 SEI NALU as part of ATSC picture data.

• MP4 video content can contain a timed track, which can hold the caption data.

The following closed captioning workflows are supported in AMS:

• Type 1 – Streaming captioning data stored in ATSC Picture User Data.

• Type 2 – Extracting the caption data from a timed track and embed the data in a ATSC Picture User Data inside H264 SEI NALU (Network Abstraction layer Units).

• Type 3 – Embedding captioning data inside an AMS-defined onCaptionInfo AMS message.

Type 1 – CC in ATSC Picture User Data

In this type, the CC input is natively supported for HLS. The Type 1 input can be just-in-time converted to an OnCaptionInfo message to support caption display on HDS and RTMP. The content providers can embed closed captioning data as part of the ATSC Picture User Data inside an H264 SEI NALU. However, this method is only suitable for video streams supporting H264 codec types.

note: This type is widely used for MPEG2 TS streams.

HLS players can natively support such modified stream. AMS can be used to just-in-time package this embedded format into AMS defined onCaptionInfo messages to display captions over HDS and RTMP. Note that this is the ideal way of embedding the caption data. To prepare/package video content in this form, you can:

• Use AMS for packaging the caption data.

• Convert a natively available MPEG2 TS stream to this type.

Type 2 – Extracting CC in Timed Track

The closed caption data can also reside inside a separate timed track. Apple has defined a timed track format for QuickTime movie files.You can use the tool provided by AMS to convert this type of timed tracks containing caption data into an ATSC Picture User Data of Type I.

Type 3 – CC in AMF Message

The closed caption data can be just-in-time extracted from Type 1 data and stored inside the AMS-defined onCaptionInfo AMF message. The following file types are supported:

• MP4

• F4V

You can use AMS to package such files capitalizing on the onCaptionInfo message as a way of sending caption data as part of the live/VOD stream.

Note that HLS is not supported for this type.

The following table shows the summary of supported types for HLS, HDS, and RTMP:

Packaging captions

The HLS protocol demands packaging the caption data as user_data_registered_itu_t_t35, embedded as H264 SEI NALU in video access unit within the MPEG2 TS stream. Therefore, input video frames will be exactly translated into the MPEG2 transport stream. No repackaging in terms of caption data will be required. In case of HDS and RTMP, H264 SEI NALU will be converted to the onCaptionInfo data messages. In the incoming video stream, a NALU of type 6 with a payload of type 4 (user_data_registered_itu_t_t35) will be parsed for caption data. If there are multiple NALU in one access unit of the video frame, they will be packaged into a single data message. In this case, original caption data and corresponding NALU inside H264 access unit will remain intact. This will be done at the time of content packaging.

Configurations

AMS provides easy configurations to turn on and off the processing of the caption data on the server side. However, these configurations will not extract out the caption data encapsulated in any of the types defined in the previous section.

VOD use case

Configuring httpd.conf at the Server level (Only for HDS-VOD)

You can configure closed captioning at the server level by modifying the JitGenerateCaptionInfo element in Apache Server’s httpd.conf file as follows:<Location /hds-vod> JitGenerateCaptionInfo true ... </Location>

Configuring jit.conf at the content level (Only for HDS-VOD)

You can configure closed captioning at the content level by modifying the cc-info-enabled element in jit.conf file as follows:<hds> <cc-info-enabled> true </cc-info-enabled> </hds>

Configuring server.xml (only for RTMP)

For RTMP, you can configure closed captioning by adding the following code snippet in the server.xml file:<Mp4> <Playback> <GenerateCCInfo>true</GenerateCCInfo > </Playback> </Mp4>

Live use case

Application level (HDS and RTMP)

You can configure closed captioning at the application level by modifying the GenerateCCInfo element in Application.xml file as follows:<StreamManager> <Live> <GenerateCCInfo>true</GenerateCCInfo > </Live> </StreamManager >

Stream level (HDS and RTMP)

You can also override the application level configuration by modifying the application’s main.asc file by setting the Stream function generateCCInfo:Stream.generateCCInfo = true;

Support for OSMF

Closed captioning can be enabled for OSMF players through the OSMFCCDecoder class. The OSMFCCDecoder class uses the displayObject, updates the video size, and resets the decoder. You can perform the following actions using the OSMFCCDecoder:

• Enable or disable the display of captions.

• Select caption channels.

• Modify various caption display attributes like font, text, and background colors.

The following sections describe the steps needed to get started with closed captioning in OSMF:

Step 1 - Preparing the assets

Before you begin, you need to prepare your assets to hold the closed captionining data.

Enable GenerateCCInfo in Server.xml for VOD playback as follows:<Root> <Server> <Streams> <Mp4> <Playback> <GenerateCCInfo>true</GenerateCCInfo> </Playback> </Mp4> </Streams> </Server> </Root>

When the GenerateCCInfo option is enabled, AMS will process the video packets for closed captions and will generate the onCaptionInfo message, which will be sent to the client.

Enable GenerateCCInfo in Application.xml for live playback as follows: <Application> <StreamManager> <Live> <GenerateCCInfo>true</GenerateCCInfo> </Live> </StreamManager> </Application>

Step 2 - Writing the ActionScript code

You need to initialize the OSMFCCDecoder and expose any customisable attributes as follows: var player:MediaPlayer = ...; // Initialize the player. // Listen for MediaPlayerStateChangeEvent on player and // when the state is MediaPlayerState.READY, then initialize the // decoder as shown below: var ccDecoder:OSMFCCDeco d er; // Create a new instance ccDecoder = new OSMFCCDecoder(); // Bind the instance to your media player ccDecoder.mediaPlayer = player; // Bind the instance to the media player view area // that can be used to render. ccDecoder.mediaContainer = mediaContainer; // Enable the instance ccDecoder.enabled = true; // Optionally, select a type and service // At present, only one type is supported; CEA-708 wrapped over 608, // which is identified by the value CCType.CEA708. ccDecoder.type = CCType.CEA708; ccDecoder.service = CEA708Service.CC1;

The OSMFCCDecoder contains a number of properties that can be exposed through the UI controls. Most of these properties enable overriding the font and color attributes and are required under the FCC guidelines. You are responsible for adding the UI controls to expose these properties. Also, you should add the necessary code to save and restore the settings between sessions. UI controls must be provided in the video player to enable/disable closed captioning display and to select the desired caption service/channel. UI controls may be provided to allow the end user to select values for the displayed font size, background color, and opacity.

The following code snippet describes the steps involved in exposing custom attributes through ActionScript: /** * The type of closed captioning data to decode and render. * A value of null indicates no selected type. * Valid values are values of the CC Type enumeration class. * The default value is null . * * @throws ArgumentError if the specified type is not supported. * / public function gettype():String; public function settype(value:String):void; / ** * The MediaPlayer this OSMFCCDecoder is associated with. * Assig ning any value disassociates the OSMFCCDecoder from a * currently assigned MediaPlayer (if any). * If the assigned value is non-null, the OSMFCCDecoder will * automatically register itself as an event listener on * the provided MediaPlayer and access its view property * to use as the video surface to render captions over. */ public function get mediaPlayer():MediaPlayer public function set mediaPlayer(value:MediaPlayer):void /** * Whether caption data decoding and rendering is enabled or disabled. * The default value is true. * When this property transitions from true to false a side-effect * is a reset of internal decoder state and * any currently rendered captions are cleared. */ public function get enabled():Boolean public function set enabled(value:Boolean):void /** * The caption "service" to decode and render. * For CEA-708 wrapped over 608 closed captioning, supported values * are values of the CEA708Service enumeration class. * CEA708Service.CC1, CEA708Service.CC2, CEA708Service.CC3, CEA708Service.CC4 * @throws ArgumentError if the assigned service value is not supported * or available for the closed captioning * type specified at construction time. */ public function get service():String public function set service(value:String):void /** * Font override to apply to rendered text. * Default value is null, indicating no override, and text will * assume its intrinsic font according to caption * authoring. * To override the font assign an embedded or system font * name and to disable an override assign null. */ public function get font():String public function set font(value:String):void /** * Text color override to apply to rendered text. * Default value is -1, indicating no override, and text * will assume its intrisic color which may vary according * to caption authoring. * To override the text color assign a RGB hex value (e.g. 0xFF00FF) * and to disable an override assign -1. * @throws ArgumentError if the value is not -1 and not within the RGB hex range. */ public function get textColor():int public function set textColor(value:int):void /** * Background color override to apply to the background of text character cells. * Default value is -1, indicating no override, and cell backgrounds * will assume their intrinsic color which * may vary according to caption authoring. * To override the background color assign a RGB hex value (e.g. 0xFF00FF) * and to disable an override assign -1. * @throws ArgumentError if the value is not -1 and not within the RGB hex range. */ public function get backgroundColor():int public function set backgroundColor(value:int):void /** * Text opacity override to apply to rendered text. * Default value is -1, indicating no override, and text * will assume its intrinsic opacity which may vary * according to caption authoring. * To override text opacity assign a value between 0 (fully transparent) * to 1 (fully opaque) and to disable * an override assign -1. * @throws ArgumentError if the value is not -1 and not within the range of [0, 1]. */ public function get textOpacity():Number public function set textOpacity(value:Number):void /** * Background opacity override to apply to the background of text character cells. * Default value is -1, indicating no override, and backgrounds will * assume their intrinsic opacity which may * vary according to caption authoring. * To override background opacity assign a value between 0 (fully transparent) * to 1 (fully opaque) and to * disable an override assign -1. * @throws ArgumentError if the value is not -1 and not within the range of [0, 1]. */ public function get backgroundOpacity():Number public function set backgroundOpacity(value:Number):void /** * Resets the OSMFCCDecoder's internal state, clearing any * currently rendered captions. * The currently selected caption service and display overrides are not changed. */ public function reset():void /** * The default font to use when no custom font is specified. * This is set to a reasonable default for the device. When using custom devices, * you may alter this property to change the default font. */ public function get defaultFont():String public function set defaultFont(value:String):void /** * Edge type override to apply to rendered text. * * Valid values are indicated by the EdgeType enumeration, or null to * indicate no override. Valid Values: * EdgeType.NONE: Indicates that no edge decoration should be used. * EdgeType.DEPRESSED: Indicates that the edge will appear as depressed. * EdgeType.UNIFORM: Indicates that the edge will with a uniform border. * EdgeType.LEFT_DROP_SHADOW: Indicates that the edge will with a * shadow falling to the left. * EdgeType.RIGHT_DROP_SHADOW: Indicates that the edge will with a * shadow falling to the right. * * @default null * * @throws ArgumentError if the value is not null and not a memory of * the EdgeType enumeration. **/ public function get edgeType():String public function set edgeType(value:String):void /** * Edge color override to apply to rendered text. * The value should be an RGB hex value (e.g. 0xFF00FF). * * A value of -1 indicates no override. A reasonable * default edge color will be selected. * * @default -1 * * @throws ArgumentError if the value is not -1 and not within the RGB hex range. **/ public function get edgeColor():int public function set edgeColor(value:int):void

Using the ccConvertor tool

The ccConvertor tool is a command line utility (bundled with Adobe Media Server) that can be used to convert any MOV files with Closed Caption content (clcp tracks) into an MP4 file with embedded caption NALUs in 708 format (having 608 wrapped in it).

Usage

The usage of the ccConvertor tool on Windows is as follows:ccConvertor.exe --input-file=<file> --[setting]=[arg]...

Parameters

The following table describes the supported input parameters:

Note that if a relative path is passed through the configuration file, the path is relative to the directory containing the configuration file.

To understand the configuration file format, see the sample configuration file available at <AMS_INSTALL_DIR>\tools\ccConvertor\sample_cfg.xml.