Timed metadata in HLS and HDS streams

The timed metadata information can be used by clients as cuepoints, for information display, to invoke time-aligned actions, and so on. This information is available to the client on the playback timeline in the form of ID3 tags in HLS.

The client interprets the ID3 tags encapsulated by the encoder or the publisher. Adobe Media Server packages this information in the HLS and HDS stream, so that it can be retrieved using iOS and Flash APIs.

This section provides:

• Information about inserting the timed metadata in the live and VOD streams.

• Configuration to define the custom tag information.

• Workflow for tag insertion and delivery.

In a live stream, input message are in form of AMF command message with the name onMetaInfo. onMetaInfo contains an object with one property for each meta item. A property has { name, value } string pairs.

The following is a sample server-side ActionScript code to illustrate how to push the data via AMF messages.sendTimedMetaData = function(streamObj) { metaData = new Object(); udURL = new Object(); udURL.data = "http://example.com"; metaData.UserDefinedURL = udURL; delete udURL; udURL = null; commInfoURL = new Object(); commInfoURL.data = "www.adobe.com"; metaData.CommercialInformationURL = commInfoURL ; delete commInfoURL; commInfoURL = null; cmpsr = new Object(); cmpsr.data = "Shankar"; cmpsr.description ="Comment about Shankar" metaData.Composer = cmpsr; delete cmpsr; cmpsr = null; udText = new Object(); udText.data = "UserTextRicha"; metaData.UserText = udText; delete udText; udText = null; privData = new Object(); //base64 encoded binary data as String. privData.data = "VGhpcyBpcyBiYXNlNjQgZW5jb2RlZCBzdHJpbmc="; privData.ownerId = "0"; metaData.PrivateData = privData; delete privData; privData = null; comnt = new Object(); comnt.data = "This is Comment"; comnt.language = "eng"; metaData.Comment = comnt; delete comnt; comnt = null; genObj = new Object(); genObj.data = "VGhpcyBpcyBnZW5lcmFsaXplZCBvYmplY3QgYmFzZTY0IGVuY29kZWQ="; genObj.filename = "myFile.txt"; metaData.GeneralObject = genObj; delete genObj; genObj = null; syLyr = new Object(); syLyr.data = "This is SyncLyrics"; syLyr.language = "eng"; metaData.SyncLyrics = syLyr; delete syLyr; syLyr = null; syTxt = new Object(); syTxt.data = "This is SyncText"; syTxt.language = "eng"; metaData.SyncText = syTxt; delete syTxt; syTxt = null; streamObj.send("onMetaInfo", metaData); delete metaData; metaData = null; } application.onPublish = function(clientObj, streamObj) { .... setInterval(sendTimedMetaData, 500,streamObj); }

Also, any information contained in the onMetadata command message is inserted in the HLS stream, if property name is understandable as defined below.

In case of VOD, the following types of input are supported:

• onMetaInfo data message as defined for live.

• iTunes tags in the list box of the MP4 are parsed and send as timed-metadata.

• onMetaData information is sent as an initial metadata at timestamp 0 or first message timestamp.

Output in HDS

In HDS, the following function is called: ns.onMetaInfo = function(infoObject:Object) { for (var propName:String in infoObject) { trace(propName + " = " + infoObject[propName]); } };

Output in HLS

In case of HLS, onMetaInfo message properties are converted to the ID3 metadata tags. For more information on how these tags appear in HLS stream, see these specifications. For information about ID3 metadata tags, see http://id3.org/id3v2.3.0.

ID3 tag introduction

A basic understanding of the ID3 tag is provided below. For detailed information, see http://id3.org/id3v2.3.0. An ID3 tag has the following structure. Complete information about ID3 header is available at http://id3.org/id3v2.3.0#ID3v2_header.

An ID3 tag can contain one or more ID3 frames.

Compressed size flag is defined if the frame is compressed. In case of AMS this field is omitted. The supported ID3 frame types are listed below.

Supported ID3 frames for HLS delivery

ID3 frames supported in the AMS for HLS delivery and their corresponding input AMF input syntax are described below. For all kinds of ID3 frames, the information comes as AMF data message with method onMetaInfo or onMetaData with data as AMFObject of one or multiple name value properties.

URL Link frames

According to the ID3 specifications, with these frames dynamic data can be added to the tag. Some examples of usage are webpages with touring information, price information, and plain ordinary news.

The required AMFObject properties are:

• Property Name:String - <Name>, where the values for name are defined below.

• Value syntax: The value can be defined in the following ways:

  • String - URL string

  • AMFObject with name-value property as follows

The default property names and the corresponding ID3 tags are: { "CommercialInformationURL", "WCOM"}, { "CopyrightInformationURL", "WCOP"}, { "AudioFileURL", "WOAF"}, { "ArtistWebURL", "WOAR"}, { "AudioSourceWebURL", "WOAS"}, { "InternetRadioStationWebURL", "WORS"}, { "PaymentURL", "WPAY"}, { "PublisherWebURL", "WPUB"},

User-defined link frame (WXXX)

This frame is intended for URL links related to the audio file in a similar way to the other W-frames.

The required AMFObject properties are:

• Property Name:String is UserDefinedURL.

• Value syntax:String: The value can be defined in the following ways:

  1. String - <value> The default output encoding is UTF-8.

  2. AMFObject with name-value property as follows.

     Property name	Property value	AMF0 type	Optional	Default
     textEncoding	UTF-8 or UTF-16	String	Yes	UTF-8
     description	User text description	String	Yes	UserDefinedURL
     data	URL	String	No	-
     groupIdentifier	Id3frame group ID	String	Yes	-

Text information frame

According to the ID3 specifications, the text information frames are the most important frames. Such frames contain information like artist, album, and more.

The required AMFObject properties are:

• Property Name:String - <Name> where names are defined below.

• Value syntax:String - The value can be defined in the following ways:

  • String - <value> The default output encoding in this case is UTF-8.

  • AMFObject with name-value property as follows.

    Property name	Property value	AMF0 type	Optional	Default
    textEncoding	UTF-8 or UTF-16	String	Yes	UTF-8
    data	Text data	String	No	-
    groupIdentifier	Id3frame group ID	String	Yes	-

The default property names and the corresponding ID3 tags are: { "Title", "TALB"}, { "BPM", "TBPM"}, { "Composer", "TCOM"}, { "ContentType", "TCON"}, { "Copyright", "TCOP"}, { "Date", "TDAT"}, { "PlaylistDelay", "TDLY"}, { "EncodedBy", "TENC"}, { "Lyricist", "TEXT"}, { "FileType", "TFLT"}, { "Time", "TIME"}, { "ContentGroupDesc", "TIT1"}, { "ContentDesc", "TIT2"}, { "ContentSubtitle", "TIT3"}, { "InitialKey", "TKEY"}, { "Language", "TLAN"}, { "Duration", "TLEN"}, { "MediaType", "TMED"}, { "OriginalTitle", "TOAL"}, { "OriginalFilename", "TOFN"}, { "OriginalWriter", "TOLY"}, { "OriginalArtist", "TOPE"}, { "OriginalYear", "TORY"}, { "Licensee", "TOWN"}, { "Artist", "TPE1"}, { "BandName", "TPE2"}, { "PerformerRefinement", "TPE3"}, { "RemixedBy", "TPE4"}, { "PartOfSet", "TPOS"}, { "ContentPublisher", "TPUB"}, { "TrackNumber", "TRUK"}, { "RecordingDate", "TRDA"}, { "InternetRadioStation", "TRSN"}, { "InternetRadioStationOwner", "TRSO"}, { "ContentSize", "TSIZ"}, { "ISRC", "TSRC"}, { "EncodingSettings", "TSSE"}, { "PublishingYear", "TYER"},

User-defined text information frame (TXXX)

The required AMFObject properties are:

• Property Name:String - UserText

• Value syntax:String - The value can be defined in the following ways:

  • String - <identifier> The default output encoding is UTF-8.

  • AMFObject with name-value property as follows.

    Property name	Property value	AMF0 type	Optional	Default
    textEncoding	UTF-8 or UTF-16	String	Yes	UTF-8
    description	User text description	String	Yes	UserText
    data	Text data	String	No	-
    groupIdentifier	Id3frame group ID	String	Yes	-

Private data frame

This frame can contain information that does not fit into the other frames. A software producer can embed the information that its program uses. It is recommended to keep the number of such frames as low as possible.

The required AMFObject properties are:

• Property Name:String - PrivateData

• Value syntax:String - The value can be defined in the following ways:

  • AMFObject with name-value property as follows.

    Property name	Property value	AMF0 type	Optional	Default
    ownerId	Unique identification	String	No	-
    data	Base 64 encoded binary data as String	Sring	No	-
    groupIdentifier	Id3frame group ID	String	Yes	-

Comment frame

This frame is intended to carry full text information that does not fit in any other frame. Newline characters are allowed in the comment text string.

The required AMFObject properties are:

• Property Name:String - Comment

• Value syntax:String - The value can be defined in the following ways:

  • AMFObject with name-value property as follows.

    Property name	Property value	AMF0 type	Optional	Default
    textEncoding	UTF-8 or UTF-16	String	Yes	UTF-8
    language	ISO-639-2 (3 character representation)	String	No	eng
    description	User text description	String	Yes	Comment
    data	Comment string	String	No	-
    groupIdentifier	Id3frame group ID	String	Yes	-

    The language is represented with three characters according to ISO-639-2 standard. Extra characters in the Language String in AMF are dropped when generating ID3 message.

Generalized encapsulated object

Any type of file can be encapsulated in this frame. The required AMFObject properties are:

• Property Name:String - GeneralObject

• Value syntax:String - The value can be defined in the following ways:

  • AMFObject with name-value property are as follows.

Synchronized lyrics or text (SYLT)

According to the ID3 specifications, this is another way of incorporating a transcript or lyrics in the audio file as text. This method offers a way for implementors to synchronize the content with the audio.

The required AMFObject properties are:

• Property Name:String - SyncText or SyncLyrics

• Value syntax:String - The value can be defined in the following ways:

AMFObject with name-value property as follows.

The text encoding is UTF-16 (with BOM). The property types can be:

Other

Lyrics

Text transcription

Movement or part name

Events

Chord

Trivia or pop up information

User-defined PropertyName and ID3 tag mapping

The property names defined in this document are the default names that are used by Adobe Media Server. However, users can provide custom XML files to,

• Map the property name to specified ID3 tag name.

• Add new property name and ID3 tag name under similar type.

XML syntax

If a property name is not overridden in a custom XML, the default values are used. Similarly, if a frame type is not defined, the default value for that frame is used. You can define multiple property names for the same ID3 tag.

Configuration required

In httpd.conf file, under the Location tag for HLS VOD or HLS Live, mention HLSAMFToId3MappingFilePath c:/mytags.xml.

A sample XML file with the default AMS amf to ID3 mappings is available at [rootinstall]\Apache2.4\conf\AMF_to_Id3_Mapping.xml.