Open audio analytics protocol specification. A unified spec for providing a common interface for sending audio analytics related events.
The simple protocol defines a common data interchange format and behaviors to allow a variety of mobile and desktop applications to emit/publish podcast analytics related events over the internet. Applications/clients may send one or more events in one call which enables queuing and batching. The data sent in the protocol is not sensitive data (it is still recommended to send data over a HTTPS
connection), yet is a sufficient amount of data for analytics services conforming/consuming to the protocol to provide analytics services based on that data.
[
{
"name":"media.play",
"props":{
"author":"Jonathan Gill",
"title":"Example Title",
"playbackRate":1,
"volume":1,
"networkState":1,
"readyState":4,
"muted":false,
"loop":false,
"paused":false,
"currentTime":2.997729,
"duration":3599.574513
},
"time":"2016-10-16T00:23:13.411Z"
},
]
The protocol/specification is designed to have the ability to be efficiently utilized in both clientside and serverside scenarios and leverage pervasive and known technologies and standards. Some of the efficiency comes from aggregate network traffic at scale, limited code footprint, and the familiarity with concepts originating in specs like HTML
, HTML5
, etc. The protocol/specification's ability to allow queuing and batching of events allows systems to react to a variety of scenarios including intermitten internet connections.
Casing of property names is also something that was taken into account. Properties like currentTime
are in lower camel case
since their origin is likely from a language or variable that is already in lower camel case
such as HTML5 Media/Audio resulting in less "variable name/value translation overhead." For custom variables snake case
may be more appropriate. The structure of the data payload for core scenarios is also minimally nested by design. The protocol uses existing platform agnostic standards like ISO 8601
formatted dates vs. ['Unix/Epoch Time](https://en.wikipedia.org/wiki/Unix_time) when appropriate and property names and values like
playbackRatemirror
HTML5 Audio/Media` properties.
JSON
is the data interchange format utilized by the protocol (ideally withoutwhitespace
inproduction
) as there is common support across numerous programming languages and environments. There are many reasonsJSON
was chosen as the data serialization format over formats likeXML
(one of these reasons is that inXML
attributes natively only support simple values and do not support complex values or custom types).HTTP
headers are also leveraged due to their inherent support in web requests. Headers likeX-Forwarded-For
can be utilized in serverside requests to replace and/or append theIP address
of the original requesting client instead of the server IP address.
Property Name | Type | Required | Description |
---|---|---|---|
name | string(255) | Yes | Name of the event and generally takes the form of major_topic.sub_topic |
nonce | string(255) | No | Check value on uniqueness of an event. When specified the value shall be an arbitrary string and functions as as a nonce. The property is also useful in retry scenarios. |
props | object | Yes | A dictionary of arbitrary key-value pairs of data where the values may be simple values (e.g. strings, integers, doubles/decimals/floats) or other nested JSON objects |
time | date/string | Yes | An ISO 8601 compatible date/time stamp in UTC of the time the event occurred |
id | string(255) | No | An access key , public API key , project key , id , etc. that may be utilized by analytics providers to know how to route the data being passed. This field may also serve as a method of authentication in particular scenarios. |
Second level properties are children of the property props
. Custom properties are allowed, however all custom properties shall be children or descendants of the props
property similar to the known and expected properties. In the table below the Mirrors HTML5
column indicates if the property name and value mirrors an HTML5 Media and/or Web Audio API property of the same name.
Property Name | Type | Required | Description | Mirrors HTML5 |
---|---|---|---|---|
author | string(255) | No | Name of the author or artist of the media/work | No |
client | string(255) | No | Unique identifier or name of the client or media player performing the action related to the request | No |
currentTime | number/double | Yes | Current time of the client/user in media playback in seconds. | Yes |
duration | number/double | Yes | Length of in media in seconds. | Yes |
explicit | boolean | No | If true, the media contains explicit content. | Yes |
loop | boolean | No | Indication of if the media is set to loop on the end of playback. | Yes |
media_ids | array(Media Id Type ) |
No | Array of Media Id Type . See Media Id Type for a type definition. |
No |
muted | boolean | Yes | Indication of if the media is muted. For example the usual volume setting of the media may be at 1 (100%), however the client has the media muted. | Yes |
networkState | integer/short | No | An integer in the set of 0-3 that indicates the current network state . |
Yes |
paused | boolean | Yes | Indication of if the media is paused. | Yes |
playbackRate | number/double | Yes | A number like 1 or 1.5 that indicates the relative speed of playback of the media where 1 = normal speed and values above or below 1 indicate a speed/rate change. Zero (0) is not a valid value. | Yes |
publisher | string(255) | No | Name of the publisher of the media/work related to the request (this may be different than the author) | No |
readyState | integer/short | No | An integer in the set of 0-4 that indicates the current media readiness state for playback. |
Yes |
title | string(255) | No | Title of the media/work | No |
user_id | string(255) | No | Unique identifier for the user performing the action related to the request. This identifier is typically unique to an application or organization. | No |
volume | number/double | Yes | A number between 0 and 1 (where 0 = 0% and 1 = 100%) that indicates the volume setting of the media. Example: .75 = 75% volume | Yes |
Property Name | Type | Required | Description |
---|---|---|---|
id | string(255) | Yes | Unique id of the media |
type | string(255) | Yes | Type of media id, see Known Media Id Types below for known values. This is an open-ended property and not a restricted enum so the type value may be any valid string. |
Type Name | Description |
---|---|
barcode | Unique, generally commercially registered, idenitider where the value can be a upc , ean , etc. (UPC and EAN are really just both versions of the same underlying barcode structure) |
uuid | "Arbitrary" universally unique identifier. This is sometimes called a guid or "Globally Unique Identifier". |
isbn | International Standard Book Number |
issn | International Standard Serial Number |
isrc | International Standard Recording Code |
iswc | International Standard Musical Work Code |
org_uuid | Organization, application, publisher identifier such as a release number or an internal id that is unique to an organization |
- Construct the
JSON
formatted data payload representing one or more events - Convert the data payload to an array of bytes
- Convert the array of bytes to a
Base 64
encoded string (with padding) - Transmit the HTTP request as a
GET
orPOST
request - Receive the HTTP response
When sending a HTTP request, standard HTTP headers such as User-Agent
should be included. User-Agent
shall also be populated for requests originating from with mobile applications.
When sending a HTTP GET
request, populate the querystring parameter d
with the base64
encoded data. When sending a HTTP POST
request, populate the body
of the request with the base64
encoded data and set the Content-Type
header of the HTTP request to ``application/x-www-form-urlencoded`.
If/when receiving a HTTP response, typical HTTP status codes
apply where a static code in the 200 range indicates success and 400 and 500 series status codes indicate a failure.
Here is an example of what an event submission using the protocol represented in HTTP request headers.
GET /?d=ICBbDQogICAgew0KICAgICAgIm5hbWUiOiJtZWRpYS5wbGF5IiwNCiAgICAgICJwcm9wcyI6ew0KICAgICAgICAgImF1dGhvciI6IkpvbmF0aGFuIEdpbGwiLA0KICAgICAgICAgInRpdGxlIjoiRXhhbXBsZSBUaXRsZSIsDQogICAgICAgICAicGxheWJhY2tSYXRlIjoxLA0KICAgICAgICAgInZvbHVtZSI6MSwNCiAgICAgICAgICJuZXR3b3JrU3RhdGUiOjEsDQogICAgICAgICAicmVhZHlTdGF0ZSI6NCwNCiAgICAgICAgICJtdXRlZCI6ZmFsc2UsDQogICAgICAgICAibG9vcCI6ZmFsc2UsDQogICAgICAgICAicGF1c2VkIjpmYWxzZSwNCiAgICAgICAgICJjdXJyZW50VGltZSI6Mi45OTc3MjksDQogICAgICAgICAiZHVyYXRpb24iOjM1OTkuNTc0NTEzLA0KICAgICAgICAgImN1c3RvbV9wcm9wZXJ0eTEiOiAiQW55dGhpbmcgeW91IHdhbnQiLA0KICAgICAgICAgImN1c3RvbV9wcm9wZXJ0eTIiOiAzMy4zMywNCiAgICAgIH0sDQogICAgICAidGltZSI6IjIwMTYtMTAtMTZUMDA6MjM6MTMuNDExWiINCiAgICB9LA0KICAgIHsNCiAgICAgICJuYW1lIjoibWVkaWEudGltZXVwZGF0ZSIsDQogICAgICAicHJvcHMiOnsNCiAgICAgICAgICJhdXRob3IiOiJKb25hdGhhbiBHaWxsIiwNCiAgICAgICAgICJ0aXRsZSI6IkV4YW1wbGUgVGl0bGUiLA0KICAgICAgICAgInBsYXliYWNrUmF0ZSI6MSwNCiAgICAgICAgICJ2b2x1bWUiOjEsDQogICAgICAgICAibmV0d29ya1N0YXRlIjoxLA0KICAgICAgICAgInJlYWR5U3RhdGUiOjQsDQogICAgICAgICAibXV0ZWQiOmZhbHNlLA0KICAgICAgICAgImxvb3AiOmZhbHNlLA0KICAgICAgICAgInBhdXNlZCI6ZmFsc2UsDQogICAgICAgICAiY3VycmVudFRpbWUiOjMuMTU0NDM0LA0KICAgICAgICAgImR1cmF0aW9uIjozNTk5LjU3NDUxMw0KICAgICAgfSwNCiAgICAgICJ0aW1lIjoiMjAxNi0xMC0xNlQwMDoyMzoxMy40MTFaIg0KICAgIH0sDQogIF0= HTTP/1.1
Host: example.com
Connection: keep-alive
User-Agent: Mozilla/5.0 (NeXTStep 3.3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36
Accept-Language: en-US,en;q=0.8
[
{
"name":"media.play",
"props":{
"author":"Jonathan Gill",
"title":"Example Title",
"playbackRate":1,
"volume":1,
"networkState":1,
"readyState":4,
"muted":false,
"loop":false,
"paused":false,
"currentTime":2.997729,
"duration":3599.574513,
"custom_property1": "Anything you want",
"custom_property2": 33.33
},
"time":"2016-10-16T00:23:13.411Z"
},
{
"name":"media.timeupdate",
"props":{
"author":"Jonathan Gill",
"title":"Example Title",
"playbackRate":1,
"volume":1,
"networkState":1,
"readyState":4,
"muted":false,
"loop":false,
"paused":false,
"currentTime":3.154434,
"duration":3599.574513
},
"time":"2016-10-16T00:23:13.411Z"
},
]
The following querystring parameters shall be supported for GET
requests only. The d
querystring parameter is ignored on POST
requests since that content will be in the body
of the request and the callback
can only be used successfully on GET
requests due to the limitations of JSONP
.
Parameter Name | Type | Required | Description | HTTP Methods Supported |
---|---|---|---|---|
d | string | GET Only |
Base64 encoded (with padding) version of the data payload |
GET |
callback | string | No | JavaScript function name to call on the return of the request's response. When utilized the Content-Type of the response will be application/javascript . |
GET |
https://example.com/?d=<data>&callback=<callback>
https://example.com/?d=ICBbDQogICAgew0KICAgICAgIm5hbWUiOiJtZWRpYS5wbGF5IiwNCiAgICAgICJwcm9wcyI6ew0KICAgICAgICAgImF1dGhvciI6IkpvbmF0aGFuIEdpbGwiLA0KICAgICAgICAgInRpdGxlIjoiRXhhbXBsZSBUaXRsZSIsDQogICAgICAgICAicGxheWJhY2tSYXRlIjoxLA0KICAgICAgICAgInZvbHVtZSI6MSwNCiAgICAgICAgICJuZXR3b3JrU3RhdGUiOjEsDQogICAgICAgICAicmVhZHlTdGF0ZSI6NCwNCiAgICAgICAgICJtdXRlZCI6ZmFsc2UsDQogICAgICAgICAibG9vcCI6ZmFsc2UsDQogICAgICAgICAicGF1c2VkIjpmYWxzZSwNCiAgICAgICAgICJjdXJyZW50VGltZSI6Mi45OTc3MjksDQogICAgICAgICAiZHVyYXRpb24iOjM1OTkuNTc0NTEzLA0KICAgICAgICAgImN1c3RvbV9wcm9wZXJ0eTEiOiAiQW55dGhpbmcgeW91IHdhbnQiLA0KICAgICAgICAgImN1c3RvbV9wcm9wZXJ0eTIiOiAzMy4zMywNCiAgICAgIH0sDQogICAgICAidGltZSI6IjIwMTYtMTAtMTZUMDA6MjM6MTMuNDExWiINCiAgICB9LA0KICAgIHsNCiAgICAgICJuYW1lIjoibWVkaWEudGltZXVwZGF0ZSIsDQogICAgICAicHJvcHMiOnsNCiAgICAgICAgICJhdXRob3IiOiJKb25hdGhhbiBHaWxsIiwNCiAgICAgICAgICJ0aXRsZSI6IkV4YW1wbGUgVGl0bGUiLA0KICAgICAgICAgInBsYXliYWNrUmF0ZSI6MSwNCiAgICAgICAgICJ2b2x1bWUiOjEsDQogICAgICAgICAibmV0d29ya1N0YXRlIjoxLA0KICAgICAgICAgInJlYWR5U3RhdGUiOjQsDQogICAgICAgICAibXV0ZWQiOmZhbHNlLA0KICAgICAgICAgImxvb3AiOmZhbHNlLA0KICAgICAgICAgInBhdXNlZCI6ZmFsc2UsDQogICAgICAgICAiY3VycmVudFRpbWUiOjMuMTU0NDM0LA0KICAgICAgICAgImR1cmF0aW9uIjozNTk5LjU3NDUxMw0KICAgICAgfSwNCiAgICAgICJ0aW1lIjoiMjAxNi0xMC0xNlQwMDoyMzoxMy40MTFaIg0KICAgIH0sDQogIF0=&callback=customJavaScriptFunction
Samples events in their unencoded format are below:
Event Name |
---|
media.play (Basic/Sample) |
media.play (Actual/Detailed) |
media.pause |
media.download |
media.ended |
media.ratechange |
media.seeked |
media.timeupdate |
media.volumechange |
ui.action |
- In a serverside request, how do I specify the IP address of the user and not the IP address of the server?
- Create or append to the
X-Forwarded-For
header (append if it already exists) in the request that is sent the analytics service.
- Create or append to the
- How can I add custom properties to events?
- Add any custom properties as descendants of the
props
property. The use of descendants and not children is to indicate that nested structures can be supported.
- Add any custom properties as descendants of the
- Why is the data sent via the protocol
Base 64
encoded?- While it may seem that
JSON
is primarily a string based data interchange format, the payload of data (which is essentially a structured string) is converted to bytes (e.g. an array of bytes) thenbase64
encoded with multiple purposes in mind.- All strings regardless of text encoding can be converted to byte arrays and all bytes can be encoded as
base64
UTF-8
strings, such as 音频, can be encoded inbase64
if converted to bytes first, which opens up the protocol for use with values from different languages.- Encoding potential querystring parameter values as
base64
eliminates the need to alsourl encode
or escape the value(s) - A light amount of obfuscation of the data
- All strings regardless of text encoding can be converted to byte arrays and all bytes can be encoded as
- While it may seem that
- How do I know
base64
encode data?- Most modern programming languages have built-in support for
base64
encoding and for languages and environments likeJavaScript/Node
, there are open source modules and libraries that can perform thebase64
encoding and decoding.
- Most modern programming languages have built-in support for
- How do I know if my
base64
encoded data is padded correctly?- The
mod
4 of a base64 encoded string should be 0 and if the result is not 0 then there's some missing padding in the encoding algorithm. This may seem crazy, but often this just means your encoding algorithm just needs to add equal signs=
at the end of the string until themod 4
of the encoded string equals 0.
- The
- Is there a recommended querystring parameter to use for data in a
GET
request?- Yes, it is recommended to use
d
as the querystring parameter due to brevity (d
=data
).
- Yes, it is recommended to use
- How do I send multiple events in one request?
- Multiple events should be represented as a
JSON
array where each event is aJSON
object. Here is an unencoded representation of data before it isbase64
encoded:
[ { "name":"media.play", "props":{ "author":"Jonathan Gill", "title":"Example Title", "playbackRate":1, "volume":1, "networkState":1, "readyState":4, "muted":false, "loop":false, "paused":false, "currentTime":2.997729, "duration":3599.574513 }, "time":"2016-10-16T00:23:13.411Z" }, { "name":"media.timeupdate", "props":{ "author":"Jonathan Gill", "title":"Example Title", "playbackRate":1, "volume":1, "networkState":1, "readyState":4, "muted":false, "loop":false, "paused":false, "currentTime":3.154434, "duration":3599.574513 }, "time":"2016-10-16T00:23:13.411Z" }, ]
- Multiple events should be represented as a
- What happens if there is an error in sending multiple events in one request like a formatting error in one of the events?
- All of the events in the set are rejected/not accepted and a
400 series
HTTP status code is returned as the status code for the response.
- All of the events in the set are rejected/not accepted and a
- What should the maximum length be of a custom
string
property? - The maximum length for any string is 255 characters. While your requirements may be that the string is shorter (which it can be), the maximum length for data storage of that string in any implementing analytics system will be 255 characters.
- Are there any reserved characters in property names?
- Yes, the characters that may not be used in property names are:
/
,.
, and any character or complete phrase that is disallowed in a JavaScript property or variable name (e.g.true
is not a valid property name).
- Yes, the characters that may not be used in property names are:
- Are there any reserved property names?
- Yes.
time
shall always represent anISO 8601
compatible date/time format when used regardless of its location in a hierarchy.
- Should
GET
andPOST
requests both support receiving multiple events in one request? - Yes
- What should the
Content-Type
HTTP header be ofPOST
requests in the protocol?- The
Content-Type
of aPOST
request shall beapplication/x-www-form-urlencoded
.
- The
- Is whitespace important?
- We recommend that
JSON
not have whitespace outside of the whitespace within a string value in production usage. Whitespace outside of this use is not significant in this protocol and simply increases the data payload size.
- We recommend that
- Why do you use the term
uuid
instead ofguid
?- Because
guid
aka Globally Unique Identifier does not quite fit for an interplanetary species.
- Because
The specification described on this page or document is available under the Apache 2.0 License. This has been another Backtracks, Inc. & Friends joint.