This tool provides a straightforward (if computationally expensive) way of comparing the performance of encoding ladders.
It takes 3 arguments:
- A JSON file specifying viewer information
- The location on local disk of mezzanine video content
- An HLS master manifest matching the given mezzanine
The tool then leverage's Netflix's VMAF to estimate "average viewer vmaf", which provides a rough mechanism of comparing encoding ladders
As video engineers we are often asked the question "how good is my encode". This question is tricky to answer, but newer tools like VMAF have given us a mechanism of estimating this "goodness" in a way that's easy for humans to understand.
Things get far more complex when we're asked to answer the question:
"how good is my encoding ladder"
To answer this question requires calculating not only VMAF, but actually understanding the properties of the target audience, their viewing resolutions and bitrates, and an understanding of the devices those viewers are consuming content.
Our goal is to show that by taking a holistic view of viewer experience, we can create encoding ladders that optimize not just for on-paper quality, but the quality that our viewers will actually experience out in the wild.
This tool requires a recent version of FFmpeg to be installed on the system, alongside Netflix's VMAF.
You will also need version 1.10 or higher of golang: https://golang.org/dl/
Usage: vmaf_analyzer [--subsample n] [--threads n] [--model vmaf_v0.6.1.pkl] [--datafile data.json] mezzanine.mp4 https://example.com/hls_stream.m3u8
-datafile string
Location of the data file to use for processing (default "data.json")
-model string
vmaf model to use (default "vmaf/model/vmaf_v0.6.1.pkl")
-subsample int
What vmaf subsampling factor to use (default 30)
-threads int
How many threads used to run vmaf (default 10)
This tool can be used locally if the following dependencies are avilable on host:
- ffmpeg
- vmaf
The tool can also be run on a Docker container with the provided Docker image that installs all necessary tools:
docker build -t muxinc/vmaf_analyzer .
Map a local directory containing the mezzanine.mp4 file and run the docker container:
docker run --rm \
--volume $(pwd)/videos:/videos \
--volume $(pwd)/data:/data \
muxinc/vmaf_analyzer:latest ./vmaf_analyzer --datafile=/data/data.json /videos/mux-video-intro.mp4 https://stream.mux.com/pnQZ4GRsFpAljZEf4EmFEwjlpe5sV4lu.m3u8
There are two pieces of viewer information required by this tool;
- Bitrate Distribution: Sum of users with a bitrate, in 100kbps buckets
- Resolution Distribution: Sum of users with a resolution, in 16 pixels buckets
We assume that resolution and bitrate are independent
- Explicitly handle mobile and 4k viewing conditions
- Validate our assumptions that bitrates and resolutions are independent variables