This repository contains a set of scripts for doing analysis on results from runs of web-platform-tests uploaded to wpt.fyi. It utilizes a git repository, wpt-results, as a storage and compression mechanism for the results data.
The analysis files are expected to be run via node, and should be
independently documented inside the file.
Some of the charts generated from this repository are displayed on our associated GitHub Pages page - https://ecosystem-infra.github.io/wpt-results-analysis/. They are updated regularly using a GitHub Action (see update_gh_pages.yml)
Run npm install; this will install any necessary dependencies and clone the
wpt-results repo locally.
This repository stores results from WPT runs as a flat forest. Each run is an orphan commit with no parent, and is tagged with the run id. This allows for an excellent compression ratio, whilst still having reasonable lookup time.
The repository can be updated via git-write.js, though this happens
automatically upstream (via a cronjob). As such, to fetch new runs you should
just cd into the wpt-results directory and run git pull.
$ node browser-specific-failures.js --help
Usage: node browser-specific-failures.js [options]
Options:
--from: Starting date (inclusive)
(default: "2018-07-01")
--to: Ending date (exclusive)
(default: "2020-05-21")
--baseline: A YYYY-MM-DD date to 'pin' WPT to. Any test name not in
existence on the baseline date will be ignored.
(default: null)
--products: Browsers to compare. Must match the products used on
wpt.fyi
(default: ["chrome","firefox","safari"])
--output: Output CSV file to write to. Defaults to {stable,
experimental}-browser-specific-failures.csv
(default: null)
--[no]experimental: Calculate metrics for experimental runs.
(default: false)