ember-macro-bench

A tool for measuring changes in Ember to applications.

This uses chrome-tracing and har-remix to measure the effects of small changes to Ember on applications.

This is also a demonstration of using these tools to measure applications.

Setup

Prerequisites

These instructions assume Mac and using homebrew.

Install R

brew tap homebrew/science
brew install r

Run R

Then install ggplot2

install.packages("ggplot2")
q()

Install yarn

brew install yarn

Install

git clone git@github.com:krisselden/ember-macro-benchmark.git
cd ember-macro-benchmark
yarn

Run the Server

Edit the config.json

har: the emberaddons.com HAR file, the example is a recording of emberaddons.com built with Ember 2.13.0-beta.1
ember: path to an ember dist to splice into the vendor.js, the example results were tested against Ember commit 2c45a3b.
dist: path to full emberaddons.com build. Use this when testing ember versions that require wire format changes. Note that you must use an unfingerprinted production build.
- If the dist property is set you may also set fingerprinted to false to ensure that assets are correctly served when you have opted out of fingerprinting. If fingerprinted is undefined it will assume assets are fingerprinted.

Caveat: the templates are in the recorded app, you can only test with an Ember with compatible template wire-format as the recording.

yarn run serve

Run the Benchmark

yarn run bench

Generate the Report

Now we need to update the report.R to indicate which of the results are the control and experiment.

Update the following line:

experiment = phases[phases$set == '<enter experiment here>' & phases$phase == 'render' & phases$type == 'cumulative',]$ms
control = phases[phases$set == '<enter control here>' & phases$phase == 'render' & phases$type == 'cumulative',]$ms

Run the report:

./report.R

View the results:

open results/*.png