Automagical graphs

[maciejhirsz]

So, I've the json.rs domain at my disposal. If this benchmark can be made to dump a JSON (duh) file with the results of the benchmark, which then could be uploaded to github, I can put up some simple webpage on that address that grabs the JSON from the repo and draws the graphs for it in JavaScript. That way we can have something nice to present, without having to do much of manual updates every time, just build -> run -> commit and push.

How does that sound?

Edit: Actually given the formatting of the log, I might be able to visualize it by scraping the readme.

[dtolnay]

That sounds great! If you haven't already, take a look at perf.rust-lang.org for inspiration. They even have graphs of performance at a couple-hours granularity.

The other thing I would love to have that is relevant to this is a cloud server for running the benchmark so it isn't just "David's 4 year old laptop." Since you are paying for json.rs I can pay for an AWS instance to run once a day or so.

[maciejhirsz]

Aye, EC2 would be nice. It should be relatively managable to get nativejson-benchmark to run on it too. That way the page can serve not just as a curiosity for us, but also help popularize the language itself.

[maciejhirsz]

Ok, this is just strapped together with an insane amount of duct tape, but it works:

http://json.rs/

Need to abstract things out, do graphs for all files, etc. etc.

It's parsing the README from the repo here, so changes will update the graph.

[maciejhirsz]

@dtolnay the website is now somewhat usable, should we include it in the README / description?

[dtolnay]

Very nice! Yes go ahead and include a link.

Have you tried transposing the results so all the parses are next to each other and all the stringifies are next to each other?

serde_json dom parse
serde_json struct parse
json-rust dom parse
rustc_serialize dom parse
rustc_serialize struct parse
rapidjson dom parse

serde_json dom stringify
serde_json struct stringify
json-rust dom stringify
rustc_serialize dom stringify
rustc_serialize struct stringify
rapidjson dom stringify

You almost never want to compare parse performance vs stringify performance so it seems like the current arrangement makes it unnecessarily hard to compare the things you do want to compare, which is performance across libraries. We can use this as

I have no time this weekend but I would like to include a little more explanation/analysis of each test. We should be able to use this to explain at a technical level what features account for the results - things like:

• The difference between DOM vs struct parsing and how struct parsing will likely do fewer memory allocations so that probably accounts for much of the difference there
• How the json-rust number type works.
• Difference between how serde and rustc_serialize do struct parsing that accounts for the reversed relationship of faster DOM vs faster struct.
• What it means to "optimize for key lookup performance" (that line means nothing to me even though I mostly understand the implementation details).

We can hide the analysis behind an accordion if you want to keep the landing page minimal.

[maciejhirsz]

I was thinking about organizing the graphs better, will look into it tomorrow, I'm pretty done for tonight :). You can disable parsing / stringifying on the graphs as they are, but the rescaling seems buggy, makes me want to look into some other graph libs.

[tshepang]

I can't reach the site, http://json.rs

[nico-abram]

I can't reach the site either @maciejhirsz

[maciejhirsz]

It's been a long time since 2016, I've let the domain go :P.