R7RS Benchmarks

Benchmarks for various Scheme implementations. Taken with kind permission from the Larceny project, based on the Gabriel and Gambit benchmarks.

See the results here

Running the tests

Just run

./bench <scheme> <test>

Schemes that should work:

This should result in a file ./results.<scheme>.

All of these Schemes are packaged for ArchLinux. The tests are run with whatever that package produces.

Run

make csv

to produce a file all.csv with the test results (<scheme>,<test>,<seconds>).

The environment variable CPU_TIME can be set, to limit the time for compilation and for running (each individually).

CPU_TIME=3600 ./bench chez all

You can set an environment variable to specify the path to each Scheme.

LARCENY=/tmp/larceny/larceny MIT=/tmp/mit/mit-scheme ./bench "mit larceny" "fib ack"

The chicken test code assumes the presence of

Install them by running

chicken-install vector-lib r7rs

Install the r7rs package installation-wide.

Run:

sudo raco pkg install --scope installation r7rs

Precompile system libraries (optional). It is still going to be slow.

echo "(compile-system-libraries)" | mono IronScheme.Console-v4.exe

The default binary for IronScheme (can be changed by exporting IRONSCHEME in your shell before running bench) is called ironscheme.

If you unpack IronScheme in your home, you can create a shell script like the following:

#!/bin/bash
mono ~/IronScheme/IronScheme.Console-v4.exe -nologo -emacs $*

and put that somewhere in your $PATH.

Before calling Stalin, the code is run through alexpander (by default using Chicken’s csi). Then stalin is invoked on the result.

Missing a lot of necessary functions, read for example.

Problems with redirecting input, read chokes. No timing functions.