Microbenchmarking collection, functional APIs of the VM multiparadigm languages Java 8, Scala, C#, F# on Windows and on Linux.
The results are discussed in our ICOOOLPS'14 workshop paper:
A snapshot of the benchmarks in zip-file:
http://www.di.uoa.gr/~biboudis/clashofthelambdas-icooolps14-benchmarks.zip
To run the benchmarking suite as is, you will need a system with approximately 769mb of free space for heap allocation. Regarding execution time, a run on a single platform takes approximately 15-20 minutes on an Intel Core i5.
- Java 8 (jvm / jmh)
- sequential
- parallel
- Scala (jvm / jmh)
- sequential
- parallel
- optimized sequential/parallel with ScalaBlitz
- C# (clr & mono / Lambda Microbenchmarking1)
- sequential
- parallel
- optimized sequential/parallel with LinqOptimizer
- F# (clr & mono / Lambda Microbenchmarking1)
- sequential
- parallel
- optimized sequential/parallel with LinqOptimizer
- Install JDK8. Create the
JAVA_HOME
env variable (for maven to work) and add toPath
theJAVA_HOME\bin
directory - Install Gnu Make for Windows and put the
bin
directory toPATH
- Extract Maven and put the
bin
directory toPATH
- Install the C#, F# compilers (if you have Visual Studio 2013 you already have them)
- Update the paths in
Makefile
- Install/Extract JDK8 and update PATH
sudo apt-get install make
sudo apt-get install maven
- Follow the instructions on fsharp.org
- Compile all tests with
make
- Clean with
make clean
- Compile a specific suite with
make {java, csharp, fsharp, scala}
- Run microbenchmarks with
make bench-{java, csharp, fsharp, scala}
(e.g.,make bench-java
).
If you want to process the unified results we offer a gawk script (that also makes use of the dos2unix command to convert windows result file encoding).
- Run the benchmark suite with on both windows and linux with
make bench > results.{windows, linux}
(any filename). - See the unified results for all languages, platforms and benchmarks (the comma
separated values are benchmark, mean, mean error, sdev) by running
./process results.linux results.windows > results.processed
(make the script executable first).
@biboudis, @NickPalladinos and Y. Smaragdakis
Footnotes:
- A small utility that was factored out as a seperate project.