jeremiedbb/scikit-learn_benchmarks

Benchmark suite for scikit-learn performances

scikit-learn_benchmarks

Benchmark suite for scikit-learn performances using airspeed velocity.

To get started,

• Create a conda environment and activate it and install pip in it (you can get conda by installing miniconda).

• Install asv (airspeed velocity):

pip install asv

• Clone this repo:

git clone https://github.com/jeremiedbb/scikit-learn_benchmarks.git
cd scikit-learn_benchmarks

• To run the full benchmark suite (warning: this can take several hours) use the following (see the docs to change the environments config or run only selected benchmarks):

asv run -b _bench

• To run the benchmarks for a specific model, use the -b <modelname> flag, e.g.:

asv run -b KMeans

• You can configure the benchmarks by editing the benchmark/config.json file.

• To publish the results in html format, run asv publish and asv preview

Special instructions to run the benchmarks with the daal4py patches of scikit-learn:

• Create a conda environment with scikit-learn, joblib, pillow and daal4py installed.
• Install asv with pip install git+git://github.com/jeremiedbb/asv.git@commit-label (see this PR)
• To benchmark scikit-learn vanilla, run:

asv run --python=same --commit-label=vanilla_sklearn -b _bench

• To benchmark scikit-learn with the patches, first edit the benchmarks/__init__.py file with:

import daal4py.sklearn
daal4py.sklearn.patch_sklearn()

• Then run:

asv run --python=same --commit-label=daal4py_sklearn -b _bench

Compare benchmarks

• To get a quick overview of the comparison of 2 benchmarks, run:

asv compare commit_hash_1 commit_hash_2

Commit hashes can be replaced by labels if you run the benchmarks against an existing environment (see above), for instance:

asv run --python=same --commit-label=daal4py_sklearn -b _bench

• To get a csv file of the results run:

python compare.py commit_hash_1 commit_hash_2

Commit hashes can be replaced by labels also.