This is a Collective Knowledge repository providing functionality for portable, customizable, eproducible and automated experimental workflows. It lets users automatically detect multiple versions of different software (compilers, libraries, tools, models, data sets) using CK software detection plugins or install missing packages in a unified way across diverse hardware with Linux, Windows, MacOS and Android operating systems. It also allows users to collect information about a given platform in a unified way.
Further info:
- First CK steps
- CK portable experimental workflows
- CK documentation including "Getting Started Guide"
- Reproducible SW/HW co-design competitions for deep learning and other emerging workloads using CK
- General Motors and dividiti use CK to crowdsource benchmarking and optimization of deep learning with different versions of compilers, libraries, models and data sets: public CK repo
- ARM and the cTuning foundation use CK to systematize SW/HW co-design: HiPEAC Info'45 page 17, ARM TechCon'16 presentation and demo, public CK repo
- BSD, 3-clause
The minimal installation requires:
- Python 2.7 or 3.3+ (limitation is mainly due to unitests)
- Git command line client.
You can install CK in your local user space as follows:
$ git clone http://github.com/ctuning/ck
$ export PATH=$PWD/ck/bin:$PATH
$ export PYTHONPATH=$PWD/ck:$PYTHONPATH
You can also install CK via PIP with sudo to avoid setting up environment variables yourself:
$ sudo pip install ck
First you need to download and install a few dependencies from the following sites:
- Git: https://git-for-windows.github.io
- Minimal Python: https://www.python.org/downloads/windows
You can then install CK as follows:
$ pip install ck
or
$ git clone https://github.com/ctuning/ck.git ck-master
$ set PATH={CURRENT PATH}\ck-master\bin;%PATH%
$ set PYTHONPATH={CURRENT PATH}\ck-master;%PYTHONPATH%
$ ck pull repo:ck-env
$ ck list soft
$ ck list package
You can easily detect and register in the CK all the instances of GCC and LLVM as follows:
$ ck detect soft:compiler.gcc
$ ck detect soft:compiler.llvm
You can now see multiple versions of the detected software registered in the CK as follows:
$ ck show env
You can then compile and run unified CK benchmarks shared by the community using any of the above compiler instances (GCC, LLVM, ICC ...) and their versions simply as follows:
$ ck pull repo:ck-autotuning
$ ck pull repo:ctuning-programs
$ ck compile program:cbench-automotive-susan --speed
$ ck run program:cbench-automotive-susan
If you have Android NDK and SDK installed, CK can automatically detect it together with compiler versions (GCC, LLVM), register them and let you compile and run benchmarks on Android simply via:
$ ck compile program:cbench-automotive-susan --speed --target_os=android21-arm-v7a
$ ck run program:cbench-automotive-susan --target_os=android21-arm-v7a
You can find further details about our customizable and cross-platform package/environment manager here.
The concepts have been described in the following publications:
@inproceedings{ck-date16,
title = {{Collective Knowledge}: towards {R\&D} sustainability},
author = {Fursin, Grigori and Lokhmotov, Anton and Plowman, Ed},
booktitle = {Proceedings of the Conference on Design, Automation and Test in Europe (DATE'16)},
year = {2016},
month = {March},
url = {https://www.researchgate.net/publication/304010295_Collective_Knowledge_Towards_RD_Sustainability}
}
All CK-related publications: link
You are welcome to get in touch with the CK community if you have questions or comments!