Linux/Mac OS |
Windows |
|
|---|---|---|
 |
   |
tiny-dnn is a C++11 implementation of deep learning. It is suitable for deep learning on limited computational resource, embedded systems and IoT devices.
- Features
- Comparison with other libraries
- Supported networks
- Dependencies
- Build
- Examples
- Contributing
- References
- License
- Mailing list
Check out the documentation for more info.
- 2016/9/14 tiny-dnn v1.0.0alpha is released!
- 2016/8/7 tiny-dnn is now moved to organization account, and rename into tiny-dnn :)
- 2016/7/27 tiny-dnn v0.1.1 released!
- reasonably fast, without GPU
- with TBB threading and SSE/AVX vectorization
- 98.8% accuracy on MNIST in 13 minutes training (@Core i7-3520M)
- portable & header-only
- Run anywhere as long as you have a compiler which supports C++11
- Just include tiny_dnn.h and write your model in C++. There is nothing to install.
- easy to integrate with real applications
- no output to stdout/stderr
- a constant throughput (simple parallelization model, no garbage collection)
- work without throwing an exception
- can import caffe's model
- simply implemented
- be a good library for learning neural networks
| tiny-dnn | caffe | Theano | TensorFlow | |
|---|---|---|---|---|
| Prerequisites | Nothing(Optional:TBB,OpenMP) | BLAS,Boost,protobuf,glog,gflags,hdf5, (Optional:CUDA,OpenCV,lmdb,leveldb etc) | Numpy,Scipy,BLAS,(optional:nose,Sphinx,CUDA etc) | numpy,six,protobuf,(optional:CUDA,Bazel) |
| Modeling By | C++ code | Config File | Python Code | Python Code |
| GPU Support | No | Yes | Yes | Yes |
| Installing | Unnecessary | Necessary | Necessary | Necessary |
| Windows Support | Yes | No* | Yes | No* |
| Pre-Trained Model | Yes(via caffe-converter) | Yes | No* | No* |
*unofficial version is available
- core
- fully-connected
- dropout
- linear operation
- power
- convolution
- convolutional
- average pooling
- max pooling
- deconvolutional
- average unpooling
- max unpooling
- normalization
- contrast normalization
- batch normalization
- split/merge
- concat
- slice
- elementwise-add
- tanh
- sigmoid
- softmax
- rectified linear(relu)
- leaky relu
- identity
- exponential linear units(elu)
- cross-entropy
- mean squared error
- mean absolute error
- mean absolute error with epsilon range
- stochastic gradient descent (with/without L2 normalization and momentum)
- adagrad
- rmsprop
- adam
Nothing. All you need is a C++11 compiler.
tiny-dnn is header-ony, so there's nothing to build. If you want to execute sample program or unit tests, you need to install cmake and type the following commands:
cmake .
Then open .sln file in visual studio and build(on windows/msvc), or type make command(on linux/mac/windows-mingw).
Some cmake options are available:
| options | description | default | additional requirements to use |
|---|---|---|---|
| USE_TBB | Use Intel TBB for parallelization | OFF* | Intel TBB |
| USE_OMP | Use OpenMP for parallelization | OFF* | OpenMP Compiler |
| USE_SSE | Use Intel SSE instruction set | ON | Intel CPU which supports SSE |
| USE_AVX | Use Intel AVX instruction set | ON | Intel CPU which supports AVX |
| USE_OPENCV | Use OpenCV for sample/test programs | ON | Open Source Computer Vision Library |
| BUILD_TESTS | Build unit tests | OFF | -** |
| BUILD_EXAMPLES | Build example projects | ON | - |
| BUILD_DOCS | Build documentation | OFF | Doxygen |
*tiny-dnn use c++11 standard library for parallelization by default
**to build tests, type git submodule update --init before build
For example, type the following commands if you want to use intel TBB and build tests:
cmake -DUSE_TBB=ON -DBUILD_EXAMPLES=ON .You can edit include/config.h to customize default behavior.
construct convolutional neural networks
#include "tiny_dnn/tiny_dnn.h"
using namespace tiny_dnn;
using namespace tiny_dnn::activation;
using namespace tiny_dnn::layers;
void construct_cnn() {
using namespace tiny_dnn;
network<sequential> net;
// add layers
net << conv<tan_h>(32, 32, 5, 1, 6) // in:32x32x1, 5x5conv, 6fmaps
<< ave_pool<tan_h>(28, 28, 6, 2) // in:28x28x6, 2x2pooling
<< fc<tan_h>(14 * 14 * 6, 120) // in:14x14x6, out:120
<< fc<identity>(120, 10); // in:120, out:10
assert(net.in_data_size() == 32 * 32);
assert(net.out_data_size() == 10);
// load MNIST dataset
std::vector<label_t> train_labels;
std::vector<vec_t> train_images;
parse_mnist_labels("train-labels.idx1-ubyte", &train_labels);
parse_mnist_images("train-images.idx3-ubyte", &train_images, -1.0, 1.0, 2, 2);
// declare optimization algorithm
adagrad optimizer;
// train (50-epoch, 30-minibatch)
net.train<mse>(optimizer, train_images, train_labels, 30, 50);
// save
std::ofstream ofs("weights");
ofs << net;
// load
// std::ifstream ifs("weights");
// ifs >> net;
}construct multi-layer perceptron(mlp)
#include "tiny_dnn/tiny_dnn.h"
using namespace tiny_dnn;
using namespace tiny_dnn::activation;
using namespace tiny_dnn::layers;
void construct_mlp() {
network<sequential> net;
net << fc<sigmoid>(32 * 32, 300)
<< fc<identity>(300, 10);
assert(net.in_data_size() == 32 * 32);
assert(net.out_data_size() == 10);
}another way to construct mlp
#include "tiny_dnn/tiny_dnn.h"
using namespace tiny_dnn;
using namespace tiny_dnn::activation;
void construct_mlp() {
auto mynet = make_mlp<tan_h>({ 32 * 32, 300, 10 });
assert(mynet.in_data_size() == 32 * 32);
assert(mynet.out_data_size() == 10);
}more sample, read examples/main.cpp or MNIST example page.
Since deep learning community is rapidly growing, we'd love to get contributions from you to accelerate tiny-dnn development! For a quick guide to contributing, take a look at the Contribution Documents.
[1] Y. Bengio, Practical Recommendations for Gradient-Based Training of Deep Architectures. arXiv:1206.5533v2, 2012
[2] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86, 2278-2324.
other useful reference lists:
The BSD 3-Clause License
google group for questions and discussions: