Layer-sequential unit-variance (LSUV) initialization
This is sample code for LSUV and Orthonormal initializations, implemented in python script within Caffe framework. The original Caffe README is reproduced below the line. LSUV initialization is described in:
Mishkin, D. and Matas, J.,(2015). All you need is a good init. arXiv preprint arXiv:1511.06422.
Orthonormal initialization is described in:
Andrew M. Saxe, James L. McClelland, Surya Ganguli (2013). Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. arXiv preprint arXiv:1312.6120
Examples
See examples and sample log outputs in examples/lsuv. Run get_datasets.sh and training_*.sh for experiments reproduction. Note, than we haven`t freezed random seed and results may vary a little from run to run. Initialization script itself is in tools/extra/lsuv_init.py
Citation
Please cite us if you use this code:
@ARTICLE{LSUVInit2015,
author = {{Mishkin}, D. and {Matas}, J.},
title = "{All you need is a good init}",
journal = {arXiv preprint arXiv:1511.06422},
year = 2015,
month = nov
}
Caffe
Caffe is a deep learning framework made with expression, speed, and modularity in mind. It is developed by the Berkeley Vision and Learning Center (BVLC) and community contributors.
Check out the project site for all the details like
- DIY Deep Learning for Vision with Caffe
- Tutorial Documentation
- BVLC reference models and the community model zoo
- Installation instructions
and step-by-step examples.
Please join the caffe-users group or gitter chat to ask questions and talk about methods and models. Framework development discussions and thorough bug reports are collected on Issues.
Happy brewing!
License and Citation
Caffe is released under the BSD 2-Clause license. The BVLC reference models are released for unrestricted use.
Please cite Caffe in your publications if it helps your research:
@article{jia2014caffe,
Author = {Jia, Yangqing and Shelhamer, Evan and Donahue, Jeff and Karayev, Sergey and Long, Jonathan and Girshick, Ross and Guadarrama, Sergio and Darrell, Trevor},
Journal = {arXiv preprint arXiv:1408.5093},
Title = {Caffe: Convolutional Architecture for Fast Feature Embedding},
Year = {2014}
}