This project is a work-in-progress, which provides a framework to experiment heterogeneous neural network topologies. A preliminary release will be visible soon.
Creator: Jianbo Ye
- template vs. module
- neural network operators
- autoencoders (w or w/o tiled weight)
- activation functions: logistic, tanh, ReLU, softplus
- metrics (L1, L2, Mahalanobis, Softmax)
- regularization: weight decay, activation sparsity, dropout, maxout
- data parallel framework: atomic parameters + distributed states
- optimization: LBFGS, SGD, SAGD, SGD with momentum
- recursive neural network
- Basics
- Auto-Encoder
- Examples: Please see more examples under folder
src/main/scala/neuron/tutorials/ - Scaladoc: TBA
-
How is neuron different from other deep learning libraries (such as theano, torch7, etc), besides it is Scala based?
We argue that not only the number of parameters contributes to the representation ability of neural network, but also its infrastructure (network topology, train strategy, etc.) Neuron focuses on fast prototyping novel network architecture. Using Scala, we attempt to make the implementation of neural network in a mixed functional and imperative way ... neuron is not at the mature shape to be industrial proven.
-
How is the speed of neruon?
Neuron is currently backed by breeze for numerical computation, which should be fast. And the extra cost for data control flow is minimized. Neuron provides data parallelization, but does not support model parallelization.
- Breeze and Nak: a set of libraries for machine learning and numerical computing
- UFLDL Tutorial: a Stanford course, find solutions at Github
- DeepLearnToolbox: Matlab toolbox for deep learning
- Torch/nn: neural network modules backed by Torch
- ScalaSTM: a lightweight software transactional memory for Scala
The MIT License (MIT)
Copyright (c) 2014 Jianbo Ye
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.