Author: A/Prof Mark D. McDonnell, University of South Australia Email: mark.mcdonnell@unisa.edu.au Date: April 2016 Citation: If you find this code useful, please read and cite the following papers: [1] M. D. McDonnell, M. D. Tissera, T. Vladusich, A. van Schaik and J. Tapson. Fast, simple and accurate handwritten digit classification by training shallow neural network classifiers with the "extreme learning machine" algorithm. PLOS One, 10: Article Number e0134254, 2015. [2] M. D. Tissera and M. D. McDonnell. Modular Expansion of the Hidden Layer in Single Layer Feedforward Neural Networks. Accepted for Proc. IJCNN, Vancouver, 2016. This toolbox provides code useful for implementing computation of least squares optimal computation of weights matrices, such as for application in single-hidden layer neural networks with a linear output layer, applied to classifying images, as in refs [1] and [2] above. The main computation required is the optimisation of output weights to minimise mean square error between training targets and data. Several methods for doing this are provided. Several example scripts are also provided, showing application to classification of the standard MNIST image database. 1. The generally simplest method is to solve for all training data in a single batch using linear equations using a Cholesky solver. 2. The main deficiency in the single-batch Cholesky method is its high memory burden for large hidden layers. Four methods here can circumvent this: the conjugate gradient method, the modular method, the epsilon-NLMS method and the `Big Data' method. 3. The standard Cholesky method also is not good for non-stationary data. The RLS and epsilon-NLMS methods are better for non-stationary data, as in the OPIUM and OPIUM-light methods (see comments in E_NLMS.m). 4. The modular method offers several advantages: less memory intensive, and also faster than the standard method for the same total number of hidden units. However it loses some accuracy and generally a higher number of hidden units is necessary to get close to the same performance. 5. The 'Big Data' method simply iteratively computes the Gram matrix for use in the standard Cholesky solver approach.
McDonnell-Lab/Matlab-ELM-toolbox
Matlab code for implementing single layer ELM networks using various methods
MatlabNOASSERTION