jbalma/BoltzmannBaby

BoltzmannBaby is a C/C++ OpenMP-4.0 RBM-based deep learning research code used to understand the underlying thermodynamic properties of deep networks in terms of Temperature, Pressure and Volume -- as well as energy density and entropy currents. It uses general binary input structures by way of binning up (mostly) arbitrary 2-d data structures. Almost any 2-d data can be used as inputs through the use of the binning mechanism. Two binning subroutines are currently available: one for time-series or spatial location data (e.g. images or f(x) style data), and another for character-based data (raw text) in which characters are represented by the y-axis of the matrix, and the spatial location in a sentence or word is represented by the x axis. The current benchmark problems use either the character/text based data, or else a noisy sinusoidal function. The learning rate is automatically adjusted using a simple AdaGrad technique. The bias neurons are updated each epoch or kept fixed depending on user preference. A number of shifted sub-samples can be used to enlarge the data set (useful in text based learning). The default setup uses a set of early childhood reading samples, fables, Kafka short stories to train layer wise each RBM. Arbitrary numbers of additional RBMs can be stacked in the chain, but the default configuration uses eight. Full documentation is in the works.

C++