Stacked autoencoder with noise and sparse penalties. Usual layer-wise pretraining and fine tuning. It requires cudamat package.
Example of learned features on raw images of cifar-10 (cute but mostly useless).
Pretraining using l0.bin, l1.bin activations.
python pretrain.py -f l0.bin -a l1.bin -o l0.par -l 0.1 -x 1000 -b 700 -e 60
python pretrain.py -f l1.bin -a l2.bin -o l1.par -l 0.1 -x 500 -b 700 -e 60
Create the full model.
cat l0.par l1.par > l01.par
Fine tune.
python tune.py -f l0.bin -d corpus.bin -o l01.par -l 0.01 -b 700 -e 60