/deep-belief-networks-fast-learning

Implementation of 'Fast Learning Algorithm for Deep Belief Networks' by Hinton et al., demonstrating unsupervised pre-training with RBMs and DBNs for effective deep neural network training.

Primary LanguagePython

Fast Learning for Deep Belief Networks

Implementation of the seminal paper "A Fast Learning Algorithm for Deep Belief Nets" by Geoffrey Hinton, Simon Osindero, and Yee-Whye Teh.

📖 Paper Summary

Deep Belief Networks (DBNs) are generative models composed of stacked Restricted Boltzmann Machines (RBMs). This project demonstrates:

  • Greedy layer-wise pre-training
  • Contrastive Divergence for RBM training
  • Fine-tuning with backpropagation

Key contributions:

  • Solving the vanishing gradient problem
  • Effective weight initialization
  • Efficient unsupervised learning

🛠️ Implementation Details

  • RBM Layer: Binary and Gaussian units with Contrastive Divergence
  • DBN: Greedy layer-wise stacking of RBMs
  • Optimization: Adam optimizer with early stopping based on reconstruction error
  • Fine-tuning: Supervised backpropagation (optional)

🖼️ Results

  • Significant improvements with pre-training
  • Visualizations of reconstructed images
  • Comparison of performance with/without pre-training

📦 Usage

# Clone the repo
git clone https://github.com/adityagh006/deep-belief-networks-fast-learning.git
cd deep-belief-networks-fast-learning

# Install dependencies
pip install -r requirements.txt

# Run the training script
python train_dbn.py