/Weight-Sharing

The source code associated with the paper, Learning in the Machine: To Share or not to Share?

Primary LanguageJupyter Notebook

Learning in the Machine: To Share or not to Share?

This repository is associated with our paper.

@article{ott2020learning,
  title={Learning in the machine: To share or not to share?},
  author={Ott, Jordan and Linstead, Erik and LaHaye, Nicholas and Baldi, Pierre},
  journal={Neural Networks},
  year={2020},
  publisher={Elsevier}
}

Pipeline

  1. Hyperparameter grid search
  • Number of layers
  • Learning rate
  1. Full experiments
  • Types of augmentation
    • Edge noise
    • Translation
    • Rotation
    • Noise
    • Quadrant swap
  1. Results
  • Metrics
    • Loss
    • Validation loss
    • Training accuracy
    • Validation accuracy
    • Edge noise augmented validation accuracy
    • Translation augmented validation accuracy
    • Noise augmented validation accuracy
    • Rotation augmented validation accuracy
  • Approximate weight sharing
    • Observing the distance between free convolutional filters within a layer
  • Variable Connection Patterns
    • Varying the probability