/adaptive-regularization

Hyperparameter-free weight decay implemented in TensorFlow.

Primary LanguagePythonApache License 2.0Apache-2.0

Adaptive Regularization

This is a work in progress.

Instructions

Run python ptb.py --reg <reg_arg> where <reg_arg> is one of none, static, or adaptive. If static is selected, the network will be run with dropout, following the design in Recurrent Neural Network Regularization by Zarembra et al. (2014)

Coming Soon

  • ArXiV paper
  • Experiments on CIFAR-10/100 with LeNet, VGG and residual networks.