This repository contains the code to train neural nets uising Path-SGD optimization method:
Path-SGD: Path-Normalized Optimization in Deep Neural Networks
Behnam Neyshabur, Ruslan Salakhutdinov, Nathan Srebro
- Install Python 3.6 and PyTorch 0.4.1.
- Clone the repository:
git clone https://github.com/bneyshabur/path-sgd.git
- The following command trains a fully connected feedforward network with two hidden layer, each of which with 4000 hidden units on CIFAR10 dataset, using Path-SGD:
python main.py --dataset CIFAR10 --optimizer path-sgd
--no-cuda
: disables cuda training--datadir
: path to the directory that contains the datasets (default: datasets)--dataset
: name of the dataset (options: MNIST | CIFAR10 | CIFAR100 | SVHN, default: CIFAR10). If the dataset is not in the desired directory, it will be downloaded.--nunits
: number of hidden units (default: 4000)--optimizer
: name of the optimizer (options: sgd | path-sgd, default: sgd).