Implementation of BCD for DNNs in Global Convergence of Block Coordinate Descent in Deep Learning (Zeng et al., 2019) in PyTorch with the MNIST dataset (see also T. T.-K. Lau et al., 2018).
bcd_dnn_mlp_mnist.ipynb: 3-layer MLP, MNIST, BCD (PyTorch)
bcd_dnn_mlp_mnist_deep.ipynb: 10-layer MLP, MNIST, BCD (PyTorch)
sgd_dnn_mlp_mnist_deep_keras.ipynb: 10-layer MLP, MNIST, SGD (TensorFlow and Keras; adapted from this repo)
plot.ipynb: Produces plots in the paper
J. Zeng*, T. T.-K. Lau*, S. Lin and Y. Yao (2019). Global Convergence of Block Coordinate Descent in Deep Learning. In Proceedings of the 36th International Conference on Machine Learning (ICML).
*Equal contribution
T. T.-K. Lau, J. Zeng, B. Wu and Y. Yao (2018). A Proximal Block Coordinate Descent Algorithm for Deep Neural Network Training. In International Conference on Learning Representations (ICLR), Workshop Track.