- This repository is by Brandon Amos and contains the full source code and data to produce my thesis document.
- The slides are available in pdf and pptx format.
- Chapter 2 provides some preliminaries and background information on differentiable convex optimization layers, including derivations for the optimization (or variational) viewpoints of the ReLU, sigmoid, and softmax.
- Chapter 7 presents an early version of differentiable CVXPY layers, which is now available here. As a bibliographic note, the cone program differentiation derivation in section 7.3 here remains unpublished in this thesis and was done concurrent to and independent of Differentiating Through a Cone Program.
Some of the content here is behind these publications:
|
Differentiable Convex Optimization Layers A. Agrawal*, B. Amos*, S. Barratt*, S. Boyd*, S. Diamond*, and J. Kolter* NeurIPS 2019 [1] [pdf] [code] |
|
Differentiable MPC for End-to-end Planning and Control B. Amos, I. Rodriguez, J. Sacks, B. Boots, and J. Kolter NeurIPS 2018 [2] [pdf] [code] |
|
Depth-Limited Solving for Imperfect-Information Games N. Brown, T. Sandholm, and B. Amos NeurIPS 2018 [3] [pdf] |
|
Learning Awareness Models B. Amos, L. Dinh, S. Cabi, T. Rothörl, S. Colmenarejo, A. Muldal, T. Erez, Y. Tassa, N. de Freitas, and M. Denil ICLR 2018 [4] [pdf] |
|
Task-based End-to-end Model Learning P. Donti, B. Amos, and J. Kolter NeurIPS 2017 [5] [pdf] [code] |
|
OptNet: Differentiable Optimization as a Layer in Neural Networks B. Amos and J. Kolter ICML 2017 [6] [pdf] [code] |
|
Input Convex Neural Networks B. Amos, L. Xu, and J. Kolter ICML 2017 [7] [pdf] [code] |
|
Collapsed Variational Inference for Sum-Product Networks H. Zhao, T. Adel, G. Gordon, and B. Amos ICML 2016 [8] [pdf] |
|
OpenFace: A general-purpose face recognition library with mobile applications B. Amos, B. Ludwiczuk, and M. Satyanarayanan CMU 2016 [9] [pdf] [code] |
The experimental source code and libraries produced for this thesis are freely available as open source software and are available in the following repositories.
- [cvxgrp/cvxpylayers] Differentiable convex optimization layers in CVXPY.
- [locuslab/mpc.pytorch] A stand-alone PyTorch library for the differentiable model predictive control approach.
- [locuslab/differentiable-mpc] PyTorch experiments for the differentiable MPC work.
- [locuslab/qpth]: A stand-alone PyTorch library for the OptNet QP layers.
- [locuslab/optnet] PyTorch experiments for OptNet.
- [locuslab/icnn] TensorFlow experiments for input-convex neural networks.
- [cmusatyalab/openface] Face recognition with deep neural networks.
- [bamos/block] An intelligent block matrix library for numpy, PyTorch, and beyond.
- [bamos/dcgan-completion.tensorflow] Image Completion with Deep Learning in TensorFlow.
- [bamos/densenet.pytorch] A PyTorch implementation of DenseNet.
- This repository started from Cyrus Omar's thesis code, which is based on a CMU thesis template by David Koes and others before.
- Of standalone interest, refs.sort.sh uses biber to alphabetize and standardize my bibliography in refs.bib so it doesn't get too messy. This uses the configuration in refs.conf.
- I use update-pdf.sh to keep the latest PDF only in HEAD, although Git LFS or a related project may be a better solution.
The BibTeX for this document is:
@phdthesis{amos2019differentiable,
author = {Brandon Amos},
title = {{Differentiable Optimization-Based Modeling for Machine Learning}},
school = {Carnegie Mellon University},
year = 2019,
month = May,
}