/thesis

Differentiable Optimization-Based Modeling for Machine Learning

Primary LanguageTeX

Differentiable Optimization-Based Modeling for Machine Learning

Unpublished work in this thesis

  • Chapter 2 provides some preliminaries and background information on differentiable convex optimization layers, including derivations for the optimization (or variational) viewpoints of the ReLU, sigmoid, and softmax.
  • Chapter 7 presents an early version of differentiable CVXPY layers, which is now available here. As a bibliographic note, the cone program differentiation derivation in section 7.3 here remains unpublished in this thesis and was done concurrent to and independent of Differentiating Through a Cone Program.

Publications behind this thesis

Some of the content here is behind these publications:

Differentiable Convex Optimization Layers
A. Agrawal*, B. Amos*, S. Barratt*, S. Boyd*, S. Diamond*, and J. Kolter*
NeurIPS 2019
[1] [pdf] [code]
Differentiable MPC for End-to-end Planning and Control
B. Amos, I. Rodriguez, J. Sacks, B. Boots, and J. Kolter
NeurIPS 2018
[2] [pdf] [code]
Depth-Limited Solving for Imperfect-Information Games
N. Brown, T. Sandholm, and B. Amos
NeurIPS 2018
[3] [pdf]
Learning Awareness Models
B. Amos, L. Dinh, S. Cabi, T. Rothörl, S. Colmenarejo, A. Muldal, T. Erez, Y. Tassa, N. de Freitas, and M. Denil
ICLR 2018
[4] [pdf]
Task-based End-to-end Model Learning
P. Donti, B. Amos, and J. Kolter
NeurIPS 2017
[5] [pdf] [code]
OptNet: Differentiable Optimization as a Layer in Neural Networks
B. Amos and J. Kolter
ICML 2017
[6] [pdf] [code]
Input Convex Neural Networks
B. Amos, L. Xu, and J. Kolter
ICML 2017
[7] [pdf] [code]
Collapsed Variational Inference for Sum-Product Networks
H. Zhao, T. Adel, G. Gordon, and B. Amos
ICML 2016
[8] [pdf]
OpenFace: A general-purpose face recognition library with mobile applications
B. Amos, B. Ludwiczuk, and M. Satyanarayanan
CMU 2016
[9] [pdf] [code]

The experimental source code and libraries produced for this thesis are freely available as open source software and are available in the following repositories.


  • This repository started from Cyrus Omar's thesis code, which is based on a CMU thesis template by David Koes and others before.
  • Of standalone interest, refs.sort.sh uses biber to alphabetize and standardize my bibliography in refs.bib so it doesn't get too messy. This uses the configuration in refs.conf.
  • I use update-pdf.sh to keep the latest PDF only in HEAD, although Git LFS or a related project may be a better solution.


The BibTeX for this document is:

@phdthesis{amos2019differentiable,
  author       = {Brandon Amos},
  title        = {{Differentiable Optimization-Based Modeling for Machine Learning}},
  school       = {Carnegie Mellon University},
  year         = 2019,
  month        = May,
}