input-inference-for-control

Approximate Inference for Stochastic Optimal Control

What is it?

Input Inference for Control (i2c) is an inference-based optimal control algorithm. The current implementation, Gaussian i2c, can perform trajectory optimization, model predictive control and covariance control via a Gaussian approximation of the optimal state-action distribution. This yields time-varying linear (Gaussian) controllers, and is approximately equivalent to quadratic optimal control methods like differential dynamic programming, iterative-/sequential LQR.

For more information, see the following papers:

[1] J. Watson and H. Abdulsamad and R. Findeisen and J. Peters. Stochastic Control through Approximate Bayesian Input Inference. Submitted to IEEE Transactions on Automatic Control Special Issue, Learning and Control 2021. (arXiv)

[2] J. Watson and J. Peters. Advancing Trajectory Optimization with Approximate Inference: Exploration, Covariance Control and Adaptive Risk. American Control Conference (ACC) 2021 (arXiv)

[3] J. Watson and H. Abdulsamad and J. Peters. Stochastic Optimal Control as Approximate Input Inference. Conference on Robot Learning (CoRL) 2019. (arXiv)

Installation

Create environment i2c and install

    cd input-inference-for-control && conda create -y -n i2c pip python=3.7 && conda activate i2c && pip3 install -r requirements.txt && pip install -e .

Example

To optimize pendulum swing-up with cubature quadrature, run

    python scripts/i2c_run.py pendulum_known_quad

the output directory results should look like this

Experiments

Prior experiments are preserved here. All results are stored in /_results.

LQR Equivalence

Section 3.1 of [3]

    python scripts/LQR_compare.py

Nonlinear Trajectory Optimization

Section 3.2 of [3], Section IV.A of [1]

    python scripts/i2c_run.py -h

results are in _results/

Linear Gaussian Covariance Control

Linear Gaussian covariance control. Section IV.C of [2]

    python scripts/linear_covariance_control.py

Nonlinear Gaussian Covariance Control

Pendulum swing-up with covaraince control. Section IV.C of [2]

    python scripts/nonlinear_covariance_control.py

Model Predictive Control with State Estimation

Runs i2c and iLQR MPC with a cubature kalman filter for an acrobatic quadropter task. Section IV.C of [1]

    python scripts/mpc_state_est/mpc_quad.py 0 --plot

Baselines

For iLQR use https://github.com/hanyas/trajopt

Citing Input Inference for Control

To cite i2c, please reference the appropriate paper

@misc{watson2021stochastic,
      title={Stochastic Control through Approximate Bayesian Input Inference}, 
      author={Watson, Joe and Abdulsamad, Hany and Findeisen, Rolf and Peters, Jan},
      year={2021},
      eprint={2105.07693},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

@inproceedings{i2cacc,
	author    = {Watson, Joe and Peters, Jan},
	title     = {Advancing Trajectory Optimization with Approximate Inference: Exploration, Covariance Control and Adaptive Risk},
	booktitle = {American Control Conference},
	year      = {2021},
}

@inproceedings{i2ccorl,
	author    = {Watson, Joe and  Abdulsamad, Hany and Peters, Jan},
	title     = {Stochastic Optimal Control as Approximate Input Inference},
	booktitle = {Conference on Robot Learning},
	year      = {2019},
}

JoeMWatson/input-inference-for-control