/ra-hmc

Code for Repelling-Attracting Hamiltonian Monte Carlo (https://arxiv.org/abs/2403.04607)

Primary LanguageJupyter Notebook

RAHMC

This repository contains the code accompanying:

Repelling-Attracting Hamiltonian Monte Carlo
Siddharth Vishwanath and Hyungsuk Tak
arXiv:2403.04607

Getting Started

To get started, first clone the repository and start Julia.

$ git clone https://github.com/sidv23/ra-hmc.git
$ cd ./ra-hmc
$ julia

From the Julia REPL, you can enter the package manager by typing ], and activate the project environment with the required packages from the Project.toml file as follows.

julia> ]
pkg> activate .
pkg> instantiate

Alternatively, if you use the DrWatson.jl package, then you can quickly activate the project environment as follows.

julia> using DrWatson
julia > @quickactivate "ra-hmc"

If you use Github Codespaces, the .devcontainer/devcontainer.json file is configured to setup the right Julia environment for you on initialization.

Contents

The notebooks directory contains the Jupyter notebooks for the experiments and simulations. The directory contains the following files:

  • unimodal.ipynb: Comparison of RAHMC and HMC mixing for an standard (unimodal and isotropic) Gaussian distributions when $d \in {2, 5, 10, 50, 100}$.

  • benchmark.ipynb: Comparison of RAHMC, HMC, RAM and PEHMC for the benchmark dataset from Kou et al. (2005) comprising of a mixture of 20 Gaussian distributions in 2 dimensions.

  • circles-concentric.ipynb and circles-nested.ipynb: Comparison of RAHMC, HMC, RAM, PEHMC and Wormhole HMC for distributions with "higher dimensional modes" supported on the boundary of $\ell_1$ balls in $\mathbb{R}^d$

  • anisotropic.ipynb: Comparison of RAHMC, HMC, RAM and PEHMC for a mixture of two Gaussian distributions with correlation structure in high dimensions.

  • funnel.ipynb: Comparison of RAHMC and HMC for a mixture of two funnel distributions (Neal, 2003).

  • tuning.ipynb: Illustration of Nesterov dual-averaging for tuning the parameters of RAHMC in high dimensions.

  • bayesian_nn.ipynb: Comparison of RAHMC and HMC for training a Bayesian neural network with known multimodality.

The scripts directory contains the .jl source-code for the notebooks.

Troubleshooting

The code here uses Julia v1.10. For any issues, please click here.