Companion code for Efficiently Sampling Functions from Gaussian Process Posteriors. This software has been provided on an "as is" basis and depends heavily upon GPflow.
git clone https://github.com/j-wilson/GPflowSampling
cd GPflowSampling
pip install -e .
To install the dependencies needed to run examples, use pip install -e .[examples].
Location-scale-based sampling is implemented for scalar output <gpflow.model.GPR> and <gpflow.model.SVGP>.
from gpflow_sampling.samplers import location_scale
model = gpflow.model.GPR(...)
sampler = location_scale(model, model.kernel, sample_shape=[...], full_cov=True)
Random Fourier features are currently available for Matérn kernels, e.g. <Matern52> and <SquaredExponential>.
from gpflow_sampling.utils import RandomFourierBasis
kernel = gpflow.kernels.Matern52()
rff = RandomFourierBasis(kernel, num_basis=1024)
Naïve Fourier-feature-based sampling is provided for <gpflow.model.GPR>.
from gpflow_sampling.samplers import finite_fourier
model = gpflow.model.GPR(...)
sampler = finite_fourier(model, model.kernel, sample_shape=[...], num_basis=5000)
Decouple sampling is supported for <gpflow.model.GPR> and <gpflow.model.SVGP>. Additional coverage is provided for some simple multi-output kernels.
from gpflow_sampling.samplers import decoupled
kernel = gpflow.kernels.LinearCoregionalization(...)
model = gpflow.model.SVGP(kernel=kernel, ...)
sampler = decoupled(model, model.kernel, sample_shape=[...], num_basis=500)
As a running example, assume that model is multioutput GP accepting 3-dimensional inputs and predicting 2-dimensional function values. For <decoupled> and <finite_fourier> samplers, the following tips apply:
- The third-to-last axis
axis=-3acts as a batch axis for sample paths:sampler = decoupled(model, model.kernel, sample_shape=[32], num_basis=500 from_2d = sampler(inputs=tf.random.uniform([100, 3])) # 32x100x2, broadcasted evaluation from_3d = sampler(inputs=tf.random.uniform([32, 100, 3])) # 32x100x2, pathwise evaluation - We recommend use of custom mean functions, since GPflow's standard options do not broadcast --- which will cause pathwise evaluations like that of
from_3d(above) to fail. sampler.reset_random_variables(reset_basis=True)can be used to resample paths without recreating<tf.Variable>. This is useful whenmodelmay have changed (see examples/training_via_sampling.ipynb).- When generating samples in batches, typically use
sampler.reset_random_variables(reset_basis=False):a = [] # draws from a Gaussian distribution b = [] # draws from a Gaussian mixture x = tf.random.uniform([100, 3]) for _ in range(10): sampler_a.reset_random_variables(reset_basis=False) a.append(sampler_a(x)) # drawn from the same Gaussian sampler_b.reset_random_variables(reset_basis=True) b.append(sampler_b(x)) # drawn from a new Gaussian
If our work helps you in a way that you feel warrants reference, please cite the following paper:
@inproceedings{wilson2020efficiently,
title={Efficiently sampling functions from Gaussian process posteriors},
author={James T. Wilson
and Viacheslav Borovitskiy
and Alexander Terenin
and Peter Mostowsky
and Marc Peter Deisenroth},
booktitle={International Conference on Machine Learning},
year={2020},
url={https://arxiv.org/abs/2002.09309}
}