Scaling Continuous Latent Variable Models as Probabilistic Integral Circuits

This repository contains the code for reproducing the experiments of the paper Scaling Continuous Latent Variable Models as Probabilistic Integral Circuits, which has been accepted at NeurIPS 2024 as a spotlight. The repository can be cloned with

git clone --recurse-submodules https://github.com/gengala/ten-pics

Our implementation is also available in cirkit, which also provides an introductory notebook.

@inproceedings{
    gala2024scaling,
    title={Scaling Continuous Latent Variable Models as Probabilistic Integral Circuits},
    author={Gala, Gennaro and de Campos, Cassio and Vergari, Antonio and Quaeghebeur, Erik},
    booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
    year={2024},
    url={https://openreview.net/forum?id=Ke40kfOT2E}
}

Running experiments

We report below some example experiments.

MNIST-like datasets

python training_pic.py  --dataset mnist --rg QT --inner-layer cp     --k 512 --lr 0.005 --t0 500 --weight-decay 1e-2
python training_pic.py  --dataset mnist --rg QG --inner-layer cp     --k 512 --lr 0.005 --t0 500 --weight-decay 1e-2
python training_pic.py  --dataset mnist --rg QG --inner-layer tucker --k 64  --lr 0.005 --t0 500 --weight-decay 1e-2

python training_pc.py   --dataset mnist	--rg QT --inner-layer cp     --k 512 --lr 0.01
python training_pc.py   --dataset mnist	--rg QG --inner-layer cp     --k 512 --lr 0.01
python training_pc.py   --dataset mnist	--rg QG --inner-layer tucker --k 64  --lr 0.01

CIFAR

python training_pic.py --dataset cifar  --rg QG --inner-layer cp --k 512 --lr 0.005 --t0 500 --weight-decay 1e-2 --ycc lossy --min-delta 10

python training_pc.py --dataset cifar	--rg QG --inner-layer cp --k 512 --lr 0.01 -sil --ycc lossy --min-delta 10

CelebA

python training_pic.py  --dataset celeba --rg QG --inner-layer cp --k 512 --lr 0.005 --t0 500 --weight-decay 1e-2 --valid-freq 750 --ycc lossy --min-delta 10

python training_pc.py   --dataset celeba --rg QG --inner-layer cp --k 512 --lr 0.01 -sil --valid-freq 750 --ycc lossy --min-delta 10

imagenet32 & imagenet64

To use imagenet32 and imagenet64, please download them from https://image-net.org/download-images. For each dataset, concatenate the batches and create two pytorch uint8 matrices called train.pt and valid.pt, both of shape (#instances, #features). Place these matrices in data/imagenet32 and data/imagenet64.

python training_pic.py --dataset imagenet64 --k 256 --rg QG --inner-layer cp  --lr 0.005 --valid-freq 2000 --t0 500 --weight-decay 1e-2  --ycc lossy --min-delta 30

python training_pc.py  --dataset imagenet64 --k 256 --rg QG --inner-layer cp  --lr 0.01 -sil --valid-freq 2000 --ycc lossy --min-delta 30