Instructions for likelihood evaluation using AIS?

[meta-inf]

Hi,

Thanks for making your code available!

I'm having problem evaluating log likelihood on the MNIST dataset. When I run

python ais.py --exp exp-name --dataset mnist  --resume_iter 15000 --pdist 5000

the output is like

Positive sample probability  -0.00042194081 0.0062082466
Total values of lower value based off forward sampling -0.62608874 0.0075839674
Total values of upper value based off backward sampling -0.6260886 0.007481599

It seems that the code should output the test log likelihood, but the magnitude of the output doesn't match (the numbers in Fig 22 of your paper are around 10^2 to 10^3). This is strange since I've checked visually that the model is generating plausible samples. So is there anything wrong in my experiment configuration, or should I do some scaling to obtain the log likelihood figures in the table?

The model is obtained with

 --dataset=mnist --num_steps=60 --batch_size=128 --step_lr=10.0 --proj_norm=0.01 --zero_kl --replay_batch --epoch_num 50

[yilundu]

Hi, when evaluating the test log likelihood of the models, we actually use a different temperature (the energy function learns a scaled down version of the probability distribution). If you run the above code, but with

python ais.py --exp exp-name --dataset mnist  --resume_iter 15000 --pdist 5000 --temperature 10000

With some tuned temperature, you should obtain the reported log likelihoods.

[meta-inf]

It worked. Thanks.