About the Derivation of Variational Lower Bound in Appendix A

Question

About the Derivation of Variational Lower Bound in Appendix A

Hannofdhu opened this issue 3 years ago · 3 comments

I read the derivation you provided in appendix A, but I don't know how the last approximation is obtained, I hope to get your answer, thank you very much

Answer 1 · 2022-03-08T12:02:31.000Z

It is very crude monte carlo approximation with sampling size 1 for z_x ~ q(z_x|x,c).

Answer 2 · 2022-03-22T06:59:31.000Z

Thank you for your answer, I still don't understand something. Regarding the lower bound of the evidence derived in the paper, the coefficients in front of the KL divergence are all -1, why in the code, their coefficients become 0.1. Looking forward to your answer, thank you

Answer 3 · 2022-07-05T04:34:09.000Z

Hi, sorry for the late reply.
If the coefficient is larger than -1 (beta=1 in the code), we cannot guarantee that it is lower bound of the marginal likelihood. But if we put the coefficient smaller than -1, we observe the posterior collapse. So, following [beta-vae],(https://openreview.net/forum?id=Sy2fzU9gl) we scale down the kl-divergence for training stability.