jmtomczak/vae_vampprior

can we calculate KL divergence of VampPrior and posterior without sampling?

ShellingFord221 opened this issue · 0 comments

Hi, in line 78~80 of VAE.py, when calculating the KL divergence of VampPrior p(z) and posterior q(z|x), we put z_q into N(z_p_mean, z_p_logvar) and N(z_q_mean, z_q_logvar) then calculate the difference of log outputs of two Gaussian function.

log_p_z = self.log_p_z(z_q)

log_q_z = log_Normal_diag(z_q, z_q_mean, z_q_logvar, dim=1)

KL = -(log_p_z - log_q_z)

Since we have already get mean and variance of prior and posterior, can we directly calculate KL divergence of two Gaussian distribution? i.e.

image

This is because in line 226, z_q is just drawn from N(z_q_mean, z_q_logvar):

z_q = self.reparameterize(z_q_mean, z_q_logvar)

So can we skip this step? Thanks!