openai/pixel-cnn

not understand "log probability in the center of the bin"

Beronx86 opened this issue · 2 comments

Could anybody explain the default values to be used in extreme cases.

68    # log probability in the center of the bin, to be used in extreme cases
69    # (not actually used in our code)
70    log_pdf_mid = mid_in - log_scales - 2. * tf.nn.softplus(mid_in)

@Beronx86 check the definition logistic distribution on wikipedia.
log_pdf_mid is the log pdf of logistic distribution.

why this is used in extreme case? particularly:

inner_inner_cond = (cdf_delta > 1e-5).float()
inner_inner_out  = inner_inner_cond * torch.log(torch.clamp(cdf_delta, min=1e-12)) + (1. - inner_inner_cond) * (log_pdf_mid - np.log(127.5))
inner_cond       = (x > 0.999).float()
inner_out        = inner_cond * log_one_minus_cdf_min + (1. - inner_cond) * inner_inner_out
cond             = (x < -0.999).float()
log_probs        = cond * log_cdf_plus + (1. - cond) * inner_out
log_probs        = torch.sum(log_probs, dim=3) + log_prob_from_logits(logit_probs)

why is log_pdf_mid - np.log(127.5)?