Question about the scaling operation of score function of VP and VE
XikunHuang opened this issue · 1 comments
XikunHuang commented
Hello, thanks for your amazing work!
I am wondering why scale neural network output by standard deviation and flip sign
in VP score function, and NOT in VE score function?
Thanks a lot!
def score_fn(x, t):
# Scale neural network output by standard deviation and flip sign
if continuous or isinstance(sde, sde_lib.subVPSDE):
# For VP-trained models, t=0 corresponds to the lowest noise level
# The maximum value of time embedding is assumed to 999 for
# continuously-trained models.
labels = t * 999
score = model_fn(x, labels)
std = sde.marginal_prob(torch.zeros_like(x), t)[1]
else:
# For VP-trained models, t=0 corresponds to the lowest noise level
labels = t * (sde.N - 1)
score = model_fn(x, labels)
std = sde.sqrt_1m_alphas_cumprod.to(labels.device)[labels.long()]
########################################
score = -score / std[:, None, None, None]
###########################################
return score
def score_fn(x, t):
if continuous:
labels = sde.marginal_prob(torch.zeros_like(x), t)[1]
else:
# For VE-trained models, t=0 corresponds to the highest noise level
labels = sde.T - t
labels *= sde.N - 1
labels = torch.round(labels).long()
score = model_fn(x, labels)
return score
solitaryTian commented
Hello! I am also interested in this problem. Have you solved it?