I doubt the total loss of InfoNCE and DCL should be averaged?
newzhx opened this issue · 0 comments
newzhx commented
I think the total loss of InfoNCE as well as DCL should be averaged.
should be
and
should be
the code does this
return (positive_loss + negative_loss).mean()