DCLW bug?

[tangzhy]

In DCLW, your code is like:

weight_fn = lambda z1, z2: 2 - z1.size(0) * torch.nn.functional.softmax((z1 * z2).sum(dim=1) / sigma, dim=0).squeeze()

I think the right way shall be like:

weight_fn = lambda z1, z2: 2 - torch.nn.functional.softmax((z1 * z2).sum(dim=1) / sigma, dim=0).squeeze()

z1.size(0) is not a variable introduced in the origin paper.

What do you think of it?

[raminnakhli]

Hi Zhengyang,
Thanks for the feedback. Actually, for the DCLW loss, the paper uses the expectation of the exp(z1, z2) in the denominator, which is basically the mean over these values. So, that is why I've multiplied the results with z1.size(0) (the number of elements that we're applying the mean on).
Please let me know if this doesn't make sense.

[raminnakhli]

I'll close this issue as there hasn't been an activity for a while. Please open another issue if the question still stands.