Implementation of Contrastive Loss
ChenPaulYu opened this issue · 1 comments
ChenPaulYu commented
Thanks for your great work, the code is clean and it is work appropriate.
However, I got some problem about your implementation about contrastive loss. Is it possible to explain it more details, because it is quite different with the description of original paper.
In the mean time, I try some contrastive loss implementation by others, but it is not work, is it possible to explain the difference?
# Custom Contrastive Loss
class ContrastiveLoss(torch.nn.Module):
"""
Contrastive loss function.
Based on: http://yann.lecun.com/exdb/publis/pdf/hadsell-chopra-lecun-06.pdf
"""
def __init__(self, margin=2.0):
super(ContrastiveLoss, self).__init__()
self.margin = margin
def forward(self, output1, output2, label):
euclidean_distance = F.pairwise_distance(output1, output2)
loss_contrastive = torch.mean((1-label) * torch.pow(euclidean_distance, 2) +
(label) * torch.pow(torch.clamp(self.margin - euclidean_distance, min=0.0), 2))
return loss_contrastive
adambielski commented
I don't see how my implementation is different from the paper. The snippet you provide has a different order of power/clamping than the original.