lucidrains/pixel-level-contrastive-learning

Does using RandomResizedCrop/HorizontalFlip in the augmentaion pipeline break the pixel alignment?

Closed this issue · 4 comments

cjrd commented

I may be reading the code incorrectly, but the line

image_one, image_two = self.augment1(x), self.augment2(x)

return images with two different random scale and crops and possible a horizontal flip.
However, the cutout coordinates are determined on the original image:

 cutout_coordinates_one, _ = cutout_coordinates(x)
 cutout_coordinates_two, _ = cutout_coordinates(x)

but after the RandomResizedCrop+random horizontal flip, image_one and image_two no longer share the same coordinates as the original image x, and it looks like the pixel coordinate alignment assumes that it does? Or at least, I'm not sure how the code could take into account the transformations from the augmentations pipeline to align the pixels.

lucidrains commented

@cjrd omg, yes I believe you are right, fixed in the latest version https://github.com/lucidrains/pixel-level-contrastive-learning/releases/tag/0.0.3

i don't have the random horizontal flip yet, but i'll add that tonight as well (the coordinates will need to be also flipped)

cjrd commented

Ah great - this looks good to me: thanks for the quick response!

  • 🚀1
lucidrains commented

@cjrd thank you Colorado!