baaivision/EVA

Error when loading&training EVAVisionTransformer with patch_dropout=0.5

chenmc1996 opened this issue · 0 comments

chenmc1996 commented

Hi, thanks for the good EVA-clip.
I'm loading&training the EVAVisionTransformer in EVA02-CLIP-L-14 and find that setting the patch_dropout to 0.5 raise runtimeError The size of tensor a (128) must match the size of tensor b (256) at non-singleton dimension 2 in

EVA/EVA-CLIP/rei/eva_clip/rope.py

Line 137 in b947f3f

return t * self.freqs_cos + rotate_half(t) * self.freqs_sin

setting it to zero seem fine.
Do you have any idea of how to fix it?
Thanks!