vision-transformer problem report

Question

vision-transformer problem report

ChenDaiwei-99 opened this issue a year ago · 4 comments

ChenDaiwei-99 commented a year ago

Answer 1 · 2023-09-06T18:58:56.000Z

If you wanna contribute some fixes would be happy to merge

Answer 2 · 2023-09-06T19:31:26.000Z

Sure, will do it in a few days :)

Answer 3 · 2023-10-28T01:46:52.000Z

I can confirm this example is badly broken. I added some code to compare individual labels to predictions and discovered the forward pass of ViT always returns the same tensor. No matter the input. The tensor it returns is different each time I run it, even if I load the same weights from the save file and don't do any training. It's no wonder it can't do better than 2.3. Always giving the same prediction should accidentally hit on the correct label about as often as random guessing.

Answer 4 · 2024-05-13T07:50:44.000Z

I added the printout of accuracy in the code, but during the training process, the accuracy does not improve, and the loss does not converge, even after training for many epochs. Does this model really work? I think this model has serious problems. I hope to get an answer, this is very important to me