Face negative dimension issue when running on CIFAR10
lukaswangbk opened this issue · 1 comments
Hi, I am Lukas Wang, a master's student from Columbia. I am planning to review cutting-edged VIT-based models on medium-size datasets and found your work really interesting! I was trying to run the code using CIFAT10 dataset for testing but the following error came out.
RuntimeError: Trying to create tensor with negative dimension -96: [-96, 1, 9, 9]
I have noticed that the environment variable groups is set to groups=[-1, -1, 3, 6] as default in DAT model while the operation for DAttentionBaseline in dat_block.py will compute a negative value for first two stages. Could you please check out this issue? Really appreciate your help :)!
Hello @lukaswangbk,
In the conference version of DAT, the first two stages contain no DMHA blocks, so I set a -1 to avoid misuse. By the way, a new DAT++ with an extended paper has come out, and we welcome you to have a try.
Also see #30.