Wrong encoder width in bert_large

Question

Wrong encoder width in bert_large

HyeongminLEE opened this issue a year ago · 1 comments

Hi, I found that the encoder width option in https://github.com/OpenGVLab/unmasked_teacher/blob/main/multi_modality/configs/config_bert_large.json has been set to be 768. However, as far as I know, the encoder width of bert_large should be 1024.
Please check this again.

Thanks!

Answer 1 · 2023-11-25T10:44:43.000Z

Actually, this is a placeholder, and the value will be changed in the code https://github.com/OpenGVLab/unmasked_teacher/blob/f6e0b283be2f544c83c7d1f767d5b406b0ec464c/multi_modality/models/backbones/bert/builder.py#L18C5-L18C30