huofushuo/SID

Support InternVL2‑26B

Closed this issue · 3 comments

您好,
看论文中的效果很好,在InternVL2‑26B中进行尝试,不知道需要怎么修改?
感谢!

Thanks for your attention. As for InternVL, you mainly need to modify the modeling strategy like modeling_llama.py--def forward() and corresponding decoding strategy like vcd_sample.py --def sample(). The corresponding intermediate variables also need to be passed.

Thank you for the reply.
I am new to MLLM, could you give more specific details on which file to modify and how to modify it.
And InternVL2‑26B'LLM is llama architecture,is it possible to use your code directly?
Thank you.

The corresponding intermediate variables of contrastive logits in InternVL need to be passed. InternVL forward function and decoding strategy need to be revised correspondingly.