NVlabs/VILA

Is there any way to increase the context window?

ZackBradshaw opened this issue · 4 comments

Token indices sequence length is longer than the specified maximum sequence length for this model (356885 > 4096). Running this sequence through the model will result in indexing errors

Thank you for your reply.
It seems that the only option is to delete the relevant data from the JSON file. Thank you for sharing.

We will release a long-context version soon. Please stay tuned.

@Lyken17
Thank you, I'm looking forward to it.