Is the maximum sequence length of CodeGen2.5 really 4,096?

[njuweicheng]

Hi,

Thanks for your benchmark.

I am confused about the maximum sequence length of CodeGen2.5, which is 4,096 described in your Section 3.3. But I receicve the message when using CodeGen25-7B-mono: "This is a friendly reminder - the current text generation call will exceed the model's predefined maximum length (2048). Depending on the model, you may observe exceptions, performance degradation, or nothing at all."
Moreover, we tend to get weird generated content, such as lots of meaningless duplicates.
Is this a mistake in your paper, or how can it be set up?

Thanks!

[zijwang]

Thanks for the note. We are looking into this and we will share an update shortly.

[zijwang]

CodeGen 2.5 indeed has 2k max seq len. We will update the paper accordingly in the next iteration. Thanks again for the feedback.