OUTPUT_POSTFIX in preprocess function can cause infinitively generation.

Question

OUTPUT_POSTFIX in preprocess function can cause infinitively generation.

Opened this issue a year ago · 0 comments

if role.upper() == "ASSISTANT": input_messages.append(msg+ OUTPUT_POSTFIX)

should be

if role.upper() == "ASSISTANT": input_messages.append(msg + " " + OUTPUT_POSTFIX)
For example, 3</s> is completely different from 3 </s>. The former string doesn't return the end of sequence token. It's mean the EOS token never appears in training process, so leading to infinitively generation in inference.