microsoft/aici

Question about FixedToken (insertion)?

JohnPeng47 opened this issue · 0 comments

Is the fixedToken inserted into the transformer decoding stream after aici detects that a specified character has been generated? To clarify, because I'm very unfamiliar with transformer terminology, I understand that the auto encoder/decoder generates n tokens, where the m (<n)th token is conditioned on the previous m-1 tokens. So does fixedToken actually insert a token into the m+1 generation stream, after it has detected tokens generated in the previous [0, m] range?