Token size mismatch for pre-trained model

[mdkma]

I met with the new pre-trained model is that there is still a RuntimeError saying that:

RuntimeError: Dimensions of lookup parameter /_0 lookup up from file ({100,400574}) do not match parameters to be populated ({100,400575}

It seems like the tokens used for the pre-trained models (totally 400575) has one more token than current code implementation (which is 400574). This happens for all three models targetid, frameid and argid in test or predict mode.

I got this error when using the commit e5b8ad4b8d9ea76473365558cb234e8bad4af874.

[Brandonnogithub]

similar problem.

[swabhs]

Please make sure you are using the latest commit - this problem has been fixed now.