clessig/atmorep

Misspeled embedding file name

Opened this issue · 2 comments

kacpnowak commented

In the atmorep_model.py line 284 embedding files are loaded under the name "_embed_token_info", however in line 455 are saved as "_embeds_token_info" causing a crash when multiformers are used

kacpnowak commented

I've also encoutered error with keys not matching:
RuntimeError: Error(s) in loading state_dict for Linear:
3: Missing key(s) in state_dict: "weight", "bias".
3: Unexpected key(s) in state_dict: "0.weight", "0.bias".

To solve it one can add "strict=False" in line 286

clessig commented

Ok, we don't see the issue with multiformers not loading. Which branch are you on?

@iluise : have you seen a problem with multiformers?