moomou/listening-with-llm

Inference

Closed this issue · 2 comments

dhifafaz commented

Hai i just finish the training process, i would like to now how exactly you load the .pth weight to do the inference, where should i load it, thanks

This is the model.pth keys that i get
odict_keys(['proj.weight', 'ln1.weight', 'ln1.bias'])

dhifafaz commented

i try to run the util script and keep throw error like this

...
--> [136](~/anaconda3/envs/llm-audio/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:136)     expanded_attn_mask = causal_4d_mask.masked_fill(expanded_attn_mask.bool(), torch.finfo(dtype).min)
    [138](~/anaconda3/envs/llm-audio/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:138) # expanded_attn_mask + causal_4d_mask can cause some overflow
    [139](~/anaconda3/envs/llm-audio/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:139) expanded_4d_mask = expanded_attn_mask

RuntimeError: The size of tensor a (322) must match the size of tensor b (321) at non-singleton dimension 3
dhifafaz commented

i think i found the problem, i need to use the transformers version on you requirements