RuntimeError: mat1 and mat2 shapes cannot be multiplied (4096x5120 and 1x2560)
monuminu opened this issue · 1 comments
monuminu commented
Ran all the cells of Notebook to funetune LLama2 got this error.
2023-07-20T16:08:06.067+05:30 | return forward_call(*args, **kwargs) File "/opt/conda/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward | |
---|---|---|
2023-07-20T16:08:06.068+05:30 | output = old_forward(*args, **kwargs) File "/opt/conda/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 408, in forward | |
2023-07-20T16:08:06.068+05:30 | hidden_states, self_attn_weights, present_key_value = self.self_attn( File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl | |
2023-07-20T16:08:06.068+05:30 | return forward_call(*args, **kwargs) File "/opt/conda/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward | |
2023-07-20T16:08:06.068+05:30 | output = old_forward(*args, **kwargs) File "/opt/conda/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 295, in forward | |
2023-07-20T16:08:06.068+05:30 | query_states = [F.linear(hidden_states, query_slices[i]) for i in range(self.pretraining_tp)] File "/opt/conda/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 295, in | |
2023-07-20T16:08:06.068+05:30 | query_states = [F.linear(hidden_states, query_slices[i]) for i in range(self.pretraining_tp)] | |
2023-07-20T16:08:06.068+05:30 | RuntimeError: mat1 and mat2 shapes cannot be multiplied (4096x5120 and 1x2560) |
philschmid commented
Did you make any changes? Did you make sure the requirements.txt
is provided?