Running `bert_load_from_file` from code results with ` 'embeddings.word_embeddings.weight' has wrong shape in model file`
NitayRabi opened this issue · 1 comments
NitayRabi commented
I am using Rust to compile bert.cpp
and use bert_load_from_file
as an extern
, I am able to pass the path to the module successfully, but I get the following output:
bert_load_from_file: loading model from './models/ggml-model-f32.bin' - please wait ...
bert_load_from_file: n_vocab = 30522
bert_load_from_file: n_max_tokens = 512
bert_load_from_file: n_embd = 384
bert_load_from_file: n_intermediate = 1536
bert_load_from_file: n_head = 12
bert_load_from_file: n_layer = 6
bert_load_from_file: f16 = 0
bert_load_from_file: ggml ctx size = 86.10 MB
bert_load_from_file: tensor 'embeddings.word_embeddings.weight' has wrong shape in model file: got [2, 384], expected [384, 30522]
Tried this with all quantizations, using a model downloaded using python3 models/download-ggml.py download all-MiniLM-L6-v2 {Q}
NitayRabi commented
Not an issue with bert.cpp
, running the same models as stated in the readme works.