skeskinen/bert.cpp

Running `bert_load_from_file` from code results with ` 'embeddings.word_embeddings.weight' has wrong shape in model file`

NitayRabi opened this issue · 1 comments

I am using Rust to compile bert.cpp and use bert_load_from_file as an extern, I am able to pass the path to the module successfully, but I get the following output:

bert_load_from_file: loading model from './models/ggml-model-f32.bin' - please wait ...
bert_load_from_file: n_vocab = 30522
bert_load_from_file: n_max_tokens   = 512
bert_load_from_file: n_embd  = 384
bert_load_from_file: n_intermediate  = 1536
bert_load_from_file: n_head  = 12
bert_load_from_file: n_layer = 6
bert_load_from_file: f16     = 0
bert_load_from_file: ggml ctx size =  86.10 MB
bert_load_from_file: tensor 'embeddings.word_embeddings.weight' has wrong shape in model file: got [2, 384], expected [384, 30522]

Tried this with all quantizations, using a model downloaded using python3 models/download-ggml.py download all-MiniLM-L6-v2 {Q}

Not an issue with bert.cpp, running the same models as stated in the readme works.