What is the difference between 7B_float16 and 7B_FT_float16

[mszhanyi]

When I use 7B_FT_float16
python MinimumExample/Example_ONNX_LlamaV2.py --onnx_file 7B_FT_float16/ONNX/LlamaV2_7B_FT_float16.onnx --embedding_file 7B_FT_float16/embeddings.pth --tokenizer_path tokenizer.model --prompt "What is the lightest element?"
The response is as expected:

The lightest element is hydrogen. Hydrogen is the lightest element on the periodic table, with an atomic mass of 1.00794 u (unified atomic mass units).

When I use 7B_float16
python MinimumExample/Example_ONNX_LlamaV2.py --onnx_file 7B_float16/ONNX/LlamaV2_7B_float16.onnx --embedding_file 7B_float16/embeddings.pth --tokenizer_path tokenizer.model --prompt "What is the lightest element?"
The response is strange:

What is the lightest element on the periodic table?
What is the lightest element in the universe?
What is the lightest element in the periodic table?
What is the lightest element in the universe?
What is the lightest element in the periodic table?
What is the lightest element in the universe?
What is the lightest element in the periodic table?
What is the lightest element in the universe?
What is the lightest element in the periodic table?
What is the lightest element in the universe?
What is the lightest element in the periodic table?
What is the lightest element in the universe?
What is the lightest element in the periodic table?
What is the lightest element in the universe?
What is the lightest element in the periodic table?
What is the lightest element in the universe?
What is the lightest element in the periodic table?
What is the lightest element in the universe?
What is the lightest element in the periodic table?
What is the lightest element in the universe?
What is the lightest element in the periodic table?
What is the lightest element in the universe?
What is

[kckeiks]

@mszhanyi did you ever figure this out? I think it's fine-tuned.

[purejomo]

same problem here