log using ipex-llm instead of bigdl-llm in while running native models
Opened this issue · 0 comments
songhappy commented
Logs use 'bigdl-llm' while converting and loading models into q4 binary format, should use ipex-llm
bigdl-llm: loading model from ./bigdl_llm_llama_q4_0.bin
loading bigdl-llm model: format = ggjt v3 (latest)
loading bigdl-llm model: n_vocab = 32000
loading bigdl-llm model: n_ctx = 512
loading bigdl-llm model: n_embd = 5120
loading bigdl-llm model: n_mult = 6912
loading bigdl-llm model: n_head = 40
loading bigdl-llm model: n_layer = 40
loading bigdl-llm model: n_rot = 128
loading bigdl-llm model: ftype = 2 (mostly Q4_0)
loading bigdl-llm model: n_ff = 13824
loading bigdl-llm model: n_parts = 1
loading bigdl-llm model: model size = 13B
loading bigdl-llm model: ggml ctx size = 6595.81 MB
loading bigdl-llm model: mem required = 8643.81 MB (+ 1608.00 MB per state) ```