intel-analytics/ipex-llm

log using ipex-llm instead of bigdl-llm in while running native models

Opened this issue · 0 comments

Logs use 'bigdl-llm' while converting and loading models into q4 binary format, should use ipex-llm

 bigdl-llm: loading model from ./bigdl_llm_llama_q4_0.bin                                                                                                                                                              
 loading bigdl-llm model: format     = ggjt v3 (latest)                                                                                                                                                                
 loading bigdl-llm model: n_vocab    = 32000                                                                                                                                                                            
 loading bigdl-llm model: n_ctx      = 512                                                                                                                                                                             
 loading bigdl-llm model: n_embd     = 5120                                                                                                                                                                            
 loading bigdl-llm model: n_mult     = 6912                                                                                                                                                                            
 loading bigdl-llm model: n_head     = 40                                                                                                                                                                              
 loading bigdl-llm model: n_layer    = 40                                                                                                                                                                              
 loading bigdl-llm model: n_rot      = 128                                                                                                                                                                             
 loading bigdl-llm model: ftype      = 2 (mostly Q4_0)                                                                                                                                                                
 loading bigdl-llm model: n_ff       = 13824                                                                                                                                                                          
 loading bigdl-llm model: n_parts    = 1                                                                                                                                                                              
 loading bigdl-llm model: model size = 13B                                                                                                                                                                            
 loading bigdl-llm model: ggml ctx size = 6595.81 MB                                                                                                                                                                  
 loading bigdl-llm model: mem required  = 8643.81 MB (+ 1608.00 MB per state) ```