intel-analytics/ipex-llm
Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, GraphRAG, DeepSpeed, Axolotl, etc
PythonApache-2.0
Issues
- 13
- 2
[torch 2.3 + bigdl-core-xe-23] AttributeError: module 'xe_linear' has no attribute 'forward_qkv'
#12506 opened by Nuullll - 5
- 11
update to ollama 0.4.0
#12370 opened by Matthww - 0
LLaVA-Video-7B-Qwen2 int4 quantization enabling on ARC
#12482 opened by zhangcong2019 - 3
Encounter error when running Qwen2-VL in ipex-llm processing input video with large frame number
#12469 opened by zhangcong2019 - 1
Error while deserializing header: HeaderTooLarge
#12492 opened by lvjingax - 9
- 12
- 6
- 2
Error: llama runner process has terminated: exit status 127
#12471 opened by NikosDi - 3
Kernel NULL pointer dereference in i915 driver
#12435 opened by luhuaei - 5
init-ollama.bat Not Working
#12465 opened by imabdul-dev - 8
Disable XMX
#12426 opened by NikosDi - 2
Request to upgrade "Langchain-Chatchat" based on the latest version in github.
#12448 opened by liang1wang - 12
Container cannot see Arc GPU
#12372 opened by robertvazan - 1
Inference is exceptionally slow on the L20 GPU
#12440 opened by joey9503 - 10
Error: llama runner process has terminated: error loading model: No device of requested type available
#12420 opened by fanlessfan - 3
Unable to inference with Qwen2.5 GPTQ model
#12432 opened by notsyncing - 5
llama.cpp crashes running k-quants with Intel Arc 140V Xe2 iGPU
#12318 opened by lhl - 1
- 1
nf4 still unsupported?
#12427 opened by epage480 - 4
- 1
'AutoModel' object has no attribute 'config' when using Speech_Paraformer-Large on NPU
#12412 opened by fanyhchn - 1
Update Ollama with IPEX-LLM to a newer version
#12411 opened by NikosDi - 4
Path of models using Ollama with IPEX-LLM (Windows)
#12403 opened by NikosDi - 6
using both iGPU and CPU together
#12373 opened by fanlessfan - 3
Llama-3.2 11B Vision not working with latest IPEX-LLM (vLLM version 0.6.2)
#12391 opened by HumerousGorgon - 4
assert error use ipex pytorch
#12385 opened by piDack - 6
- 11
Could not use SFT Trainer in qlora_finetuning.py
#12356 opened by shungyantham - 2
Docker - llama.cpp scripts / init-llama-cpp
#12379 opened by easyfab - 8
cant run ollama in docker container with iGPU in linux
#12363 opened by user7z - 1
performance problem about internvl image embedding using ggml.dll
#12376 opened by cjsdurj - 3
ipex-llm-cpp-xpu container
#12364 opened by user7z - 1
ValueError: If `eos_token_id` is defined, make sure that `pad_token_id` is defined
#12371 opened by fanlessfan - 3
Ollama run embedding module mxbai-embed-large failed.
#12348 opened by feiyu11859661 - 1
How to check GPU memory consumption by ipex on Linux?
#12315 opened by acane77 - 1
A770运行 ipex_llm harness 跑chatglm3-6b 出现Error Message: property 'pad_token' of 'ChatGLMTokenizer' object has no setter
#12335 opened by tao-ov - 0
Doubts about ParallelTable and ParallelCriterion
#12278 opened by clare-cn - 2
ipex-llm xpu version doesn't work on Lunar Lake
#12268 opened by HoppeDeng - 1
IPEX-LLM load qwen2.5 7B model failed
#12273 opened by HoppeDeng - 1
ipex-llm-ollama-installer-20240918.exe安装后用另一个exe调用文件夹中的start.bat会提示缺少dll等无法运行
#12334 opened by dayskk - 1
- 1
[ipex-llm] A significant deviation in accuracy between ipex llm 2.2.0b1 and 2.1.0b20240515 when running the codegeex model
#12294 opened by johnysh - 4
A770 run harness.RuntimeError: unsupported dtype, only fp32 and fp16 are supported
#12304 opened by tao-ov - 6
run harness on A770 error
#12290 opened by tao-ov - 3
Questions about performance gap between benchmark scripts and llama-bench from ipex-llm[cpp]
#12280 opened by acane77 - 0
[NPU] Slow Token Generation with Latest NPU Driver 32.0.100.3053 on LNL 226V series
#12266 opened by climh - 1
Llamacpp generation incoherent (always <eos>). Driver version on ubuntu 22.04.5?
#12258 opened by ultoris