abetlen/llama-cpp-python

Issues

• _LlamaModel.metadata() does not return `tokenizer.ggml.tokens`

  #1495 opened 3 days ago by ddh0
  7

• kv_override issue with string values

  #1487 opened 4 days ago by Erhan1706
  4

• Wrong switches: `unreachable-code-break` and `unreachable-code-return`, at least on Termux

  #1494 opened 3 days ago by Manamama
  0

• Can't install llama-cpp-python with HIPBLAS/ROCm on Windows

  #1489 opened 6 days ago by mbamg
  6

• Fix README: `Llama3VisionAlpha` not `Llama3VisionAlphaChatHandler`

  #1488 opened 4 days ago by mdnestor
  1

• Crashing with "IndexError: index 200 is out of bounds for axis 0 with size 200"

  #1438 opened 25 days ago by phishmaster
  2

• CUDA crash in llama_decode_internal, when using -ngl with Phi-3

  #1473 opened 12 days ago by themanyone
  0

• How should one accomplish plain topk sampling?

  #1484 opened 7 days ago by brandon-lockaby
  1

• "flash_attn = 0" is still still present on the newest llama-cpp-python versions

  #1479 opened 9 days ago by BadisG
  3

• Unable to make CMake find gcc and use CUDA Windows 11 CUDA tool kit installed 4gb video ram

  #1485 opened 7 days ago by Vishwamithra37
  0

• cohere multilanguage aya model seem can not use response_format feature of llama-cpp-python

  #1483 opened 8 days ago by svjack
  0

• Issues with Llava-v1.6-34b and Llava16ChatHandler

  #1482 opened 8 days ago by zenjieli
  0

• install target fails for llava

  #1481 opened 8 days ago by waheedi
  0

• Please add response_format to create_completion

  #1478 opened 9 days ago by dtkettler
  0

• OS package system builds? (Apt, yum, etc)

  #1480 opened 9 days ago by walterheck
  0

• Segmentation fault error while launching server on Apple Chip (Mac M3)

  #1474 opened 12 days ago by mhoangvslev
  1

• Unexpected output of embed

  #1469 opened 15 days ago by shizidushu
  3

• FileNotFoundError: Shared library with base name 'llama' not found

  #1475 opened 12 days ago by 0xWOLAND
  3

• Add support for llava-1-5-phi3-mini

  #1443 opened 23 days ago by CaptainOfHacks
  5

• Add RPC backend support

  #1455 opened 19 days ago by abetlen
  1

• Huge difference in performance between llama.cpp and llama-cpp-python

  #1447 opened 22 days ago by kseyhan
  2

• when the output is long , llama_decode returned error

  #1470 opened 15 days ago by Liufeiran123
  0

• model.tokenize result is different from tranformers tokenizer result

  #1468 opened 16 days ago by Liufeiran123
  0

• Slow response time and unable for multiple input prompt

  #1466 opened 16 days ago by OlivesHere
  1

• ValueError: could not broadcast input array from shape (371,) into shape (320,)

  #1421 opened a month ago by utility-aagrawal
  2

• MINGW / MSYS problem

  #1464 opened 17 days ago by Zibri
  0

• Segfault on load for embedding models lacking BOS/EOS token (such as `BAAI/bge-*`)

  #1463 opened 17 days ago by iamlemec
  1

• Support PyPI-installed `nvidia-cuda-runtime-cu12` and `nvidia-cublas-cu12`

  #1460 opened 18 days ago by Interpause
  0

• when depoly the llava-cpp-pyton server in k8s as a service , it can only answer questions about the first image

  #1458 opened 18 days ago by adogwangwang
  0

• Add support for auto setting n_gpu_layers from gguf and available vram size

  #1456 opened 19 days ago by abetlen
  0

• GPU memory released for llava multimodal

  #1451 opened 20 days ago by adogwangwang
  2

• "split_mode=2 (row)" not working -- got Aborted

  #1454 opened 19 days ago by oliverhh32
  0

• Severe Main Thread Bottleneck

  #1452 opened 20 days ago by Beinsezii
  3

• How to build the llamacpp's .so file separately and then pass it in the llama_cpp_python.

  #1435 opened 19 days ago by fastdaima
  6

• arm64 builds for CUDA

  #1446 opened 22 days ago by mcvella
  1

• CMAKE not even trying to include CUDA Toolkit during reinstallation

  #1428 opened 22 days ago by MarcusO17
  1

• Models are failing to be properly unloaded and freeing up VRAM

  #1442 opened 23 days ago by Baquara
  0

• Llama.generate: prefix-match hit is very slow.

  #1437 opened 25 days ago by ndy200
  2

• CUDA Error : CUDA driver version is insufficient for CUDA runtime version

  #1436 opened 25 days ago by VijayAsokkumar
  0

• Support for setting root_path when running llama_cpp.server

  #1420 opened a month ago by jharonfe
  1

• Add Nous Hermes 2 Pro function calling ChatHandler.

  #1429 opened a month ago by stygmate
  1

• Unable to install llama-cpp-python with CUBLAS or CUDA enabled under tensorflow-gpu docker image.

  #1431 opened a month ago by brent-halen
  1

• Which Python + CUDA version do you recommend for LLaVA on Windows 10 Pro 64-bit?

  #1418 opened a month ago by Vinventive
  1

• kv_override issue

  #1417 opened a month ago by sudharshan1234
  1

• jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'name' when running Functions.ipynb with Jupyter lab

  #1405 opened a month ago by southkorea2013
  2

• Docker image

  #1423 opened a month ago by Ravi-WingsBI
  0

• GPU not recognized on CUDA 12.3

  #1404 opened a month ago by y2ee201
  1

• cuda prebuilt wheel 12.3 error 404

  #1406 opened a month ago by xandykati98
  1

• Multimodal Llama3 Support

  #1403 opened a month ago by xx025
  1

• Implementation of LlamaDiskCache lacks capacity setting.

  #1402 opened a month ago by yamikumo-DSD
  0