ggerganov/llama.cpp

Issues

• Possible bug in the 'deepseek-coder' chat template's system message

  #7385 opened 4 hours ago by jukofyork
  0

• Different result between use llama_tokenize and python original transformers tokenizer

  #7384 opened 4 hours ago by Liufeiran123
  1

• ggml_validate_row_data finding nan value for IQ4_NL

  #7311 opened a day ago by bartowski1182
  6

• Custom `seed` values ignored by `llama.cpp HTTP server`

  #7381 opened 4 hours ago by mirekphd
  3

• /embeddings endpoint sometimes does not return embedding

  #7277 opened 5 days ago by marcingomulkiewicz
  3

• [Android/Termux] Significantly higher RAM usage with Vulkan compared to CPU only

  #7351 opened a day ago by egeoz
  2

• llama : save downloaded models to local cache

  #7252 opened 6 days ago by ggerganov
  4

• Why does the server-cuda container consume CPU time?

  #7377 opened 4 hours ago by wencan
  1

• convert-hf-to-gguf.py fails PR #7234

  #7380 opened 4 hours ago by LostRuins
  0

• Can I handle multiple images in the same context?

  #7364 opened a day ago by Eriter555
  0

• llama_model_load: error loading model: unable to allocate backend buffer

  #7366 opened a day ago by phaelon74
  1

• Funny response with LLaMa 3 8B

  #7367 opened a day ago by Sewlell
  0

• bf16 problem

  #7365 opened a day ago by Zibri
  0

• [SYCL] include shared libs in sycl release

  #7361 opened a day ago by gfody
  0

• Description of "-t N" option for server is inaccurate

  #7355 opened a day ago by tigran123
  1

• Need help on building shared libraries on Windows machine for Android x86_64 (emulator)

  #7357 opened a day ago by cmpktheo
  0

• EOT token incorrectly set for Mistral-v0.2 trained with added ChatML tokens

  #7271 opened 5 days ago by xzuyn
  4

• Improve and expand Wikipedia article about llama.cpp

  #7294 opened 4 days ago by fffelix-jan
  2

• Possible performance boost with 2-pass online softmax

  #7306 opened 4 days ago by zixuanweeei
  1

• RPC issues and comments

  #7293 opened a day ago by steampunque
  8

• convert.py still fails on llama3 8B-Instruct downloaded directly from Meta (Huggingface works)

  #7339 opened 2 days ago by aleloi
  2

• AMD ROCm: 8x22B Model Causes 100% GPU Utilization Stall

  #7344 opened a day ago by Trat8547
  0

• Flash attention implementations do not handle case where value vectors have different dimension from query vectors

  #7343 opened 2 days ago by fairydreaming
  1

• Pretokenizer not supported by conversion script

  #7338 opened a day ago by eleius
  2

• Metal (iOS): Compute function exceeds available temporary registers

  #7261 opened 5 days ago by guinmoon
  2

• Issues: Unable for multiuser prompt

  #7336 opened 2 days ago by OlivesHere
  1

• Segmentation Fault on GPU

  #7337 opened 2 days ago by djain-fujitsu
  1

• enable rpc for server

  #7292 opened 2 days ago by steampunque
  0

• Support Falcon2-11B

  #7318 opened 2 days ago by reneleonhardt
  2

• llava surgery script for new llava-arch model from Intel

  #7333 opened 2 days ago by KohakuBlueleaf
  0

• Llama3-8b & Perplexity.exe Issue

  #7291 opened 2 days ago by InferenceIllusionist
  6

• GGML_ASSERT(n_embd_gqa == n_embd_k_gqa) fails in models where key vector dimension is different from value vector dimension

  #7331 opened 3 days ago by fairydreaming
  0

• Add support for multilingual Viking models, please.

  #7309 opened 3 days ago by JohnClaw
  1

• support long context llama 3 models

  #7312 opened 3 days ago by bachittle
  1

• Support for IBM Granite models

  #7307 opened 3 days ago by ichDaheim
  1

• How to quantize fine-tune LLM into GGUF format

  #7299 opened 4 days ago by dibyendubiswas1998
  2

• relocation R_X86_64_32 against hidden symbol `__TMC_END__' can not be used when making a shared object

  #7301 opened 4 days ago by asarubbo
  0

• Llama-3 Instruct tokenizer_config.json changes in relation to the currently fetched llama-bpe configs.

  #7289 opened 4 days ago by Spacellary
  0

• In my os, the @ symbol and spaces don’t play nicely in llama.cpp directory.

  #7247 opened 4 days ago by atljoseph
  4

• Error when trying to convert a HF model which is a LORA PEFT fine tuned version of phi-128k

  #7287 opened 4 days ago by swarnava112
  2

• Windows MSYS2 compilation error. [SOLVED]

  #7275 opened 5 days ago by Zibri
  3

• Does it make sense to optimize strlen in this function with for loops?

  #7268 opened 5 days ago by GermanAizek
  4

• Possible stopping issues and bad asterik tokenization? (GGUF related)

  #7262 opened 5 days ago by Dampfinchen
  3

• Infinite update_slots issue on latest build (1265c67)

  #7283 opened 5 days ago by Leowolf93
  0

• MPI issue on raspberry pi cluster

  #7260 opened 6 days ago by zhouwul
  3

• ThunderKittens：a simple yet faster flashattention alternative

  #7276 opened 5 days ago by sorasoras
  1

• How to build the llamacpp's .so file separately and then pass it in the llama_cpp_python / wrapper libraries directly.

  #7250 opened 6 days ago by fastdaima
  1

• Text Generation task

  #7256 opened 5 days ago by rexionmars
  2

• Performance regression with CUDA after commit 9c67c277

  #7254 opened 6 days ago by rgerganov
  8

• while finetuning llama.cpp doesn't create .bin file...

  #7253 opened 6 days ago by elijiahmiro
  0