c0sogi/llama-api

Issues

• FastAPI + llamapi issue

  #29 opened 7 months ago by Samraw003
  0

• Stopped working after enabling CUDA

  #28 opened a year ago by alexellis
  0

• High RAM and CPU usage

  #27 opened a year ago by delta-whiplash
  0

• warning: failed to mlock 245760-byte buffer (after previously locking 0 bytes): Cannot allocate memory llm_load_tensors: mem required = 46494.72 MB (+ 1280.00 MB per state)

  #13 opened a year ago by Dougie777
  4

• Proxy to openAI

  #9 opened a year ago by kreolsky
  2

• Usage of embedding through langchain

  #26 opened a year ago by jordandroid
  0

• how to run this api in cpu only mode

  #23 opened a year ago by delta-whiplash
  1

• exllamav2

  #20 opened a year ago by ehartford
  2

• Support min_p sampler

  #25 opened a year ago by atisharma
  0

• How can I use a specific prompt template?

  #24 opened a year ago by Dougie777
  0

• Zephyr7b gives gobbly gook output but Mistral7b works fine.

  #22 opened a year ago by Dougie777
  0

• exllama GPU split

  #21 opened a year ago by atisharma
  1

• Support for ExLlama V2

  #15 opened a year ago by Immortalin
  2

• Any way to define embeddings model in model_definitions.py?

  #19 opened a year ago by morgendigital
  1

• Set number of cores being used on cpu?

  #16 opened a year ago by Dougie777
  2

• Long generations dont return data but server says 200 OK. Swagger screen just says LOADING forever.

  #18 opened a year ago by Dougie777
  5

• BUG: I found the model path bug!

  #17 opened a year ago by Dougie777
  2

• Generation stops at 251 tokens - works fine on oobabooga

  #14 opened a year ago by Dougie777
  3

• model_definitions.py

  #12 opened a year ago by Dougie777
  3

• Is there a way to use this on google Colab and have the url be public?

  #11 opened a year ago by ashercn97
  1

• Dumb question: definitions.py model parameters

  #10 opened a year ago by Dougie777
  2

• Using with LangChain instead openai API

  #8 opened a year ago by kreolsky
  1