hamelsmu/llama-inference

Issues

• Can you do a benchmark on what happens when you load the basic HF model with bfloat16 ?

  #4 opened a year ago by mihaipora
  0

• Incorrect results for exllama

  #2 opened a year ago by arbi-dev
  3

• How is it compared with the Deepspeed inference?

  #1 opened a year ago by allanj
  0