ProjectD-AI/llama_inference

Issues

• int8量化输出不完整

  #19 opened a year ago by zhenglinpan
  3

• 模型分片了怎么指定load_model_path

  #18 opened a year ago by caowenhero
  1

• 你好 请问支持llama 65B了嘛

  #17 opened a year ago by zcuuu
  1

• 多卡推理

  #16 opened a year ago by yingzhao27
  4

• lora模型推理啥时候能出来啊

  #15 opened a year ago by isaacxie41
  1

• fp32 精度-inference

  #10 opened a year ago by biubiobiu
  3

• 为什么LLaMa模型只有encoder没有decoder

  #14 opened a year ago by yyqi17
  1

• 生成乱码

  #13 opened a year ago by McCarrtney
  4

• RuntimeError: probability tensor contains either `inf`, `nan` or element < 0

  #11 opened a year ago by yingzhao27
  2

• 运行多轮对话时代码报错

  #12 opened a year ago by LJL00000
  1

• llama_server.py支持多卡推理吗

  #7 opened a year ago by yuxuan2015
  3

• 求老哥搞个lora的

  #6 opened a year ago by ze00ro
  1

• assert batch <= args.batch_size AssertionError

  #2 opened a year ago by baketbek
  1