Tlntin/Qwen-TensorRT-LLM

Issues

• 求一个新的 TLLM 群交流群二维码

  #113 opened 8 months ago by Missmiaom
  4

• 请问新手该如何正确的学习项目代码

  #124 opened 4 months ago by Huziyou
  1

• run.py error

  #123 opened 6 months ago by caoquanjie
  2

• 请问目前的Qwen-VL实现方式，是否仅支持输入单张图片，且图片必须在输入的开头？

  #90 opened 9 months ago by xikaluo
  5

• 安装包

  #120 opened 7 months ago by zjjznw123
  1

• tp_size=4 build 报错

  #118 opened 7 months ago by mogoxx
  2

• what's the pp_size means?

  #122 opened 7 months ago by UIHCRITT
  0

• build报错

  #121 opened 7 months ago by zjjznw123
  1

• qwen1.5 编译错误：KeyError: 'kv_cache_block_pointers_list'

  #107 opened 8 months ago by whk6688
  2

• 在使用脚本qwen build时出现bug

  #117 opened 8 months ago by zixuxu000
  3

• Qwen1.5-32B-Chat-GPTQ-Int4 构建失败

  #119 opened 7 months ago by panjican
  2

• Qwen-32b prompt cache不支持

  #116 opened 8 months ago by wangye360
  4

• 求助

  #114 opened 8 months ago by Bilibili-Mikoto
  1

• qwen1.5进行解码的时候，部分汉字出现�问题，然后跟后面一个token连起来一起解码，结果就正常了，怎么处理

  #115 opened 8 months ago by MuyeMikeZhang
  1

• 当我用perf_analyzer测试性能时，出现“Thread [0] had error: Cannot send stop request without specifying a request_id”错误

  #112 opened 8 months ago by MuyeMikeZhang
  3

• swift微调的qwen-vl支持吗

  #75 opened 8 months ago by xs818818
  1

• how to build Qwen-72B-Chat-Int4 with tp=2

  #94 opened 8 months ago by liyunhan
  27

• 有人能共享Build好的qwen或qwen1.5 int4的trt_engine（4gpu）文件吗？

  #85 opened 8 months ago by zhangjiekui
  11

• 想请教一下qwen1.5_7b和llama系的区别

  #108 opened 8 months ago by DBCGary
  1

• Triton 和 Langchain部署问题

  #95 opened 8 months ago by plt12138
  6

• Qwen 2 build.py multi gpu with 2 different GPU's issue

  #98 opened 9 months ago by teis-e
  17

• How to use multi gpus in qwen2/quantize.py?

  #105 opened 8 months ago by qy1026
  2

• 请教一个codeqwen7b模型build过程的一个问题

  #111 opened 8 months ago by shiqingzhangCSU
  2

• Qwen1.5做smoothquant时维度不对

  #110 opened 8 months ago by zgplvyou
  3

• python run 错误

  #104 opened 8 months ago by maozixi1
  9

• Hi, a error about kv_cache_block_pointers_list

  #109 opened 9 months ago by lll143653
  2

• 大佬有没有对比和VLLM的推理效果？

  #72 opened a year ago by white-wolf-tech
  2

• API for multi-GPU inference

  #106 opened 9 months ago by UIHCRITT
  4

• triton 部署， 生成乱码

  #101 opened 9 months ago by maozixi1
  18

• web_demo无法显示模型响应

  #103 opened 9 months ago by elegant-bot
  2

• qwen_14b_chat build error

  #100 opened 9 months ago by AlgoJay1991
  3

• 编译tritonserver 镜像 失败

  #96 opened 9 months ago by maozixi1
  4

• 测试hf吞吐OOM以及triton并发、流式输出问题

  #81 opened 10 months ago by dongteng
  23

• Qwen-72B-Chat-Int4 killed

  #82 opened 10 months ago by Hukongtao
  2

• triton同步异步接口询问

  #91 opened 9 months ago by dongteng
  15

• 运行run.py报错，Segmentation fault (core dumped)

  #93 opened 9 months ago by ArlanCooper
  8

• ModuleNotFoundError: No module named 'transformers.models.qwen2'

  #92 opened 9 months ago by ArlanCooper
  2

• 运行build文件报错: TypeError: RowLinear.__init__() got an unexpected keyword argument 'instance_id'

  #86 opened 9 months ago by ArlanCooper
  2

• 请问如何支持正常的batch infer ？

  #88 opened 10 months ago by zhangyu68
  2

• 请问为什么smoothquant量化后显存占用不降低呢

  #87 opened 10 months ago by tp-nan
  6

• 使用auto-gptq编译qwen_1_8B-Chat-int4官方报错'KeyError: 'transformer.h.0.attn.c_attn.qweight'

  #83 opened 10 months ago by fmozer
  5

• 想问一下，为什么72B模型是实验性的呢？架构应该是一样的呀，原因是什么呢？谢谢

  #84 opened 10 months ago by zhangjiekui
  2

• ERROR: Failed to create instance: unexpected error when creating modelInstanceState

  #71 opened 10 months ago by lyc728
  3

• Qwen1.5 GPTQ用不了

  #76 opened 10 months ago by Pevernow
  2

• Qwen1.5 GPTQ-Int4 编译失败

  #77 opened 10 months ago by ljhssga
  15

• Qwen1.5 GPTQ编译错误

  #78 opened 10 months ago by compass-star
  1

• Qwen2 编译错误

  #80 opened 10 months ago by mogoxx
  5

• TensorRT_LLM 0.7.0 编译 Qwen-7B-Chat 模型，编译后启动API似乎无法支持并发访问API？

  #79 opened 10 months ago by CedricHwong
  2

• 函数调用会报错

  #74 opened a year ago by xzmagic
  0

• 大佬请问个问题：AttributeError: 'QWenForCausalLM' object has no attribute 'embedding'

  #73 opened a year ago by dongteng
  0