bytedance/lightseq

Issues

• LightSeq on GCP

  #530 opened a month ago by mmcgra21
  1

• [FeedBack][1.0.8][com.miniclip.eightballpool]-

  #529 opened 7 months ago by ap0jwd
  4

• Is llama inference available now?

  #513 opened 2 years ago by frankxyy
  1

• is Baichuan2 model inference available now?

  #528 opened 8 months ago by cg3dland
  0

• 参数格式不对args format wrong

  #527 opened 8 months ago by whiteSteelRain
  0

• identifier "__hisnan" is undefined

  #524 opened a year ago by jimmieliu
  4

• 请问lightseq可以支持segmentAnyting的推理优化吗

  #516 opened a year ago by sanbuphy
  1

• Exception: Installed CUDA version 12.3 does not match the version torch was compiled with 12.1, unable to compile cuda/cpp extensions without a matching cuda version.

  #526 opened a year ago by skill-diver
  0

• 要求C++ 17

  #525 opened a year ago by 2020zyc
  1

• Is it normal that A10 inference speed is lower than 2080ti?

  #523 opened a year ago by qinbo23
  1

• 为什么连给的example也有bug？

  #517 opened a year ago by Moran232
  4

• lightseq是否支持clip模型的int8量化？

  #522 opened a year ago by shhn1
  0

• Can int8 in pre-training large model ???

  #521 opened a year ago by zhoumengbo
  0

• 什么时候可以支持bloom的几个版本的模型，比如6b的

  #505 opened 2 years ago by liuzhipengchd
  3

• llama inference test

  #515 opened a year ago by HandH1998
  3

• how to resolve xlm-roberta convert fail

  #520 opened a year ago by 520jefferson
  0

• question about environment

  #519 opened a year ago by etoilestar
  0

• [Question] gptj, mpt support.

  #518 opened a year ago by DongqiShen
  0

• Do you have plans to support token_type_ids?

  #514 opened a year ago by chenchongthu
  0

• Is there a plan to support the T5-v1_1 model?

  #448 opened 2 years ago by dengcunqin
  2

• 请问下可以支持llama和bloom推理加速吗

  #502 opened 2 years ago by HuiResearch
  4

• 请问lightseq在推理流程中有gemm调参这一步吗？

  #512 opened 2 years ago by frankxyy
  0

• LLaMA example 结果验证

  #511 opened 2 years ago by chenzhengda
  0

• ValueError: cannot create std::vector larger than max_size()

  #477 opened 2 years ago by JianHu-1
  2

• RuntimeError: Ninja is required to load C++ extensions even after pip install ninja

  #503 opened 2 years ago by zt991211
  1

• lightseq' Transformer expects an extra layer_norm on both encoder and decoder level

  #509 opened 2 years ago by yuting-wang-1000
  0

• Do you consider supporting the chatglm model?

  #507 opened 2 years ago by Youggls
  0

• ls_torch_hf_quant_gpt2_export.py的使用问题

  #508 opened 2 years ago by wzh232894
  0

• How to get output scores for each output tokens of LightSeq BART model when inference

  #506 opened 2 years ago by quancq
  0

• [Question]: How to compile lightseq

  #445 opened 2 years ago by FrostML
  5

• About Vit encoder output consistency during inference?

  #488 opened 2 years ago by xiao2mo
  4

• Wrong encode_output_project_bias_kv_size !

  #497 opened 2 years ago by JunchengYao
  0

• Is there a plan to support the mT5-small model?

  #496 opened 2 years ago by qibao77
  0

• About inference speed compared to TRT16? [急急如律令]

  #490 opened 2 years ago by xiao2mo
  4

• [Running Fail] Assert failed at transformer_encoder_layer.py at line 342

  #487 opened 2 years ago by AlexwellChen
  0

• en2fr和en2de的模型结构存在差异？

  #485 opened 2 years ago by MeJerry215
  2

• 有什么方法将fairseq transformer的weight 转换为 lightseq transformer的weight？

  #484 opened 2 years ago by MeJerry215
  0

• 关于wmt14 en2de数据集的问题

  #483 opened 2 years ago by MeJerry215
  0

• GPU speedup vs CTranslate2

  #469 opened 2 years ago by nghuyong
  0

• beam search

  #455 opened 2 years ago by hg123-we
  0

• 转化之后的hdf5模型太大，lightseq 加载的时候支持多卡部署吗？

  #450 opened 2 years ago by Liuchunyangboy
  1

• Performance measurement with Pytorch/MLIR, Tensorflow/XLA

  #451 opened 2 years ago by Kangmo
  1

• How to enable accelerated amp trainer and gpu memory management?

  #452 opened 2 years ago by KimmiShi
  0

• 请问对于LSTM的量化，有用cuBLASLt库实现的方案吗？

  #449 opened 2 years ago by piaohe20221128
  2

• How to ensemble lightseq models? & the memory usage is too big when generating

  #440 opened 2 years ago by baoguo1995
  3

• How to use lightseq for bert's mlm training?

  #446 opened 2 years ago by ingale726
  2

• Possible memory leak in DecSelfAttentionLayer

  #443 opened 2 years ago by Kangmo
  2

• Question : About construction of total_cache_k, total_cache_v in Transformer

  #442 opened 2 years ago by Kangmo
  4

• Multi-calorie running model problem

  #444 opened 2 years ago by ingale726
  2

• Is it compatible with swin-t

  #439 opened 2 years ago by Chaoran-F
  1