ztxz16/fastllm

Issues

• 希望作者考虑按照我的设计思路增加对deepseek-R1-671B及类deepseek-R1-671B的Moe模型的支持

  #503 opened 5 days ago by a793181018
  0

• UnicodeDecodeError: 'utf-8' codec can't decode byte 0xce in position 4411: invalid continuation byte

  #489 opened 5 months ago by huqiangDu
  1

• 转换 Llama-2-7b 模型出错

  #501 opened 2 months ago by JocelynPanPan
  0

• Main.exe compiled with w64devkit v.1.21.0 silently terminates after launching.

  #499 opened 2 months ago by JohnClaw
  0

• Thanks for great llm inference engine. Could you make a dll version of it, please?

  #498 opened 2 months ago by JohnClaw
  0

• 如何配置参数使服务处理请求并发数最大化

  #494 opened 4 months ago by xiaoshizijiayou
  0

• error: no suitable user-defined conversion from "__half" to "__nv_bfloat16" exists

  #493 opened 4 months ago by xiaoshizijiayou
  0

• ModuleNotFoundError: No module named 'ftllm'

  #492 opened 5 months ago by mingyue0094
  4

• 如何通过参数方式直接加载adapter?

  #491 opened 5 months ago by xiaoshizijiayou
  28

• 加速llama3-sqlcoder-8b (Finetuned from model: [Meta-Llama-3-8B-Instruct])模型时，输出内容错误，全部是"!!!!!"

  #487 opened 5 months ago by Juvember
  1

• pytorch模型转flm模型Killedt

  #488 opened 5 months ago by scutzhe
  1

• 结果返回一直是<unk>

  #452 opened 9 months ago by VincentLore
  3

• 模型权重转化之后和原来的模型回答的内容不一致

  #486 opened 5 months ago by Whylickspittle
  1

• 编译完之后运行模型时报错

  #484 opened 6 months ago by supercj92
  1

• chatglm 失去 function calling 能力

  #485 opened 6 months ago by NingRiCheng
  0

• 请问一下国产显卡Ascend 910 and Hygon DCU如何安装fastllm？

  #482 opened 7 months ago by cgq0816
  1

• GLM4-V-9B什么时候会出部署代码呢？

  #481 opened 7 months ago by GalSang17
  0

• 如何多卡部署

  #480 opened 7 months ago by longcheng183
  1

• OSError: libcublas.so.ll: cannot open shared odject file: No such file or directory

  #471 opened 8 months ago by lichengyang666
  1

• Meta-Llama-3-70B-Instruct

  #470 opened 8 months ago by longcheng183
  5

• make -j过程中报错

  #459 opened 8 months ago by AIlaowong
  3

• 请问什么时候支持GLM-4 ？

  #462 opened 8 months ago by Stupid-Ai
  4

• GLM-4-6B-Chat转换成flm格式后不能加载

  #465 opened 8 months ago by HofNature
  5

• 请问现在支持deepseekv2量化吗

  #457 opened 8 months ago by fw2325
  1

• H800 docker 编译, half类型转换 编译报错

  #463 opened 8 months ago by ShadowTeamCN
  1

• qwen1.5 int4模型回复出现解码问题：UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 72-73: invalid continuation byte

  #461 opened 8 months ago by zhang415
  0

• [CMakeFiles/Makefile2:100: CMakeFiles/pyfastllm.dir/all]

  #454 opened 9 months ago by ttaop
  0

• chatglm3 相同提示词生成结果一致

  #450 opened 10 months ago by ttaop
  0

• Do you have a plan to implement the CudaCatOp?

  #448 opened 10 months ago by dp-aixball
  0

• 中文输入无法识别；webui打开的地址无法访问。

  #447 opened 10 months ago by Mihubaba
  1

• 千问qwen1.5-14B-chat解码错误

  #446 opened 10 months ago by yiguanxian
  2

• cmake -j报错

  #445 opened 10 months ago by gggdroa
  2

• 无法安装fastllm_pytools

  #443 opened 10 months ago by bailingchun
  1

• 流式输出中断问题

  #440 opened a year ago by lwinhong
  0

• 模型转换的时候是不是不能用量化过的模型

  #437 opened a year ago by shum-elli
  1

• 是否支持qwen1.5的滑动窗口的方式

  #436 opened a year ago by aofengdaxia
  0

• 大佬您好，这个性能和chatglm.cpp比起来，会更好吗

  #435 opened a year ago by ericjing83
  0

• Error: cublas error during MatMul in Attention operator.

  #433 opened a year ago by pingyuan2016
  3

• fastllm是否支持使用bitsandbytes量化的chatglm3-6b-base int4模型

  #434 opened a year ago by levinxo
  0

• /api/chat_stream The result returned by postman is empty

  #432 opened a year ago by Dong09
  0

• ResponseBatch 返回结果不正确

  #429 opened a year ago by Liufeiran123
  5

• 请求支持Grouped Query Attention

  #416 opened a year ago by TylunasLi
  0

• batch padding mask 处理的相关代码

  #427 opened a year ago by Liufeiran123
  0

• qwen输出结果错误

  #418 opened a year ago by Liufeiran123
  1

• 如何贡献代码 ？

  #422 opened a year ago by hadoop2xu
  0

• 后续能否支持ChatGLM3的多轮

  #419 opened a year ago by chenyangjun45
  2

• 目前PEFT仅支持chatglm，什么时候可以支持其他模型，比如baichuan2呢？或者需要改哪些地方，很乐意contribute。

  #421 opened a year ago by jiahuanluo
  1

• 转化模型格式(.bin->.flm)时

  #413 opened a year ago by ColorfulDick
  2

• 大佬 想问下 利用率只跑到60% 是什么情况?

  #414 opened a year ago by Chenhuaqi6
  2

• 当输出数据特别长的时候报错。

  #409 opened a year ago by aofengdaxia
  2