Suggestion to add DeepSeek v2.5 support

Question

arisau opened this issue 3 months ago · 4 comments

Recently DeepSeek 2.5 MoE model was released. Please consider adding a support for it. (Q5_K_M or Q4_K_M)

Answer 1 · 2024-09-24T10:31:11.000Z

It appears to be supported automatically, as some users have already tested it successfully.

Answer 2 · 2024-09-24T21:04:31.000Z

Perfect. Thank you.
just to confirm, only Q4_K_M works?

Answer 3 · 2024-09-25T04:45:04.000Z

We have supported all Qx_km format~

Answer 4 · 2024-09-25T08:49:30.000Z

Understood. Thank you once again.