Support static shape
Opened this issue · 0 comments
hans00 commented
Feature request
Support static shape (auto padding to max length on input) and static shape KV cache for LLM.
Motivation
Static shape will be problem when enable NPU, WebNN or CoreML.
Your contribution
I can submit a PR.
But I'm not pro of models, not sure correct implement of static KV cache.