lapp0/lm-inference-engines

Comparison for other frameworks as well

ogencoglu opened this issue · 2 comments

lapp0 commented

Not familiar with any of these other than FlexFlow unfortunately. Happy to include PRs for any of these if they are uniquely valuable inference engines.

ogencoglu commented

Each has its own focus. Some benchmarks can be found on their docs e.g., https://github.com/InternLM/lmdeploy?tab=readme-ov-file#performance