anarchy-ai/llm-speed-benchmark
Benchmarking tool for assessing LLM models' performance across different hardwares
PythonMIT
Issues
- 0
question
#7 opened by geraldstanje1 - 1
Add option to change dtype
#4 opened by MehmetMHY - 1
4bit/Llama-2-7b-chat-hf failed to run
#3 opened by MehmetMHY