premAI-io/benchmarks

Issues

• Incorrect results reported for TensorRT-LLM

  #187 opened 8 months ago by lopuhin
  5

• Max Engine by Mojo

  #188 opened 7 months ago by Anindyadeep
  0

• Use exllamav2's smart 4-bit KV cache for memory benchmark

  #185 opened 8 months ago by Interpause
  1

• aphrodite-engine

  #186 opened 8 months ago by Anindyadeep
  0

• Benchmark along with a modular inference endpoint provider.

  #102 opened 8 months ago by Anindyadeep
  1

• For quality checks reference should be taken from actual PyTorch version of model

  #162 opened 8 months ago by Anindyadeep
  1

• We should also include the memory consumption for the devices.

  #106 opened 8 months ago by Anindyadeep
  1

• Improvements on logging and storing results

  #157 opened 8 months ago by Anindyadeep
  1

• Code modularity, refactoring and re-usability.

  #91 opened 8 months ago by Anindyadeep
  1

• Fix required. Change `model_path` to `model_dir_path`

  #113 opened 8 months ago by Anindyadeep
  1

• A nice blog containing all the analysis of benchmarks

  #146 opened 8 months ago by Anindyadeep
  1

• Add model loading time for each benchmarks

  #181 opened 8 months ago by Anindyadeep
  0

• Add `torch.inference_session` on runner function

  #180 opened 8 months ago by Anindyadeep
  0

• Complete ML Engines Table

  #149 opened 8 months ago by nsosio
  1

• Change Main readme

  #176 opened 8 months ago by Anindyadeep
  1

• Setup new md files from templates

  #179 opened 8 months ago by Anindyadeep
  1

• Comments on output quality on main README

  #166 opened 8 months ago by Anindyadeep
  2

• An issue page which faced when running all benchmarks in A100 GPU.

  #94 opened 8 months ago by Anindyadeep
  3

• ONNX benchmark is not running.

  #100 opened 8 months ago by Anindyadeep
  3

• Additional performance benchmarks metric to give a overall picture of choosing a backend / framework.

  #107 opened 9 months ago by Anindyadeep
  2

• An Evaluation Dataset for quality benchmarking of different inference engine implementation.

  #116 opened 9 months ago by Anindyadeep
  1

• llm-awq

  #77 opened 9 months ago by nsosio
  2

• JAX

  #81 opened a year ago by Anindyadeep
  2

• Check if flash attention is supported on the issue or not, and accordingly update on the benchmark specific readme

  #83 opened a year ago by Anindyadeep
  2

• LM Deploy

  #147 opened 9 months ago by Anindyadeep
  0

• Additional command line argument `--clean` for clean installation.

  #143 opened a year ago by Anindyadeep
  0

• Check for FP-8 format also for Optimum Nvidia

  #115 opened 9 months ago by Anindyadeep
  1

• PowerInfer

  #105 opened a year ago by filopedraz
  0

• An optional docker container creation for each.

  #103 opened 9 months ago by Anindyadeep
  1

• Benchmarks should be computed on other machines too

  #98 opened 9 months ago by filopedraz
  0

• AirLLM

  #87 opened 9 months ago by Anindyadeep
  0

• Lorax

  #84 opened 9 months ago by nsosio
  0

• Mojo

  #82 opened 9 months ago by Anindyadeep
  0

• Setup script have some problems in setting up the environment.

  #78 opened 9 months ago by Anindyadeep
  2

• TinyGrad benchmark not working.

  #101 opened 9 months ago by Anindyadeep
  1

• MLX

  #89 opened a year ago by Anindyadeep
  0

• New engines

  #156 opened a year ago by ogencoglu
  0

• ML Engines

  #148 opened a year ago by nsosio
  0

• Add a readme under each benchmark.

  #88 opened a year ago by Anindyadeep
  1

• Update and re-iterate the missing parts of benchmarks.

  #131 opened a year ago by Anindyadeep
  0

• Latest benchmarks not updating.

  #124 opened a year ago by Anindyadeep
  0

• A template file for only tables as a single source of truth.

  #120 opened a year ago by Anindyadeep
  0

• Include results in main README

  #97 opened a year ago by filopedraz
  1

• Optimum With Nvidia

  #92 opened a year ago by Anindyadeep
  0

• AutoAWQ

  #76 opened a year ago by nsosio
  0

• DeepSpeed FastGen

  #110 opened a year ago by Anindyadeep
  0

• Lightning AI

  #80 opened a year ago by Anindyadeep
  0

• AutoGPTQ

  #75 opened a year ago by nsosio
  0

• Missing Unit of measure (Benchmarking ML engines)

  #96 opened a year ago by vittoriop17
  1

• gpt-fast

  #74 opened a year ago by nsosio
  1