bnellnm

Pinned Repositories

• text-generation-inference

  Large Language Model Text Generation Inference

  Language:Python8.7k961.3k1k

• deepsparse

  Sparsity-aware deep learning inference runtime for CPUs

  Language:Python3k56134171

• nm-vllm

  A high-throughput and memory-efficient inference and serving engine for LLMs

  Language:Python2478189

• sparseml

  Libraries for applying sparsification recipes to neural networks with a few lines of code, enabling faster and smaller models

  Language:Python2k48205140

• sparsezoo

  Neural network model repository for highly sparse and sparse-quantized models with matching sparsification recipes

  Language:Python362262124

• sparsify

  ML model optimization product to accelerate inference.

  Language:Python316272228

• vllm

  A high-throughput and memory-efficient inference and serving engine for LLMs

  Language:Python25.6k2214.2k3.7k

bnellnm's Repositories