Neural Magic

Pinned Repositories

• AutoFP8

  Language:Python181152824

• compressed-tensors

  A safetensors extension to efficiently store sparse quantized tensors on disk

  Language:Python91121710

• deepsparse

  Sparsity-aware deep learning inference runtime for CPUs

  Language:Python3.1k55143181

• docs

  Top-level directory for documentation and general content

  Language:MDX1202237

• guidellm

  Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

  Language:Python232183623

• nm-vllm

  A high-throughput and memory-efficient inference and serving engine for LLMs

  Language:Python26282011

• nm-vllm-certs

  General Information, model certifications, and benchmarks for nm-vllm enterprise distributions

  11511

• sparseml

  Libraries for applying sparsification recipes to neural networks with a few lines of code, enabling faster and smaller models

  Language:Python2.1k47210151

• sparsezoo

  Neural network model repository for highly sparse and sparse-quantized models with matching sparsification recipes

  Language:Python382232326

• sparsify

  ML model optimization product to accelerate inference.

  Language:Python326262430

Neural Magic's Repositories