truenorth8

Pinned Repositories

• amazon-ecs-ami

  Packer recipes for building the official ECS-optimized Amazon Linux AMIs

  Language:Shell212263949

• text-generation-inference

  Large Language Model Text Generation Inference

  Language:Python9.9k1031.5k1.2k

• text-generation-inference

  IBM development fork of https://github.com/huggingface/text-generation-inference

  Language:Python60141533

• ZipLM

  Code for the NeurIPS 2023 paper: "ZipLM: Inference-Aware Structured Pruning of Language Models".

  21210

• deepsparse

  Sparsity-aware deep learning inference runtime for CPUs

  Language:Python3.1k56143181

• text-generation-webui

  A Gradio web UI for Large Language Models with support for multiple inference backends.

  Language:Python42.9k3453.8k5.5k

• exllamav2

  A fast inference library for running LLMs locally on modern consumer-class GPUs

  Language:Python3.8k35489290

truenorth8's Repositories