/TensorRT-LLM

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

Primary LanguageC++Apache License 2.0Apache-2.0

Pinned issues

[RFC]Topics you want to discuss with TensorRT-LLM team in the upcoming meet-ups

#3124 opened by juney-nvidia

Open11

[RFC]Feedback collection about TensorRT-LLM 1.0 Release Planning and API Compatibility Commitment

#3148 opened by juney-nvidia

Open4

Issues