Installation and usage guide for Triton TRT-LLM v0.7.1
- Follow the NGC guide if you are using NGC Triton TRT-LLM container and want to deploy GPT model
- Follow the Build Your Own Container (BYOC) guide if you are building your Triton TRT-LLM container and want to deploy GPT model
Use README_LLaMA_BYOCTritonTRTLLM for LLaMa model
Use Falcon_BYOCTritonTRTLLM.md guide to deploy Falcon-40B on 8 A100 GPU