triton_trtllm_guide

Installation and usage guide for Triton TRT-LLM v0.7.1

GPT Model

1. Follow the NGC guide if you are using NGC Triton TRT-LLM container and want to deploy GPT model
2. Follow the Build Your Own Container (BYOC) guide if you are building your Triton TRT-LLM container and want to deploy GPT model

LLaMa

Use README_LLaMA_BYOCTritonTRTLLM for LLaMa model

Falcon-40B

Use Falcon_BYOCTritonTRTLLM.md guide to deploy Falcon-40B on 8 A100 GPU