Text Generation Inference

Hugging Face's production toolkit for serving and deploying LLMs.

Text Generation Inference (TGI) is Hugging Face's production-grade toolkit for deploying and serving LLMs. It provides optimized inference with features like tensor parallelism, continuous batching, token streaming, and quantization, and integrates closely with the Hugging Face model ecosystem. It powers Hugging Face's hosted inference and is deployable via container for self-hosting.