Aphrodite Engine

High-throughput LLM inference engine powering large-scale serving.

Aphrodite Engine is the open-source inference backend developed by PygmalionAI for serving LLMs at scale. Built on the vLLM core with PagedAttention and continuous batching, it adds broad quantization support, many sampling methods, and speculative decoding, exposing an OpenAI-compatible API. It is designed to maximize throughput when serving many concurrent users from a single endpoint.