MLC LLM

Universal LLM deployment engine that compiles models to any device.

MLC LLM is an open-source machine-learning compilation engine for deploying LLMs natively across a wide range of hardware, including iOS, Android, browsers (WebGPU), and CPUs/GPUs. Built on Apache TVM, it compiles and optimizes models per-target, and ships MLCEngine with an OpenAI-compatible API for both local and server-side serving. It is a leading project for running LLMs on edge and consumer devices.