Ollama

Run Llama, Mistral and friends locally with one command.

Ollama is an open-source tool for running large language models locally with a single command. It packages model weights, configuration, and a runtime (built on llama.cpp) into a simple CLI and local HTTP server, handling model downloading, quantization formats, and GPU acceleration across macOS, Linux, and Windows. It exposes a REST API and an OpenAI-compatible endpoint, making local models easy to call from applications.