Osaurus

Osaurus is an open-source, native local inference server for Apple Silicon Macs. It loads language models onto your machine and serves them through an OpenAI-compatible API, so agents, editors, and scripts that already speak the ChatGPT API format can point at localhost instead of a cloud endpoint. Everything — prompts, code, files — stays on the device. Nothing leaves your Mac.

Built by Dinoki AI on Apple's MLX framework rather than a generic cross-platform runtime, Osaurus is tuned specifically for the M-series unified memory architecture. That's the pitch against the more familiar Ollama: a lightweight, Swift-native app that targets Mac hardware directly instead of running the same binary everywhere. The OpenAI-compatible endpoint is the important part for developers — it means you can wire it into existing coding assistants, agent frameworks, and automation flows without rewriting your integration.

It's aimed at developers who want offline, privacy-preserving AI for coding and local automation: people working with sensitive codebases, folks on flaky connections, or anyone who'd rather not send every keystroke to a third-party API. The trade-off is the obvious one for local models — you're capped by your Mac's RAM and by whatever open-weight models you can realistically run, which won't match frontier cloud models on the hardest tasks.