Local model inference with OpenAI, Anthropic and Ollama APIs
Run local models through a native command-line server and browser console.
MLX-Serve provides OpenAI, Anthropic and Ollama-compatible HTTP APIs and
downloads models from Hugging Face.
This Linux package requires a supported NVIDIA GPU and host driver 580 or
newer. It includes the CUDA runtime and supports amd64 and arm64 systems.
The optional system daemon is disabled until explicitly configured.
(Ownership verified)
The publisher has verified that they own this domain.
It does not guarantee the Snap is an official upload
from the upstream project.
Snaps are applications packaged with all their dependencies to run on all popular Linux distributions from a single build. They update automatically and roll back gracefully.
Snaps are discoverable and installable from the Snap Store, an app store with an audience of millions.
Enable snapd
On Debian 9 (Stretch) and newer, snap can be installed directly from the command line:
sudo apt update
sudo apt install snapd
After this, install the snapd snap in order to get the latest snapd:
sudo snap install snapd
Install MLX-Serve
To install MLX-Serve, simply use the following command: