FreeToken is an edge-native Mixture-of-Experts serving engine. It runs frontier-scale open-weight models on consumer NVIDIA GPUs by sharing the work between the GPU, the CPU and host memory, and serves them through OpenAI and Anthropic-compatible HTTP APIs.
FreeToken Desktop can connect to the optional user backend to share this snap's engine, CUDA runtime and models.
Examples
freetoken serve --model ~/models/Qwen3.6-35B-A3B
freetoken serve --model Qwen/Qwen3.6-35B-A3B --port 1919
freetoken shell
freetoken ctl stats
freetoken checkpoint --model ~/models/<hf-dir> --out ~/models/<ftw-dir>
Requirements
Local inference requires an NVIDIA GPU (RTX 30 series or newer) with driver 580 or newer and the CUDA component:
sudo snap install freetoken+cuda
The remote chat shell and control commands work without the component.
Files
Models downloaded from Hugging Face and runtime caches are kept in ~/snap/freetoken/common. Local models can be read from your home directory. For models on USB drives and other removable media (/media, /mnt), run:
sudo snap connect freetoken:removable-media
An optional background service is disabled until a model is configured with snap set.
You are about to open
Do you wish to proceed?
Thank you for your report. Information you provided will help us investigate further.
There was an error while sending your report. Please try again later.
Snaps are applications packaged with all their dependencies to run on all popular Linux distributions from a single build. They update automatically and roll back gracefully.
Snaps are discoverable and installable from the Snap Store, an app store with an audience of millions.
Snap can be installed on Fedora from the command line:
sudo dnf install snapd
Either log out and back in again, or restart your system, to ensure snap’s paths are updated correctly.
To enable classic snap support, enter the following to create a symbolic link between /var/lib/snapd/snap and /snap:
sudo ln -s /var/lib/snapd/snap /snap
To install FreeToken, simply use the following command:
sudo snap install freetoken
Browse and find snaps from the convenience of your desktop using the snap store snap.
Interested to find out more about snaps? Want to publish your own application? Visit snapcraft.io now.