llama.cpp packaged as a snap. The main snap ships a CPU build that works on any x86_64 or arm64 machine. Optional snap components add GPU backends:
sudo snap install llama-cpp # CPU sudo snap install llama-cpp+hip # add AMD GPU sudo snap install llama-cpp+cuda # add NVIDIA GPU sudo snap install llama-cpp+vulkan # add Vulkan GPU (Intel, AMD, etc.) sudo snap install llama-cpp+opencl # add OpenCL GPU (Adreno, Mali, etc.) sudo snap install llama-cpp+sycl # add SYCL GPU (Intel)
Select a backend persistently with:
sudo snap set llama-cpp backend={cpu,hip,cuda,vulkan,opencl,sycl,auto} # auto = probe snap get llama-cpp backend
or override it per invocation with the env var:
LLAMA_CPP_BACKEND=cpu llama-cpp cli ...
The llama.cpp web server can also be run using the llama-cpp.server app. This is disabled by default but can be enabled by running:
snap start --enable llama-cpp.server
By default, no additional arguments are passed to the server, meaning it will listen on the default port and address. Arguments can be specified by setting the server-opts configuration value, e.g.:
snap set llama-cpp server-opts="--host 0.0.0.0"
Afterwards, the server must be restarted for the new arguments to take effect:
snap restart llama-cpp.server
You are about to open
Do you wish to proceed?
Thank you for your report. Information you provided will help us investigate further.
There was an error while sending your report. Please try again later.
Snaps are applications packaged with all their dependencies to run on all popular Linux distributions from a single build. They update automatically and roll back gracefully.
Snaps are discoverable and installable from the Snap Store, an app store with an audience of millions.
Snap can be installed from the command line. Open the Konsole terminal and enter the following:
sudo apt update
sudo apt install snapd
To install llama-cpp, simply use the following command:
sudo snap install llama-cpp
Browse and find snaps from the convenience of your desktop using the snap store snap.
Interested to find out more about snaps? Want to publish your own application? Visit snapcraft.io now.