llama.cpp packaged as a snap. The main snap ships a CPU build that works on any x86_64 or arm64 machine. Optional snap components add GPU backends:
sudo snap install --edge --devmode llama-cpp # CPU
sudo snap install --edge --devmode llama-cpp+hip # add AMD GPU
sudo snap install --edge --devmode llama-cpp+cuda # add NVIDIA GPU
Select a backend persistently with:
sudo snap set llama-cpp backend={cpu,hip,cuda,auto} # auto = probe
snap get llama-cpp backend
or override it per invocation with the env var:
LLAMA_CPP_BACKEND=cpu llama-cpp cli ...
You are about to open
Do you wish to proceed?
Thank you for your report. Information you provided will help us investigate further.
There was an error while sending your report. Please try again later.
Snaps are applications packaged with all their dependencies to run on all popular Linux distributions from a single build. They update automatically and roll back gracefully.
Snaps are discoverable and installable from the Snap Store, an app store with an audience of millions.
Snap is available for CentOS 7.6+, and Red Hat Enterprise Linux 7.6+, from the Extra Packages for Enterprise Linux (EPEL) repository. The EPEL repository can be added to your system with the following command:
sudo yum install epel-release
Snap can now be installed as follows:
sudo yum install snapd
Once installed, the systemd unit that manages the main snap communication socket needs to be enabled:
sudo systemctl enable --now snapd.socket
To enable classic snap support, enter the following to create a symbolic link between /var/lib/snapd/snap and /snap:
sudo ln -s /var/lib/snapd/snap /snap
Either log out and back in again, or restart your system, to ensure snap’s paths are updated correctly.
To install llama-cpp, simply use the following command:
sudo snap install llama-cpp
Browse and find snaps from the convenience of your desktop using the snap store snap.
Interested to find out more about snaps? Want to publish your own application? Visit snapcraft.io now.