Rozalia

Rozalia lends the language model hardware sitting on your desk to the people you choose, behind one OpenAI-compatible endpoint.

The machines with the GPUs live at home, behind NAT, where nothing on the internet can dial them. So they dial out instead: rozalia-proxy opens a WebSocket to the server and holds it there, and the server pushes requests down it, to whichever machine advertises the model asked for and has the fewest requests in flight.

This snap ships the proxy. Lending a machine takes a one-time enrollment code. If your account on the server may already lend hardware, that is self-service: sign in, open your account page and create one under Enrollment codes. Otherwise ask whoever runs the server for one. Then enrol the machine, say what to offer and where, and turn it on:

 sudo rozalia.proxyctl enroll -server https://ai.example.org \
     -code rze_...
 sudo snap set rozalia \
     proxy-server=https://ai.example.org \
     proxy-backends=http://127.0.0.1:11434/v1 \
     proxy=true

proxy-backends takes a comma-separated list, for a machine running more than one inference server. proxy-concurrency says how many requests to run at once, and defaults to one, because a single GPU answering two completions at once finishes neither sooner.

The cloud half is optional and ships as a separate component, so a machine that only lends its GPU never downloads it:

 sudo snap install rozalia+server
 sudo snap set rozalia \
     base-url=https://ai.example.org \
     listen-addr=127.0.0.1:8080
 sudo snap set rozalia server=true
 sudo rozalia.serverctl account create -email you@example.org -admin

The server does not start on install: a fresh database has no accounts, and the address it should listen on is not something a package can guess.

The stable channel carries tagged releases. Edge is built from every commit to the main branch, for trying a fix before it is released.