Rozalia

Rozalia lends the language model hardware sitting on your desk to the people you choose, behind one OpenAI-compatible endpoint.

The machines with the GPUs live at home, behind NAT, where nothing on the internet can dial them. So they dial out instead: rozalia-proxy opens a WebSocket to the server and holds it there, and the server pushes requests down it, to whichever machine advertises the model asked for and has the fewest requests in flight.

This snap ships the proxy. Enrol the machine with a one-time code from whoever runs the server, say what to offer and where, and turn it on:

 sudo rozalia.proxyctl enroll -server https://ai.example.org -code rze_...
 sudo snap set rozalia \
     proxy-server=https://ai.example.org \
     proxy-backends=http://127.0.0.1:11434/v1 \
     proxy=true

proxy-backends takes a comma-separated list, for a machine running more than one inference server. proxy-concurrency says how many requests to run at once, and defaults to one, because a single GPU answering two completions at once finishes neither sooner.

The cloud half is optional and ships as a separate component, so a machine that only lends its GPU never downloads it:

 sudo snap install rozalia+server
 sudo snap set rozalia base-url=https://ai.example.org listen-addr=127.0.0.1:8080
 sudo snap set rozalia server=true
 sudo rozalia.serverctl account create -email you@example.org -admin

The server does not start on install: a fresh database has no accounts, and the address it should listen on is not something a package can guess.