zrt is a command-line tool that wraps vLLM to provide a streamlined experience for serving large language models on Linux. It handles environment setup, model lifecycle management, and exposes an OpenAI-compatible HTTP API - making it easy to pull, serve, and observe LLM workloads with a few simple commands.

Designed for Linux servers with GPU workloads, zrt is useful for developers and teams who want a production-ready alternative to Ollama backed by the vLLM inference engine.

For documentation, visit our zdocs page at https://zdocs.datascience.hp.com/docs/zruntime/overview

Found an issue? Have a feature request? Open an issue at https://github.com/HPInc/zrt-support