Deploy and Host nanollm on Railway

A lightweight LLM gateway with OpenAI Chat Completions, Responses, Anthropic Messages, and OpenAI image API support. Manage providers, models, and fallback groups in a browser.

About Hosting nanollm

This template runs nanollm as a single Docker service. Node.js and the Rust OAuth transport helper are built into the image. Configuration and request records use a persistent Railway volume; no separate database is needed.

Why Deploy nanollm on Railway?

Builds directly from the public nanollm repository using its Dockerfile.
Generates a unique NANOLLM_AUTH_TOKEN for each deployment.
Provisions a persistent volume at /data for configuration, subscription credentials, and SQLite request records.
Enables a public Railway domain and checks /health before activating the deployment.
Starts with an empty model list, ready to configure through /admin.

Common Use Cases

Aggregate multiple model providers behind one API endpoint.
Configure model fallback groups and inspect request records in a browser.
Use supported subscription providers through the gateway's admin login flow.

Dependencies for nanollm

Deployment Dependencies

A Railway account is needed to deploy this template. The template provisions its service, public domain, and persistent volume automatically. It generates a gateway access key without requiring user input.

After deploying

Copy NANOLLM_AUTH_TOKEN from the service's Variables tab.
Open https://YOUR_DOMAIN/admin?token=YOUR_TOKEN, replacing YOUR_DOMAIN and YOUR_TOKEN with your service domain and generated token.
Add your providers and models, then save.
Set your API client's base URL to https://YOUR_DOMAIN/v1 and its API key to the generated token.

No Turso database or external API key is required to start the gateway. Configure upstream credentials or subscription login in the admin page when adding models.

Configuration and credentials persist across redeploys on the /data volume. To rotate the gateway key after initialization, update server.auth.token in the configuration and restart the service; changing the environment variable alone does not overwrite an existing configuration file.

Project documentation