Deploy and Host GPT-Load on Railway

GPT-Load is a high-performance proxy and API management tool designed to distribute traffic across multiple OpenAI-compatible and LLM service providers. It offers request load balancing, detailed token usage tracking, and persistent configuration management. It is built for developers, teams, and service providers who need reliable API access management and failover support.

About Hosting GPT-Load

Hosting GPT-Load on Railway provides an isolated and scalable environment to manage and route your AI traffic. Railway automatically builds and deploys the container using the official prebuilt Docker image or repository configuration. The deployment requires internal listening on port 3001 with HTTP public networking exposed via a generated Railway domain.

Persistent storage is essential to retain SQLite configuration data and log files across service restarts. For enhanced multi-node or high-availability setups, GPT-Load can optionally integrate with Railway PostgreSQL and Redis services. Railway manages automatic SSL certificate generation, domain routing, and environment variable injection to streamline the infrastructure setup.

Common Use Cases

API Traffic Load Balancing: Distribute incoming completion and chat requests across multiple API keys or providers to prevent rate limits and ensure maximum uptime.
Centralized API Management: Unify access keys, monitor request volumes, and manage administrative settings through a single dashboard interface.
LLM Gateway Proxy: Serve as an intermediary gateway between frontend applications and backend LLM providers with customizable request timeouts and CORS settings.

Dependencies for GPT-Load Hosting

Dependency	Type	Requirement	Description
Railway Volume	Storage	Required	Persists local database configurations, logs, and application state at `/app/data`.
PostgreSQL	Database	Optional	Replaces default SQLite database for centralized data storage.
Redis	In-Memory Cache	Optional	Handles high-concurrency state caching and distributed session management.

Deployment Dependencies

Implementation Details

Docker

The application deploys using the official Docker image:

ghcr.io/tbphp/gpt-load:latest

Railway automatically handles deployment when referencing this image.

Public Networking

Setting	Value
Proxy Type	HTTP Proxy
Target Port	`3001`

Generate a public domain under Service Settings → Networking → Generate Domain.

Environment Variables

Variable	Required	Description
`AUTH_KEY`	Yes	Secret password required to secure administrative API and management UI.
`PORT`	Yes	Internal HTTP listening port (default: `3001`).
`HOST`	Yes	Interface binding address for Railway (default: `0.0.0.0`).
`DATABASE_DSN`	No	PostgreSQL connection string (`${{Postgres.DATABASE_URL}}`). Uses SQLite if empty.
`REDIS_DSN`	No	Redis connection string (`${{Redis.REDIS_URL}}`).
`ALLOW_CREDENTIALS`	No	Toggles credential allowance for CORS requests (default: `false`).
`ALLOWED_HEADERS`	No	Allowed CORS headers (default: `*`).
`ALLOWED_METHODS`	No	Allowed CORS HTTP methods (default: `GET,POST,PUT,DELETE,OPTIONS`).
`ALLOWED_ORIGINS`	No	Allowed CORS origin patterns (default: `*`).
`ENABLE_CORS`	No	Enables cross-origin resource sharing (default: `true`).
`IS_SLAVE`	No	Toggles execution in slave node mode (default: `false`).
`LOG_ENABLE_FILE`	No	Enables logging to file storage (default: `true`).
`LOG_FILE_PATH`	No	Output path for log files (default: `./data/logs/app.log`).
`LOG_FORMAT`	No	Log output format type (default: `text`).
`LOG_LEVEL`	No	Level of logging detail (default: `info`).
`MAX_CONCURRENT_REQUESTS`	No	Maximum allowed concurrent requests limit (default: `100`).
`SERVER_GRACEFUL_SHUTDOWN_TIMEOUT`	No	Graceful server shutdown timeout window in seconds (default: `10`).
`SERVER_IDLE_TIMEOUT`	No	Maximum connection idle duration in seconds (default: `120`).
`SERVER_READ_TIMEOUT`	No	Maximum request read duration in seconds (default: `60`).
`SERVER_WRITE_TIMEOUT`	No	Maximum response write duration in seconds (default: `600`).
`TZ`	No	Timezone configuration (default: `Asia/Shanghai`).

AUTH_KEY=your_secure_admin_key_here
PORT=3001
HOST=0.0.0.0
ALLOW_CREDENTIALS=false
ALLOWED_HEADERS=*
ALLOWED_METHODS=GET,POST,PUT,DELETE,OPTIONS
ALLOWED_ORIGINS=*
ENABLE_CORS=true
IS_SLAVE=false
LOG_ENABLE_FILE=true
LOG_FILE_PATH=./data/logs/app.log
LOG_FORMAT=text
LOG_LEVEL=info
MAX_CONCURRENT_REQUESTS=100
SERVER_GRACEFUL_SHUTDOWN_TIMEOUT=10
SERVER_IDLE_TIMEOUT=120
SERVER_READ_TIMEOUT=60
SERVER_WRITE_TIMEOUT=600
TZ=Asia/Shanghai

Persistent Storage

Persistent storage is required to store SQLite data and application logs.

Open your Railway Service.
Go to Settings → Volumes.
Click Add Volume.
Set the Mount Path:

/app/data

Database

SQLite (Default): Enabled automatically when DATABASE_DSN is empty. Stores data inside /app/data.
PostgreSQL (Optional): Add a PostgreSQL plugin to your Railway project and set DATABASE_DSN=${{Postgres.DATABASE_URL}}.
Redis (Optional): Add a Redis plugin to your Railway project and set REDIS_DSN=${{Redis.REDIS_URL}}.

Build & Start

No custom build or start commands are required.

Accessing the Application

Open Service Settings → Networking and click Generate Domain.
Access the dashboard via the generated Railway URL (e.g., https://.up.railway.app).
Log in to the management interface using the password configured in AUTH_KEY.
Navigate to System Settings and update the project address field with your public Railway URL.

Why Deploy GPT-Load on Railway?

Railway provides a seamless platform for deploying web applications, databases, and microservices with zero configuration. Featuring automatic SSL provisioning, integrated environment variable management, instant database deployments, and continuous delivery from source code repositories or Docker images, Railway reduces infrastructure overhead so developers can focus entirely on building software.