---
title: "Deploy gpt-vLLM"
description: "Deploy and host your own LLM with VLLM"
category: "AI/ML"
url: https://railway.com/deploy/gpt-vllm
---

# Deploy gpt-vLLM

Deploy and host your own LLM with VLLM

**[Deploy gpt-vLLM on Railway](https://railway.com/template/gpt-vllm)**

- **Creator:** Jaydip Sanghani's Projects
- **Category:** AI/ML
- **Total deploys:** 17

## Template content

### vLLM-server

- **Image:** jellydeck/vllm-server:latest

### Redis https://devicons.railway.app/i/redis.svg

- **Image:** bitnami/redis:8.2

### api-gateway

- **Source:** jellydeck/gpt-oss
- **Public domain:** Yes

## Documentation

# Deploy and Host LLM with vLLM on Railway

Host your own Large Language Model (LLM) instance for scalable, efficient AI infrastructure. Railway automates deployment, resource scaling, and provides easy API access. Just set your model name (must be [vLLM-compatible](https://docs.vllm.ai/en/stable/models/supported_models.html)) in your environment variables.

---

## How It Works

1. Set `MODEL_NAME` to any supported model in your Railway environment variables:

MODEL_NAME="gpt2"


2. The **api-gateway** listens for requests, enforces authentication with `API_KEY`, and forwards them to your vLLM backend.
3. OpenAI-compatible APIs let clients and apps connect with zero code changes.

---

**Tip:** For production, always set an `API_KEY`, use larger resource plans for best performance, and consult the [vLLM Supported Models](https://docs.vllm.ai/en/stable/models/supported_models.html) doc for compatible model names.

## API Gateway Template

**api-gateway** is an included starter template that acts as your secure front API layer. Users interact through this gateway using an `API_KEY` for authentication. The gateway forwards requests to your vLLM backend and helps enforce security and rate limiting in production.

---
## API Gateway Template

**api-gateway** is an included starter template that acts as your secure front API layer. Users interact through this gateway using an `API_KEY` for authentication. The gateway forwards requests to your vLLM backend and helps enforce security and rate limiting in production.

---

## About Hosting something-oss

Railway makes it easy to run inference servers, caching layers (like Redis), and API gateways with minimal setup. Persistent storage and private networking work out-of-the-box, so your AI stack can scale automatically and is always accessible via OpenAI-compatible API paths.

---

## Common Use Cases

- Deploy private, secure AI chatbots or assistants
- Host scalable machine learning APIs with batching and caching
- Integrate OpenAI-compatible APIs into full-stack apps

---

## Dependencies

- [vLLM](https://docs.vllm.ai/) for model serving and OpenAI-compatible routing
- [Redis](https://redis.io/docs/) for fast caching
- 5–30GB volume storage (depending on model size)
- 8–30GB RAM (depending on model size)
- 8–30 vCPUs (for higher throughput and concurrency)

---

## Pricing & Capacity

Resource usage drives hosting costs:
- Small models (up to 2B parameters): modest CPU and RAM; lower monthly price
- Large models (GPT-OSS, Llama 20B+): higher RAM and CPU needed; expect higher costs via Railway’s dashboard

---

## Why Deploy on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying GPT-vLLM on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway


## Similar templates

- [Chat Chat](https://railway.com/deploy/-WWW5r) — Chat Chat, your own unified chat and search to AI platform.
- [stella](https://railway.com/deploy/stella) — Self-host stella with web, API, Postgres, Redis, and object storage.
- [Hermes Agent | OpenClaw Alternative with Dashboard](https://railway.com/deploy/hermes-agent-or-openclaw-alternative-wit) — Self-Hosted Hermes AI Agent for Telegram, Discord & Slack

Open this page in a browser: https://railway.com/deploy/gpt-vllm