Deploy and Host PrivateGPT on Railway

PrivateGPT 1.0 is an open source API layer for private AI apps. It handles ingestion, retrieval with citations, agentic RAG, tools and MCP connectors, and exposes a Claude style Messages API. It also ships a workbench UI for chatting with your documents. PrivateGPT does not run models itself. It connects to any OpenAI compatible inference server: OpenAI by default, or your own Ollama, vLLM, llama.cpp or LiteLLM service.

About Hosting PrivateGPT

This template runs PrivateGPT as a private service with its built in API auth switched on, behind a Caddy gateway that is the only public service. The gateway puts a basic auth login in front of the workbench and the API and blocks cross site write requests. The vector store (embedded Qdrant), the document store and the skills database all sit on one volume, so documents survive redeploys.

First steps after deploy:

Open the gateway URL. Sign in as admin with the UI_PASSWORD generated on the privategpt-gateway service (Variables tab). You land on the workbench at /ui/.
Upload a document, then ask questions about it. Answers include citations.
To use the API, send the same credentials as HTTP Basic auth to the gateway URL, for example curl -u admin:$UI_PASSWORD https:///v1/models. Browse the API reference at /docs.

Common Use Cases

Chat with company documents and get answers with citations, without sending files to a hosted RAG product
Give an internal app a Messages API with retrieval, tools and MCP connectors built in
Point the same stack at your own inference server (Ollama, vLLM, LiteLLM) when you move off OpenAI

Dependencies for PrivateGPT Hosting

PrivateGPT (zylonai/private-gpt:1.0.1): private only, volume at /home/worker/app/local_data.
privategpt-gateway (caddy:2.11-alpine): the only public service, with the login prompt.
An OpenAI compatible inference server and its key (OpenAI by default).

Deployment Dependencies

PrivateGPT: https://github.com/zylon-ai/private-gpt
Caddy: https://caddyserver.com
OpenAI API keys: https://platform.openai.com/api-keys

Key Variables

Variable	Service	Purpose
`OPENAI_API_KEY`	PrivateGPT	Required. Key for the inference server. Boot fails if the default model cannot be found.
`OPENAI_API_BASE`	PrivateGPT	Inference server URL. Defaults to `https://api.openai.com/v1`.
`PGPT_LLM_DEFAULT`	PrivateGPT	Default chat model (`gpt-5.4-mini`).
`PGPT_MODELS_EMBEDDING_NAME` / `_EMBED_DIM`	PrivateGPT	Embedding model and vector size (`text-embedding-3-small`, 1536). Set both before your first upload.
`UI_USERNAME` / `UI_PASSWORD`	privategpt-gateway	Login for the gateway. PrivateGPT's own API auth uses the same pair.

Notes: anyone who has the login can run code through the built in code execution tool, read files the container can see (through ingest by URI), and reach other private services, so share it only with people you would trust with the server itself. The image is about 1.2 GB compressed; the Hobby plan or higher is recommended.

Why Deploy PrivateGPT on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying PrivateGPT on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.