
Deploy PrivateGPT
Private document chat and RAG API with a bundled Ollama, no API key needed
Just deployed
/root/.ollama
PrivateGPT
Just deployed
/home/worker/app/local_data
Deploy and Host PrivateGPT on Railway
PrivateGPT is an open-source API layer by Zylon for building private AI applications. It ingests documents, retrieves answers with citations, runs agentic RAG, tools and MCP connectors, and exposes a Claude-style Messages API. A built-in workbench UI lets you chat with your documents. It connects to any OpenAI-compatible model server.
About Hosting PrivateGPT
This template runs PrivateGPT 1.0.1 next to its own Ollama model server, so documents and prompts never leave your Railway project. Both services build from a public GitHub repo, so you get update notices when the template changes. On first start, Ollama downloads qwen3:1.7b for chat and mxbai-embed-large:335m for embeddings (about 2 GB). PrivateGPT answers its health check and waits until both models are ready, which takes a few minutes. The vector store, document store and skills database sit on a volume, and the models sit on a second one. The API is protected by HTTP Basic auth with a generated password. Models run on CPU, so expect replies to take seconds, not milliseconds.
Common Use Cases
- Chat with internal documents and get answers with citations, without sending files to a hosted AI service
- Give your own apps, agents or n8n workflows a private Messages API with retrieval, tools and MCP connectors
- Try local open models on Railway, then switch to a larger model or a hosted provider by changing a few variables
Dependencies for PrivateGPT Hosting
- PrivateGPT 1.0.1 (official image
zylonai/private-gpt, built from the template repo) - Ollama 0.40.16 (official image
ollama/ollama, private network only) with an embedding model and a chat model
Deployment Dependencies
- PrivateGPT source and releases: https://github.com/zylon-ai/private-gpt
- PrivateGPT documentation: https://docs.privategpt.dev/
- Ollama model library: https://ollama.com/library
- Template repo: https://github.com/aalfath/privategpt-railway-template
Implementation Details
After the deploy finishes and the models are downloaded:
- Open
https://YOUR-DOMAIN/ui/. In the guided setup, keep the base URL, open Authentication (HTTP Basic Auth) and enteradminand thePRIVATEGPT_PASSWORDfrom the PrivateGPT service's Variables tab. - Upload documents and ask questions about them.
- Call the API with the same credentials, for example
curl -u admin:PASSWORD https://YOUR-DOMAIN/v1/models. The API reference is at/docs.
Variables you may want to change:
OLLAMA_LLM_MODELandOLLAMA_EMBEDDING_MODELon Ollama: any exactname:tagfrom the Ollama library. Bigger models answer better but need more memory and CPU. If the embedding size isn't 1024, also setPGPT_EMBED_DIMon PrivateGPT before you ingest documents.- To use OpenAI or another hosted provider instead, set
OPENAI_API_BASE,OPENAI_API_KEY,PGPT_LLM_DEFAULTandPGPT_EMBEDDING_DEFAULTon PrivateGPT, then remove the Ollama service.
Why Deploy PrivateGPT on Railway?
Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.
By deploying PrivateGPT on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.
Template Content

