Deploy PrivateGPT

Private document chat and RAG API with a bundled Ollama, no API key needed

Deploy PrivateGPT

/root/.ollama

/home/worker/app/local_data

Deploy and Host PrivateGPT on Railway

PrivateGPT is an open-source API layer by Zylon for building private AI applications. It ingests documents, retrieves answers with citations, runs agentic RAG, tools and MCP connectors, and exposes a Claude-style Messages API. A built-in workbench UI lets you chat with your documents. It connects to any OpenAI-compatible model server.

About Hosting PrivateGPT

This template runs PrivateGPT 1.0.1 next to its own Ollama model server, so documents and prompts never leave your Railway project. Both services build from a public GitHub repo, so you get update notices when the template changes. On first start, Ollama downloads qwen3:1.7b for chat and mxbai-embed-large:335m for embeddings (about 2 GB). PrivateGPT answers its health check and waits until both models are ready, which takes a few minutes. The vector store, document store and skills database sit on a volume, and the models sit on a second one. The API is protected by HTTP Basic auth with a generated password. Models run on CPU, so expect replies to take seconds, not milliseconds.

Common Use Cases

  • Chat with internal documents and get answers with citations, without sending files to a hosted AI service
  • Give your own apps, agents or n8n workflows a private Messages API with retrieval, tools and MCP connectors
  • Try local open models on Railway, then switch to a larger model or a hosted provider by changing a few variables

Dependencies for PrivateGPT Hosting

  • PrivateGPT 1.0.1 (official image zylonai/private-gpt, built from the template repo)
  • Ollama 0.40.16 (official image ollama/ollama, private network only) with an embedding model and a chat model

Deployment Dependencies

Implementation Details

After the deploy finishes and the models are downloaded:

  1. Open https://YOUR-DOMAIN/ui/. In the guided setup, keep the base URL, open Authentication (HTTP Basic Auth) and enter admin and the PRIVATEGPT_PASSWORD from the PrivateGPT service's Variables tab.
  2. Upload documents and ask questions about them.
  3. Call the API with the same credentials, for example curl -u admin:PASSWORD https://YOUR-DOMAIN/v1/models. The API reference is at /docs.

Variables you may want to change:

  • OLLAMA_LLM_MODEL and OLLAMA_EMBEDDING_MODEL on Ollama: any exact name:tag from the Ollama library. Bigger models answer better but need more memory and CPU. If the embedding size isn't 1024, also set PGPT_EMBED_DIM on PrivateGPT before you ingest documents.
  • To use OpenAI or another hosted provider instead, set OPENAI_API_BASE, OPENAI_API_KEY, PGPT_LLM_DEFAULT and PGPT_EMBEDDING_DEFAULT on PrivateGPT, then remove the Ollama service.

Why Deploy PrivateGPT on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying PrivateGPT on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.


Template Content

More templates in this category

View Template
Chat Chat
Chat Chat, your own unified chat and search to AI platform.

okisdev
117
View Template
Weaviate
Vector database with hybrid search and an API key-protected API

GitHub Locker
1
View Template
Möbius Personal AI OS
Self-hosted AI agent that builds and runs apps for you, with memory.

bento
1