Deploy and Host AnythingLLM on Railway

AnythingLLM is an all-in-one, open-source AI application by Mintplex Labs — a private workspace for chatting with your documents (RAG), running AI agents, and managing multiple users, all in one self-hosted instance. It connects to OpenAI, Anthropic, Ollama, and many other providers, keeps your documents and conversations on infrastructure you control, and is MIT-licensed with no usage caps or paid tier for self-hosting.

This template deploys AnythingLLM in a single click, with a persistent volume so your workspaces and data survive redeploys.

What This Template Deploys

Service	Purpose
AnythingLLM	The full workspace — chat UI, RAG pipeline, agents, admin, and API on port `3001`

A single self-contained service. AnythingLLM bundles its own SQLite database and LanceDB vector store, so there is no separate database service to run — everything persists on one mounted volume.

About Hosting AnythingLLM

AnythingLLM is deliberately batteries-included. Upload PDFs, DOCX, CSV, or Markdown, and it embeds them for semantic search automatically. Point it at a model provider — cloud or local — and you have a private ChatGPT-style workspace over your own knowledge base. It supports multi-user mode with workspace-level access control, agent workflows, and a full developer API.

The one thing that matters most when hosting it: all state lives in a single storage directory, and that directory must be on a persistent volume. Workspaces, uploaded documents, vector embeddings, users, and settings all live under /app/server/storage. Without a volume mounted there, everything is wiped on every redeploy. This template mounts that volume for you, so your data persists.

Because AnythingLLM uses an embedded SQLite database, it runs as a single instance rather than scaling horizontally — which is exactly right for a self-hosted team workspace and keeps the deployment simple and cheap.

Typical cost: ~$5–10/month on Railway's Hobby plan, plus whatever your chosen model provider charges. Connect a cloud LLM and no GPU is needed — Railway runs the app on CPU and the model does the heavy lifting on the provider's side.

Deploy in Under 5 Minutes

Click Deploy on Railway and wait for the build (~2–3 minutes)
Confirm the volume is mounted at /app/server/storage and STORAGE_DIR points to it
Set LLM_PROVIDER and the matching API key (e.g. OPEN_AI_KEY or ANTHROPIC_API_KEY)
Open the generated Railway domain and complete the onboarding wizard — the first account becomes admin
Create a workspace, upload a document, and start chatting with your knowledge base

No Docker setup, no separate database to provision, no GPU required for cloud models.

Common Use Cases

Chat with your documents (RAG) — upload PDFs, contracts, docs, and CSVs and ask questions answered from their contents
Private internal knowledge base — a team AI assistant grounded in company documents, with data kept on your own infrastructure
Multi-user AI workspace — shared workspaces with per-user access control for a whole team on one instance
AI agents — build agent workflows that browse, call tools, and act on your data from inside the workspace
Bring-your-own-model flexibility — switch between OpenAI, Anthropic, Ollama, and others without changing tools
Privacy-sensitive AI — nothing leaves your container except calls to the model provider you explicitly choose

Configuration

Variable	Required	Description
`STORAGE_DIR`	Required	`/app/server/storage` — must point at the mounted volume, or all data is lost on redeploy
`JWT_SECRET`	Required	Signs user session tokens — set a long random string and keep it stable
`LLM_PROVIDER`	Required	Model provider, e.g. `openai`, `anthropic`, `ollama`
`OPEN_AI_KEY`	Conditional	OpenAI API key when `LLM_PROVIDER=openai`
`ANTHROPIC_API_KEY`	Conditional	Anthropic API key when `LLM_PROVIDER=anthropic`
`EMBEDDING_ENGINE`	Optional	Embedding provider, e.g. `openai` or `native` (default local embeddings)
`VECTOR_DB`	Pre-set	`lancedb` — the bundled vector store, no external service needed
`PORT`	Pre-set	`3001`
`PASSWORDMINCHAR`	Optional	Minimum password length for multi-user accounts
`DISABLE_TELEMETRY`	Optional	Set to `true` to disable anonymous telemetry

The volume is not optional. STORAGE_DIR must resolve to a mounted Railway volume at /app/server/storage. Workspaces, documents, embeddings, users, and settings all live there — without the volume, a redeploy erases everything.

Keep JWT_SECRET stable. It signs login sessions. Changing it after deploy logs every user out.

Dependencies for AnythingLLM Hosting

Railway account — Hobby plan (~$5–10/month) runs the workspace
An API key for a cloud LLM provider (OpenAI, Anthropic, etc.), unless using a remote Ollama
Optional: a reachable Ollama instance for local/self-hosted models (Railway provides no GPU)
Optional: an external vector database if you outgrow the bundled LanceDB

Deployment Dependencies

Implementation Details

The template runs the official mintplexlabs/anythingllm image on port 3001 with a Railway volume mounted at /app/server/storage and STORAGE_DIR set to match. AnythingLLM keeps its application database in SQLite and its embeddings in a bundled LanceDB vector store, both inside that storage directory — so a single persistent volume covers the entire application state and there is no separate database service to wire up.

JWT_SECRET secures multi-user sessions and must remain constant across restarts. VECTOR_DB defaults to lancedb, the embedded store, which keeps the deployment to one service; you can point it at an external vector database later if scale demands it.

Because the SQLite backend is single-writer, AnythingLLM is designed to run as a single instance rather than scaling horizontally. This suits a self-hosted team workspace well and keeps both the architecture and the Railway bill simple. Connecting a cloud LLM means the container only orchestrates requests, so it runs comfortably on CPU with no GPU.

Frequently Asked Questions

Does my data survive a redeploy? Only if the volume is mounted at /app/server/storage and STORAGE_DIR points to it. All workspaces, documents, embeddings, and users live there. This template mounts it for you — leave the configuration in place.

Do I need a GPU? No, not for cloud models. AnythingLLM runs on CPU and delegates inference to your provider (OpenAI, Anthropic, etc.). A GPU only matters if you run local inference, which would be on a separate Ollama host since Railway provides no GPU.

Which LLM providers are supported? OpenAI, Anthropic, Ollama, LocalAI, and many others, out of the box. Set LLM_PROVIDER and the matching key, and switch providers any time from the admin settings.

What files can I upload? PDF, DOCX, CSV, Markdown, and plain text. AnythingLLM embeds them automatically for semantic search within a workspace.

Can a whole team use one instance? Yes. Multi-user mode gives you shared workspaces with per-user access control, all on a single deployment.

Is it really free? The self-hosted version is MIT-licensed with no usage caps, no paid tier, and no feature restrictions. Your only costs are Railway infrastructure and your model provider's usage.

Can I scale it to multiple replicas? No. The embedded SQLite database is single-writer, so AnythingLLM runs as one instance. Scale vertically on Railway rather than adding replicas.

Why Deploy AnythingLLM on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying AnythingLLM on Railway you get a private, all-in-one AI workspace in a single click — RAG over your own documents, AI agents, multi-user access, automatic HTTPS, and a persistent volume that keeps your data safe across redeploys. Bring your own model, keep your documents on infrastructure you own, and run your AI workspace alongside the rest of your stack.