Deploy AnythingLLM: Private ChatGPT for Your Docs & AI Agents

Deploy and Host AnythingLLM on Railway

AnythingLLM is an open source AI workspace from Mintplex Labs that lets your team chat with its own documents, run AI agents and share private workspaces from one web app. It is MIT-licensed, has about 68,000 GitHub stars, and works with OpenAI, Anthropic, Gemini, Groq, OpenRouter, Ollama and many other model providers. Upload PDFs, Word files, spreadsheets or web pages, and it chunks, embeds and stores them in a built-in vector database so answers cite your sources. This template runs the Docker edition on Railway, the version with multi-user accounts, workspace permissions and embeddable chat widgets, on one service with a persistent volume. Hosted AnythingLLM Cloud starts at $50/month. Here the app costs about $5–10/month on Railway, plus whatever your model provider bills for tokens.

What This Template Deploys

There is no separate database service. Everything AnythingLLM stores lives on the volume, so workspaces and embeddings survive redeploys.

No public domain is set. After the first deploy, open the service's Settings and Networking and click Generate Domain for port 3001 to reach the app.

About Hosting AnythingLLM

Most "chat with your PDFs" setups need several moving parts: a chat UI, an embedding model, a vector database, a document parser and something to manage users. AnythingLLM ships all of them in one container. The default embedder runs on CPU inside the app, LanceDB stores vectors on disk, and SQLite holds users, chats and settings.

Bring your own model. AnythingLLM does not include an LLM. You pick a provider during setup and paste its API key. Cloud models need no GPU, so a small Railway service is enough. To keep everything private, point it at an Ollama server you run yourself.

Workspaces keep knowledge separate. Each workspace has its own documents, system prompt, model settings and chat history. Sales can query the pricing sheets while engineering asks about the runbooks, without either seeing the other's files.

Data connectors. Besides file uploads, AnythingLLM can pull in website pages, GitHub and GitLab repositories, YouTube transcripts and Confluence spaces, then keep them as documents you can attach to any workspace.

Lock it down on day one. The app is open to anyone with the URL until you protect it. In the setup wizard choose a password for single-user use, or turn on multi-user mode and create an admin account. Multi-user mode cannot be switched back to single-user later.

One instance, by design. Because data sits in SQLite and LanceDB on one volume, AnythingLLM runs as a single replica. Scale it up with more RAM and CPU rather than more copies.

Typical cost: ~$5–10/month on Railway for the app and volume at team scale, plus model usage. AnythingLLM Cloud is $50/month for Basic and $99/month for Pro.

Deploy AnythingLLM in Under 5 Minutes

1. Click Deploy on Railway. The image starts in a minute or two.
2. Open the service, go to Settings and Networking, and generate a public domain for port 3001.
3. Open the domain. The setup wizard asks for an LLM provider and API key; the built-in embedder and LanceDB are the defaults.
4. Choose who will use it: set a password for yourself, or enable multi-user mode and create the admin account.
5. Create a workspace, upload a few documents, and ask a question.

Answers show the source passages they came from, so you can check each reply.

Common Use Cases

• Internal knowledge assistant. Upload handbooks, policies and wikis so staff get answers with citations.
• Customer support chatbot. Train a workspace on help articles and add the embeddable chat widget to your site.
• Private ChatGPT for a team. Give everyone one chat app with shared model keys and per-workspace access.
• Research and document review. Ask questions across contracts, reports or papers instead of reading them end to end.
• AI agents. Let agents search the web, summarise pages, query SQL databases, save files and call tools over MCP.
• Local or offline models. Pair it with an Ollama or LM Studio server to keep prompts off third-party clouds.
• App backend. Use the developer API to run RAG queries against your workspaces from your own product.

Configuration

Model keys live in the UI. Provider, model, embedder and vector database are chosen in the setup wizard and changed later under Settings. AnythingLLM saves them to the volume, so no Railway variables are needed for them.

Changing providers. Switching the LLM is instant. Switching the embedder means existing documents must be embedded again, because vectors from different models don't mix.

Chat or query mode. Each workspace can run in chat mode, which mixes the model's general knowledge with your documents, or query mode, which answers only from retrieved sources and says so when nothing matches. Query mode suits support bots that must not invent answers.

Embed widget. Create a widget for a workspace in Settings, restrict it to your domains and set chat limits, then paste its script tag into your site.

External vector databases. For large collections you can move from LanceDB to Qdrant, Pinecone, Weaviate, Chroma, Milvus or another supported store in Settings.

AnythingLLM vs. Alternatives

Dependencies for AnythingLLM Hosting

• Railway account. Hobby is fine for one team; Pro for more RAM when you embed large libraries.
• An API key for at least one LLM provider, or your own Ollama or LM Studio server.
• A public domain on the service, generated after deploy.

Deployment Dependencies

• AnythingLLM on GitHub: source code and releases
• AnythingLLM Docs: providers, agents and API
• Docker Hub image: image tags
• Railway Volumes: how persistent storage works

Implementation Details

The railway tag is published by Mintplex Labs alongside each release; the current build matches version 1.19.0. Redeploying pulls the newest build of that tag, and AnythingLLM migrates its database on start. Back up the volume first if you depend on the data.

The embedded collector parses documents inside the same container, so very large uploads use more memory for a short time. Give the service more RAM if big PDFs fail to process. Server and collector logs both appear in the Railway service logs, which helps when a document fails to embed.

Frequently Asked Questions

What is AnythingLLM? An MIT-licensed AI app that turns your documents into a private chat assistant, with workspaces, agents, multi-user access and an API, running on any LLM you choose.

How much does it cost on Railway? About $5–10/month for the service and volume at small-team scale, plus your model provider's token charges. AnythingLLM Cloud starts at $50/month.

Do I need a GPU? No. Use a cloud model such as OpenAI, Anthropic or Gemini and the app runs on CPU. The default embedder also runs on CPU.

Why can't I open the app after deploying? The template has no public domain. Generate one for port 3001 under Settings and Networking, then open it.

Is my data kept private? Documents, embeddings and chats stay on your Railway volume. Text sent to the model provider in each prompt follows that provider's policy, so use Ollama if nothing may leave your servers.

Can my whole team use it? Yes. Enable multi-user mode in settings, invite users and give each one access to specific workspaces.

Why Deploy and Host AnythingLLM on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying AnythingLLM on Railway, you get a private, document-aware AI workspace for your team with persistent storage and a flat monthly cost.