Deploy ChromaDB

Deploy and Host ChromaDB on Railway

ChromaDB is an open-source embeddings database — the storage layer retrieval-augmented generation, semantic search and agent memory are built on. You hand it documents with their vectors, and it returns the nearest matches to a query vector in milliseconds, with metadata filters applied in the same call. Teams use it for product docs behind a support bot, or the memory an agent reads before it answers.

This template runs ChromaDB as a client-server deployment rather than an embedded library. The chroma service runs the official chromadb/chroma image with a 5 GB volume at /data, holding its SQLite metadata store and HNSW index files, and has no public domain. In front sits gateway, a Caddy proxy from gridalpha/chromadb-railway, which owns the public URL and checks credentials before anything reaches the database. That split matters: open-source Chroma ships no authentication, and its own guide says the basic stack "doesn't support any kind of authentication" and to put it behind an authenticating proxy. Self-host ChromaDB this way and it is reachable only through a credential you control.

Getting Started with ChromaDB on Railway

Set CHROMA_PASSWORD when you deploy — it is the one value the template needs, and the gateway refuses to start without it, so the database is never published without a lock on the door. A CHROMA_TOKEN is generated for you and is what your application code should send. Once the deploy is green, open the public URL with /docs on the end and sign in as chroma: you get a Swagger explorer for the v2 API, generated by the running server, where you can create a collection, add records and query it without writing code. /api/v2/healthcheck confirms the storage engine has opened the volume.

From your own code, point the Chroma client at the gateway host on port 443 with the bearer token as a header. Create a collection, add documents, then query — if the ids come back ranked by distance, the deployment works end to end. Everything lands on the volume, so redeploys keep your collections.

About Hosting ChromaDB

Chroma is an Apache-2.0 embeddings database from chroma-core/chroma, rewritten in Rust for the 1.x line. A single-node server keeps collection metadata in SQLite and each collection's vectors in an HNSW index, held in memory while in use and persisted to disk. Self-host it when the embeddings come from data you would rather not send to a managed service, when you want a fixed monthly cost instead of per-gigabyte charges, or when the database should sit on the app's private network.

• Collections with per-record documents, metadata and optional URIs
• Vector search with metadata filters and full-text conditions in one query
• Cosine, squared L2 and inner-product spaces, with tunable HNSW parameters
• Python, TypeScript and Rust clients, plus LangChain and LlamaIndex integrations
• A generated OpenAPI schema and Swagger UI served by the database itself

chroma is the database and stays private; its volume is the only durable state. gateway is a stock Caddy build whose entrypoint hashes your password at boot and installs two routes — an exact bearer-token match for API clients, checked first so SDKs never meet an authentication challenge, and basic auth for everything else, which lets a browser use the API explorer.

Why Deploy ChromaDB on Railway

Railway removes the parts of running a vector database that have nothing to do with vectors.

• Persistent volume attached and mounted at the data path already
• Private networking between gateway and database, no port exposed
• A public HTTPS URL with a managed certificate
• Health checks on both services, with automatic restarts
• Vertical scaling by changing a memory limit, the knob that matters

Common Use Cases for Self-Hosted ChromaDB

• Retrieval-augmented generation — chunk your docs, embed them, and retrieve the passages a language model should read before answering.
• Agent memory — a durable store of past interactions that survives restarts, queried by similarity rather than by key.
• Semantic search in a product — search a knowledge base, ticket archive or code corpus by meaning, filtered by tenant or date.
• Deduplication — find near-duplicate records by nearest neighbour.

Dependencies for ChromaDB on Railway

• chroma — chromadb/chroma:1.5.9, the official image. Listens on 8000, stores everything under /data on a 5 GB volume, private to the project.
• gateway — gridalpha/chromadb-railway on caddy:2-alpine. Holds the public domain; the only service the internet reaches.

No external database, cache or object store is needed.

Environment Variables Reference

Any key in the server's single_node_full.yaml reference config is settable as a CHROMA_-prefixed variable, double underscore for nesting: CHROMA_OPEN_TELEMETRY__ENDPOINT maps to open_telemetry.endpoint.

Deployment Dependencies

• Source: chroma-core/chroma (Apache-2.0)
• Image: chromadb/chroma
• Docs: docs.trychroma.com
• Gateway: gridalpha/chromadb-railway

Hardware Requirements for Self-Hosting ChromaDB

HNSW holds the whole index in memory, so RAM caps collection size. Chroma's benchmarks give a rule for 1024-dimensional embeddings: roughly 245,000 vectors per gigabyte, plus 1 GB overhead. Latency stays near 5 ms past a million records — memory, not CPU, runs out first.

Below 2 GB is not recommended upstream. Raise memory on chroma first.

Self-Hosting ChromaDB with Docker

The published image needs no build step. This runs a server with its data on a host directory:

docker run -d --name chroma -p 8000:8000 -v ./chroma-data:/data chromadb/chroma:1.5.9

The following is the Python client talking to a deployment of this template:

import chromadb

client = chromadb.HttpClient(
    host="your-app.up.railway.app",
    port=443,
    ssl=True,
    headers={"Authorization": "Bearer YOUR_CHROMA_TOKEN"},
)

c = client.get_or_create_collection("handbook")
c.add(ids=["kb-001"], documents=["Reset a forgotten password from settings."])

print(c.query(query_texts=["how do I reset my password"], n_results=3))

The server does not embed text for you. The Python client embeds locally when you pass documents; other clients send vectors you produce.

How Much Does ChromaDB Cost to Self-Host?

Chroma is free and open source under Apache-2.0, with no paid tier or feature gate. Chroma Cloud, the managed service from the same team, bills by usage — roughly $2.50 per GiB written and $0.33 per GiB-month stored, plus query and egress. Self-hosting replaces that with plain infrastructure cost: one small container, one Caddy container and a volume. For a steady corpus that is usually cheaper, and it is predictable.

FAQ

What is ChromaDB? An open-source embeddings database that stores documents alongside their vectors and retrieves them by semantic similarity — the retrieval layer under RAG, semantic search and agent memory.

What does this Railway template deploy? Two services: the official chromadb/chroma server on a persistent volume, kept private, and a Caddy gateway that owns the public URL and authenticates every request.

Why does this template include a separate gateway service? Open-source Chroma has no authentication, so publishing it directly would expose every collection to anyone with the URL. The gateway adds bearer-token and basic-auth credentials in front, as upstream's documentation recommends.

How do I connect to self-hosted ChromaDB from Python? Use chromadb.HttpClient with host set to your Railway domain, port=443, ssl=True, and an Authorization: Bearer header. The rest of the API matches the embedded client.

Does self-hosted ChromaDB generate embeddings for me? No. The server stores and searches vectors, it does not run an embedding model. The Python client embeds text locally; other clients expect vectors you supply.

Why does ChromaDB need a volume on Railway? Collections live in SQLite and HNSW index files under /data, and containers are recreated on every deploy. The template mounts a volume at that path already.