Deploy and Host Chroma

Chroma is an open-source, AI-native vector database built for storing, querying, and retrieving high-dimensional embeddings — the backbone of RAG pipelines, semantic search, and LLM memory systems. It is the default vector store in LangChain tutorials, natively integrated with LlamaIndex and Haystack, and downloaded over 11 million times a month.

Self-host Chroma on Railway with one click: the template deploys the official chromadb/chroma Docker image with a persistent volume pre-wired, token authentication enabled, and a public HTTPS endpoint ready to connect from any client.

How to Connect to Your Chroma Instance

Once your Railway deploy is live, grab your public domain from the Railway dashboard (Settings → Networking → Public Domain) and your bearer token from the CHROMA_SERVER_AUTHN_CREDENTIALS service variable. Use those to connect from Python or JavaScript:

import chromadb

# Connect to your Railway-hosted Chroma instance
client = chromadb.HttpClient(
    host="your-service.up.railway.app",
    port=443,
    ssl=True,
    headers={"Authorization": "Bearer YOUR_TOKEN"}
)

client.heartbeat()  # Should return a timestamp — you're connected

# Create a collection and add documents
collection = client.get_or_create_collection("my_docs")
collection.add(
    documents=["Railway makes infrastructure easy", "Chroma stores your embeddings"],
    ids=["doc1", "doc2"]
)

# Query by natural language
results = collection.query(query_texts=["cloud deployment"], n_results=2)

If your application runs in the same Railway project, connect via private network instead — no token required and no public traffic:

# Private network (same Railway project only) — faster and no auth needed
client = chromadb.HttpClient(host="chroma.railway.internal", port=8000, ssl=False)

To verify the service is healthy, hit the heartbeat endpoint: GET https://your-service.up.railway.app/api/v2/heartbeat.

About Hosting Chroma

Chroma gives AI applications a purpose-built retrieval layer. Unlike general-purpose databases with vector add-ons, Chroma is designed from the ground up around embedding operations — store, query, filter, and update vectors with a four-function API that works identically in development and production.

Key features:

• Hybrid search — combines dense vector similarity with BM25 full-text ranking and regex filtering in a single query
• Metadata filtering — filter collections by any structured attribute alongside semantic similarity
• Multi-modal — store embeddings from text, images, audio, or any model output
• Three official SDKs — Python, JavaScript/TypeScript; community clients for Rust, Java, PHP, Dart
• First-class LangChain / LlamaIndex integration — drop-in vector store for any RAG framework
• Apache 2.0 licensed — fully open source, no usage restrictions

On Railway, Chroma runs as a single Docker service. Embeddings are persisted to a mounted volume at /data, surviving restarts and redeployments without data loss.

Why Deploy Chroma on Railway

Railway removes every operational burden of self-hosting:

• Persistent volume mounted at /data — no manual storage configuration
• Auto-generated bearer token via ${{secret(32)}} — secure by default on first deploy
• HTTPS public domain provisioned automatically — no reverse proxy setup needed
• Private networking available for same-project services — zero-latency internal access
• Vertical scaling from the Railway dashboard — upgrade RAM as your embedding index grows
• One-click redeployments — pull a new chromadb/chroma tag without downtime

Common Use Cases

• RAG applications — index your documents, query semantically, and feed results as LLM context; the most common Chroma use case by far
• Semantic search — replace keyword search in internal tools, knowledge bases, or customer-facing search with meaning-aware retrieval
• LLM memory — persist agent conversation history or user-specific context as embeddings for long-term recall
• Recommendation engines — find similar products, articles, or content by embedding proximity rather than category tags

Dependencies for Chroma

• chromadb/chroma:latest — official Docker Hub image; pin to a specific tag (e.g. 1.5.3) for production
• Railway Volume — mounted at /data for embedding persistence
• No external database required — Chroma's SQLite-backed storage is self-contained

Environment Variables Reference

Deployment Dependencies

• Runtime: Docker (no additional runtime required)
• GitHub: chroma-core/chroma
• Docker Hub: chromadb/chroma
• Official docs: docs.trychroma.com

Chroma vs Pinecone vs Qdrant

Chroma's edge is zero-friction developer experience and deep framework integration. Pinecone wins when you need a fully managed SLA with no ops. Qdrant is the better pick when metadata filtering performance at scale is critical.

Minimum Hardware Requirements for Chroma

Chroma's HNSW index loads entirely into RAM — RAM is the key constraint, not CPU or storage.

Important: Once your collection approaches the available RAM ceiling, the OS begins swapping and query performance collapses rapidly. Plan Railway vertical scaling before you hit the limit.

Self-Hosting Chroma

Run Chroma on any machine with Docker:

docker run -d \
  --name chromadb \
  -p 8000:8000 \
  -v ./chroma-data:/data \
  -e CHROMA_SERVER_AUTHN_PROVIDER=chromadb.auth.token_authn.TokenAuthenticationServerProvider \
  -e CHROMA_SERVER_AUTHN_CREDENTIALS=your-secret-token \
  chromadb/chroma:latest

Or with Docker Compose:

services:
  chromadb:
    image: chromadb/chroma:latest
    ports:
      - "8000:8000"
    volumes:
      - chroma-data:/data
    environment:
      - CHROMA_SERVER_AUTHN_PROVIDER=chromadb.auth.token_authn.TokenAuthenticationServerProvider
      - CHROMA_SERVER_AUTHN_CREDENTIALS=your-secret-token

volumes:
  chroma-data:

Connect using chromadb.HttpClient(host="localhost", port=8000, headers={"Authorization": "Bearer your-secret-token"}).

Is Chroma Free?

Chroma is fully open source under the Apache 2.0 license — free to self-host with no usage restrictions. On Railway, you pay only for infrastructure: a typical single-service Chroma deployment runs $5–$10/month depending on volume size and traffic. Chroma also offers Chroma Cloud, a managed serverless tier that starts with $5 in free credits and scales on usage — useful if you want to eliminate ops entirely.

FAQ

What is Chroma? Chroma is an open-source, AI-native vector database for storing and querying embeddings. It's the default vector store in LangChain and natively integrates with LlamaIndex, Haystack, OpenAI, and Hugging Face. Developers use it to build RAG applications, semantic search, and LLM memory systems.

What does this Railway template deploy? A single service running the official chromadb/chroma Docker image with a persistent Railway volume mounted at /data, token-based authentication pre-configured, and a public HTTPS endpoint. Your embeddings survive restarts and redeployments.

How do I connect from another service in the same Railway project? Set CHROMA_HOST=:: on the Chroma service to bind to IPv6 (private network). Then connect using chromadb.HttpClient(host="chroma.railway.internal", port=8000, ssl=False) — no auth token needed on the private network.

Can I use Chroma in production with this template? Yes for small-to-medium workloads (up to a few million vectors, depending on dimensionality). Chroma's HNSW index lives in RAM, so scale your Railway instance vertically as your collection grows. For billion-scale or strict multi-tenancy requirements, consider migrating to Qdrant or Milvus.

Does Chroma work with LangChain and LlamaIndex? Yes — Chroma is the de facto default vector store for both frameworks. Install langchain-chroma or llama-index-vector-stores-chroma and point host at your Railway public domain with your bearer token.

Why does Chroma need a volume on Railway? Without a volume, all embeddings are stored in the container filesystem and wiped on every redeploy. Mounting a Railway volume at /data ensures your collections and indexed embeddings persist across deployments and restarts.