Deploy and Host ChromaDB on Railway

ChromaDB is the AI-native open-source embedding database that makes it easy to build AI applications with embeddings. ChromaDB is designed to be easy, fast and effecient and is optimized for managing high-dimensional vector data with built-in distance functions and metadata filtering.

About Hosting ChromaDB

Hosting ChromaDB provides you with a powerful embedding database which is capable of handling vector-data, similarity searches and metadata filtering at scale. ChromaDB excels at semantic search, similarity matching and retrieval-agumented generation (RAG) workflows.

ChromaDB deployments on Railway benefit from scalable CPU, RAM and storage while supporting enterprise-grade network security through built-in token authentication and Railway's secure private network features. Railway provides automated deployment and comprehensive logging to support your AI application operations.

Common Use Cases

Retrieval-Augmented Generation (RAG): Store tokenized knowledge, search-results and other data that AI applications can use for better contextual understanding and personalized responses.
Semantic Search Applications: Powers search engines and applications that seek to find patterns that match not just similarity but meaning, leading to powerfull and relevant results.
Recommendation Systems: Build strong recommendation engines for e-commerce, content platforms, and social media that suggest items and media based on user preferences.
AI Agent Memory: Serve long-term persistent memory to AI agents, which helps agents remember past interactions, learned preferences and contextual information.

Quick Start Guide

Here's a super quick rundown of how to use this template.

Assuming we have this template deployed and we have another NodeJS service next to it that we want to use to connect to ChromaDB. You can easily connect by doing the following:

Setting Up Service Variables

In your client application service, create the following service variables:

For Public Access (via Auth Proxy):

CHROMA_TOKEN="${{"Auth Proxy".API_KEY}}"
CHROMA_URL="https://${{"Auth Proxy".RAILWAY_PUBLIC_DOMAIN}}"

For Private Network (Recommended):

CHROMA_URL="${{Chroma.CHROMA_PRIVATE_URL}}:${{Chroma.CHROMA_HOST_PORT}}"

> Note: Replace service names if changed during deployment. Private network doesn't require authentication.

Creating a Client Connection

In your application, add the following code to connect to ChromaDB:

import { ChromaClient } from "chromadb";

const chromaUrl = new URL(process.env.CHROMA_URL!);

const client = new ChromaClient({
	ssl: chromaUrl.protocol === "https:",
	host: chromaUrl.hostname,
	port: parseInt(chromaUrl.port || (chromaUrl.protocol === "https:" ? "443" : "80")),
	headers: {
		Authorization: `Bearer ${process.env.CHROMA_TOKEN}`,
	},
});

// Example: Create a collection and add embeddings
const collection = await client.createCollection({
	name: "my_collection",
	embeddingFunction: undefined, // Provide embeddings manually
});

await collection.add({
	ids: ["id1", "id2"],
	embeddings: [
		[1.2, 2.3, 4.5],
		[6.7, 8.9, 1.2],
	],
	metadatas: [{ source: "doc1" }, { source: "doc2" }],
	documents: ["This is document 1", "This is document 2"],
});

> Note: Ensure you have the chromadb package installed.

Private Network Connection

For production deployments within the same Railway project, it is recommended to use Railway's private network for enhanced security and performance:

import { ChromaClient } from "chromadb";

const client = new ChromaClient({
	ssl: false,
	host: "chroma.railway.internal", // Or your service's private domain
	port: 8000,
	// No authentication needed on private network
});

// Use the same API as shown above

> Benefits: Private network connections are faster (no proxy), more secure (isolated network), and don't require authentication.

Dependencies for ChromaDB Hosting

chromadb: The core ChromaDB Python package
uvicorn: ASGI server for running the ChromaDB HTTP server
Auth Proxy: Custom Caddy implementation to act as an authentication layer

Deployment Dependencies

The official ChromaDB image: https://hub.docker.com/r/chromadb/chroma
The GitHub repo for ChromaDB: https://github.com/chroma-core/chroma

Implementation Details

Authentication Configuration

This ChromaDB template implements a Caddy proxy to act as Authorization. The token can be found in the Auth Proxy's API_KEY variable and is editable. To connect to ChromaDB through the authorization proxy you can attach Bearer {API_KEY} as an Authorization header.

Alternatively, and preferably, you can connect to the ChromaDB service directly through the private network. It does not require authentication (the private network is secure by default).

Network Configuration

ChromaDB does not dualbind to both IPv6 and IPv4, meaning it can either work on the public network or private network but not both. By default it binds to IPv6 for the private network but the host can be changed to 0.0.0.0 to bind to the public network instead (make sure to generate a public domain if you do this).

However, it is not recommended to bind ChromaDB to the public network directly because it does not support authentication natively. Instead you can connect through the Auth Proxy which is exposed publicly and privately.

Why Deploy ChromaDB on Railway?

Railway is a singular platform to deploy your AI infrastructure stack. Railway will host your vector database so you don't have to deal with configuration complexities, while allowing you to scale vertically based on your embedding storage and query needs.

By deploying ChromaDB on Railway, you are one step closer to supporting a complete AI application stack with minimal operational burden. Host your embedding database, AI platforms, application servers, and more on Railway to build powerful semantic search and RAG applications.