---
title: "Deploy Embeddings + Reranker | (Just Updated) OpenAI-Compatible Vector Embeddings API, RAG Search, API Key Pre-Set"
description: "Qwen3 text embeddings and bge reranker API, OpenAI-compatible, RAG, CPU"
category: "AI/ML"
url: https://railway.com/deploy/embeddings-reranker-or-just-updated-open
---

# Deploy Embeddings + Reranker | (Just Updated) OpenAI-Compatible Vector Embeddings API, RAG Search, API Key Pre-Set

Qwen3 text embeddings and bge reranker API, OpenAI-compatible, RAG, CPU

**[Deploy Embeddings + Reranker | (Just Updated) OpenAI-Compatible Vector Embeddings API, RAG Search, API Key Pre-Set on Railway](https://railway.com/template/embeddings-reranker-or-just-updated-open)**

Machine-readable deploy manifest (JSON, validated by TemplateCI): https://railway.com/deploy/embeddings-reranker-or-just-updated-open/manifest.json

- **Creator:** SuperSlowSloth
- **Category:** AI/ML

## Template content

### embeddings

- **Image:** ghcr.io/bon5co/embeddings-railway:1.0.0@sha256:5f19d12be63ae17a61714ff73800abd7742cdadd5683026e9df2c9c2a7743733
- **Health check:** /health
- **Public domain:** Yes

## Documentation

# Deploy and Host Embeddings + Reranker on Railway

Qwen3-Embedding-0.6B turns text into 1024-dimension vectors for semantic search and RAG, and BGE-Reranker-v2-m3 re-orders the passages a search returns by how well they answer the question. This template runs both on plain CPU behind one authenticated URL: `/v1/embeddings` is OpenAI-compatible and `/rerank` takes a query plus candidate texts. The weights are inside the image, so the first request is answered from disk.

## About Hosting Embeddings + Reranker

The image wraps Hugging Face Text Embeddings Inference 1.9.4 (Rust, candle backend, no GPU). Two engine processes run in one container: one serves Qwen3-Embedding-0.6B, one serves bge-reranker-v2-m3, and nginx on the Railway `PORT` routes `/v1/embeddings`, `/embed` and `/tokenize` to the first and `/rerank` to the second. Each engine checks the bearer `API_KEY` itself, and the container refuses to start without a key of at least 16 characters. The entrypoint starts the engines one after the other so loading two models does not stack their memory, and sizes the thread pool from the container's CPU quota instead of the host's core count.

**Measured on Railway, 8 vCPU, fp32:** a single short query embeds in **0.23 s** (median of 10). Batches of 16 to 32 passages embed at about **215 to 250 tokens per second** (64 passages of 100 words: 28.4 s; 32 passages of 400 words: 55.3 s). Reranking one query against 50 passages of 120 words took **19.0 s** and against 64 passages **23.9 s**, so rerank the top 10 to 20 results of a search rather than hundreds. The service was healthy **29 s** after the container started. Memory is **5.5 GB** in steady use and **8.5 GB** at its boot peak (Railway `MEMORY_USAGE_GB`, one-minute averages). Throughput is CPU-bound: it fits query-time embedding and reranking and small indexing jobs, not bulk indexing of millions of documents.

### Agent contract

- Public URL: `https://RAILWAY_PUBLIC_DOMAIN` (the generated domain of the `embeddings` service)
- Private URL: `http://embeddings.railway.internal:8080`
- Credentials: the `API_KEY` variable of the `embeddings` service
- Auth: send `Authorization: Bearer $API_KEY` on every route except `GET /health`; a call without the key or with a wrong one answers `401`
- Health: `GET /health` returns 200 once both models are loaded (nginx starts only after both engines answer)

```bash
# needs: RAILWAY_PUBLIC_DOMAIN, API_KEY (a variable of the embeddings service); also curl, awk.
R="https://$RAILWAY_PUBLIC_DOMAIN"
n=0; until [ "$(curl -s -o /dev/null -w '%{http_code}' "$R/health")" = 200 ]; do n=$((n+1)); [ "$n" -le 60 ] || break; sleep 5; done
curl -s -o /dev/null -w "health: %{http_code}\n" "$R/health"
curl -s -o /dev/null -w "write without key: %{http_code}\n" -X POST "$R/v1/embeddings" -H "Content-Type: application/json" -d '{"input":"hello"}'
curl -s -o /dev/null -w "write wrong key: %{http_code}\n" -X POST "$R/v1/embeddings" -H "Authorization: Bearer not-the-key" -H "Content-Type: application/json" -d '{"input":"hello"}'
curl -s -o /dev/null -w "rerank without key: %{http_code}\n" -X POST "$R/rerank" -H "Content-Type: application/json" -d '{"query":"q","texts":["a"]}'
curl -s -X POST "$R/v1/embeddings" -H "Authorization: Bearer $API_KEY" -H "Content-Type: application/json" \
  -d '{"model":"Qwen/Qwen3-Embedding-0.6B","input":["Railway deploys containers","Paris is in France"]}' \
  | grep -o '"index":[0-9]*' | sed 's/"index"://; s/^/embedding index: /'
curl -s -X POST "$R/v1/embeddings" -H "Authorization: Bearer $API_KEY" -H "Content-Type: application/json" \
  -d '{"input":"hello"}' | grep -o '"embedding":\[[^]]*\]' | awk -F, '{print "embedding dimensions: " NF}'
curl -s -X POST "$R/rerank" -H "Authorization: Bearer $API_KEY" -H "Content-Type: application/json" \
  -d '{"query":"What is the capital of France?","texts":["Berlin is the capital of Germany.","Paris is the capital of France.","The Nile is a river."]}' \
  | grep -o '"index":[0-9]*' | head -1 | sed 's/"index"://; s/^/best match index: /'
```

A fresh deploy prints `health: 200`, `write without key: 401`, `write wrong key: 401`, `rerank without key: 401`, `embedding index: 0` and `embedding index: 1`, `embedding dimensions: 1024` and `best match index: 1`. With the OpenAI Python SDK:

```python
from openai import OpenAI
client = OpenAI(base_url="https://RAILWAY_PUBLIC_DOMAIN/v1", api_key="API_KEY value")
vectors = client.embeddings.create(model="Qwen/Qwen3-Embedding-0.6B", input=["first passage", "second passage"])
print(len(vectors.data[0].embedding))  # 1024
```

Rerank sends `{"query": "...", "texts": ["...", "..."]}` to `POST /rerank` and returns `[{"index": 1, "score": 0.99}, ...]` sorted best first. `POST /tokenize` counts tokens, and `GET /info` names the served model. Requests are capped at 64 inputs and 8 MB, and inputs longer than 2048 tokens are truncated rather than rejected.

## Why Deploy Embeddings + Reranker on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying Embeddings + Reranker on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.

- **Your text stays on your infrastructure** — documents and queries are embedded by your own container, not sent to a third-party embeddings API.
- **Drop-in for the OpenAI SDK** — change the base URL and key and existing `embeddings.create` code keeps working, with 1024-dimension vectors.
- **Retrieval and reranking in one service** — one URL and one key for both steps of a RAG pipeline, with no GPU bill and no per-token fee.
- **Private by default** — a generated bearer key guards every route but the health check, and the model weights are already inside the image.

## Common Use Cases

- **RAG for an AI agent** — embed documents into a vector database, embed the question, then rerank the top hits before they go into the prompt.
- **Semantic search** — index product, support or documentation text and search it by meaning.
- **Deduplication and clustering** — compare vectors to find near-duplicate tickets, articles or records.

## Dependencies for Embeddings + Reranker Hosting

- A service with **at least 9 GB of RAM and 8 vCPU** (measured 5.5 GB steady and 8.5 GB at boot). Free, Trial and Hobby plan limits (8 GB) are too small; use a Pro workspace.
- No volume: the service keeps no state and both models are in the image. Pair it with a vector database such as Qdrant or pgvector for storage.

### Deployment Dependencies

- [Qwen/Qwen3-Embedding-0.6B](https://huggingface.co/Qwen/Qwen3-Embedding-0.6B) — the embedding model, Apache-2.0
- [BAAI/bge-reranker-v2-m3](https://huggingface.co/BAAI/bge-reranker-v2-m3) — the reranker model, Apache-2.0
- [huggingface/text-embeddings-inference](https://github.com/huggingface/text-embeddings-inference) — the inference server, Apache-2.0, version 1.9.4
- [bon5co/embeddings-railway](https://github.com/bon5co/embeddings-railway) — the Railway wrapper image this template deploys, Apache-2.0
- Weights licence: the Hugging Face model card of `Qwen/Qwen3-Embedding-0.6B` carries `license: apache-2.0` and the card of `BAAI/bge-reranker-v2-m3` carries `license: apache-2.0`; Apache-2.0 allows commercial use and requires keeping the licence notice. Attribution: Qwen3-Embedding by the Qwen team (Alibaba Cloud), BGE reranker by the Beijing Academy of Artificial Intelligence.


## Similar templates

- [Chat Chat](https://railway.com/deploy/-WWW5r) — Chat Chat, your own unified chat and search to AI platform.
- [Weaviate](https://railway.com/deploy/weaviate-1) — Vector database with hybrid search and an API key-protected API
- [Möbius Personal AI OS](https://railway.com/deploy/mobius-v01-personal-ai-os) — Self-hosted AI agent that builds and runs apps for you, with memory.

Open this page in a browser: https://railway.com/deploy/embeddings-reranker-or-just-updated-open
