Deploy Embeddings + Reranker | (Just Updated) OpenAI-Compatible Vector Embeddings API, RAG Search, API Key Pre-Set

Qwen3 text embeddings and bge reranker API, OpenAI-compatible, RAG, CPU

Deploy Embeddings + Reranker | (Just Updated) OpenAI-Compatible Vector Embeddings API, RAG Search, API Key Pre-Set

Just deployed

Deploy and Host Embeddings + Reranker on Railway

Qwen3-Embedding-0.6B turns text into 1024-dimension vectors for semantic search and RAG, and BGE-Reranker-v2-m3 re-orders the passages a search returns by how well they answer the question. This template runs both on plain CPU behind one authenticated URL: /v1/embeddings is OpenAI-compatible and /rerank takes a query plus candidate texts. The weights are inside the image, so the first request is answered from disk.

About Hosting Embeddings + Reranker

The image wraps Hugging Face Text Embeddings Inference 1.9.4 (Rust, candle backend, no GPU). Two engine processes run in one container: one serves Qwen3-Embedding-0.6B, one serves bge-reranker-v2-m3, and nginx on the Railway PORT routes /v1/embeddings, /embed and /tokenize to the first and /rerank to the second. Each engine checks the bearer API_KEY itself, and the container refuses to start without a key of at least 16 characters. The entrypoint starts the engines one after the other so loading two models does not stack their memory, and sizes the thread pool from the container's CPU quota instead of the host's core count.

Measured on Railway, 8 vCPU, fp32: a single short query embeds in 0.23 s (median of 10). Batches of 16 to 32 passages embed at about 215 to 250 tokens per second (64 passages of 100 words: 28.4 s; 32 passages of 400 words: 55.3 s). Reranking one query against 50 passages of 120 words took 19.0 s and against 64 passages 23.9 s, so rerank the top 10 to 20 results of a search rather than hundreds. The service was healthy 29 s after the container started. Memory is 5.5 GB in steady use and 8.5 GB at its boot peak (Railway MEMORY_USAGE_GB, one-minute averages). Throughput is CPU-bound: it fits query-time embedding and reranking and small indexing jobs, not bulk indexing of millions of documents.

Agent contract

  • Public URL: https://RAILWAY_PUBLIC_DOMAIN (the generated domain of the embeddings service)
  • Private URL: http://embeddings.railway.internal:8080
  • Credentials: the API_KEY variable of the embeddings service
  • Auth: send Authorization: Bearer $API_KEY on every route except GET /health; a call without the key or with a wrong one answers 401
  • Health: GET /health returns 200 once both models are loaded (nginx starts only after both engines answer)
# needs: RAILWAY_PUBLIC_DOMAIN, API_KEY (a variable of the embeddings service); also curl, awk.
R="https://$RAILWAY_PUBLIC_DOMAIN"
n=0; until [ "$(curl -s -o /dev/null -w '%{http_code}' "$R/health")" = 200 ]; do n=$((n+1)); [ "$n" -le 60 ] || break; sleep 5; done
curl -s -o /dev/null -w "health: %{http_code}\n" "$R/health"
curl -s -o /dev/null -w "write without key: %{http_code}\n" -X POST "$R/v1/embeddings" -H "Content-Type: application/json" -d '{"input":"hello"}'
curl -s -o /dev/null -w "write wrong key: %{http_code}\n" -X POST "$R/v1/embeddings" -H "Authorization: Bearer not-the-key" -H "Content-Type: application/json" -d '{"input":"hello"}'
curl -s -o /dev/null -w "rerank without key: %{http_code}\n" -X POST "$R/rerank" -H "Content-Type: application/json" -d '{"query":"q","texts":["a"]}'
curl -s -X POST "$R/v1/embeddings" -H "Authorization: Bearer $API_KEY" -H "Content-Type: application/json" \
  -d '{"model":"Qwen/Qwen3-Embedding-0.6B","input":["Railway deploys containers","Paris is in France"]}' \
  | grep -o '"index":[0-9]*' | sed 's/"index"://; s/^/embedding index: /'
curl -s -X POST "$R/v1/embeddings" -H "Authorization: Bearer $API_KEY" -H "Content-Type: application/json" \
  -d '{"input":"hello"}' | grep -o '"embedding":\[[^]]*\]' | awk -F, '{print "embedding dimensions: " NF}'
curl -s -X POST "$R/rerank" -H "Authorization: Bearer $API_KEY" -H "Content-Type: application/json" \
  -d '{"query":"What is the capital of France?","texts":["Berlin is the capital of Germany.","Paris is the capital of France.","The Nile is a river."]}' \
  | grep -o '"index":[0-9]*' | head -1 | sed 's/"index"://; s/^/best match index: /'

A fresh deploy prints health: 200, write without key: 401, write wrong key: 401, rerank without key: 401, embedding index: 0 and embedding index: 1, embedding dimensions: 1024 and best match index: 1. With the OpenAI Python SDK:

from openai import OpenAI
client = OpenAI(base_url="https://RAILWAY_PUBLIC_DOMAIN/v1", api_key="API_KEY value")
vectors = client.embeddings.create(model="Qwen/Qwen3-Embedding-0.6B", input=["first passage", "second passage"])
print(len(vectors.data[0].embedding))  # 1024

Rerank sends {"query": "...", "texts": ["...", "..."]} to POST /rerank and returns [{"index": 1, "score": 0.99}, ...] sorted best first. POST /tokenize counts tokens, and GET /info names the served model. Requests are capped at 64 inputs and 8 MB, and inputs longer than 2048 tokens are truncated rather than rejected.

Why Deploy Embeddings + Reranker on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying Embeddings + Reranker on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.

  • Your text stays on your infrastructure — documents and queries are embedded by your own container, not sent to a third-party embeddings API.
  • Drop-in for the OpenAI SDK — change the base URL and key and existing embeddings.create code keeps working, with 1024-dimension vectors.
  • Retrieval and reranking in one service — one URL and one key for both steps of a RAG pipeline, with no GPU bill and no per-token fee.
  • Private by default — a generated bearer key guards every route but the health check, and the model weights are already inside the image.

Common Use Cases

  • RAG for an AI agent — embed documents into a vector database, embed the question, then rerank the top hits before they go into the prompt.
  • Semantic search — index product, support or documentation text and search it by meaning.
  • Deduplication and clustering — compare vectors to find near-duplicate tickets, articles or records.

Dependencies for Embeddings + Reranker Hosting

  • A service with at least 9 GB of RAM and 8 vCPU (measured 5.5 GB steady and 8.5 GB at boot). Free, Trial and Hobby plan limits (8 GB) are too small; use a Pro workspace.
  • No volume: the service keeps no state and both models are in the image. Pair it with a vector database such as Qdrant or pgvector for storage.

Deployment Dependencies

  • Qwen/Qwen3-Embedding-0.6B — the embedding model, Apache-2.0
  • BAAI/bge-reranker-v2-m3 — the reranker model, Apache-2.0
  • huggingface/text-embeddings-inference — the inference server, Apache-2.0, version 1.9.4
  • bon5co/embeddings-railway — the Railway wrapper image this template deploys, Apache-2.0
  • Weights licence: the Hugging Face model card of Qwen/Qwen3-Embedding-0.6B carries license: apache-2.0 and the card of BAAI/bge-reranker-v2-m3 carries license: apache-2.0; Apache-2.0 allows commercial use and requires keeping the licence notice. Attribution: Qwen3-Embedding by the Qwen team (Alibaba Cloud), BGE reranker by the Beijing Academy of Artificial Intelligence.

Template Content

More templates in this category

View Template
Chat Chat
Chat Chat, your own unified chat and search to AI platform.

okisdev
117
View Template
Weaviate
Vector database with hybrid search and an API key-protected API

GitHub Locker
1
View Template
Möbius Personal AI OS
Self-hosted AI agent that builds and runs apps for you, with memory.

Tracklayer
1