---
title: "Deploy Infinity Embeddings + Reranker API"
description: "Fast CPU embeddings and reranking API with model cache and API key"
category: "AI/ML"
url: https://railway.com/deploy/infinity-embeddings-reranker-api
---

# Deploy Infinity Embeddings + Reranker API

Fast CPU embeddings and reranking API with model cache and API key

**[Deploy Infinity Embeddings + Reranker API on Railway](https://railway.com/template/infinity-embeddings-reranker-api)**

Machine-readable deploy manifest (JSON, validated by TemplateCI): https://railway.com/deploy/infinity-embeddings-reranker-api/manifest.json

- **Creator:** Protemplate
- **Category:** AI/ML
- **Total deploys:** 2

## Template content

### Infinity https://avatars.githubusercontent.com/u/63565275?v=4

- **Image:** ghcr.io/michaelfeil/infinity_v3:cpu-3.0.1
- **Health check:** /health
- **Public domain:** Yes

## Documentation

# Deploy and Host Infinity Embeddings + Reranker API on Railway

Infinity is a high throughput inference server for embedding, reranking and classification models. Version 3 is written in Rust and serves several Hugging Face models from one process, with an OpenAI compatible `/v1/embeddings` route, a `/rerank` route, token aware dynamic batching, Prometheus metrics and Swagger docs. This template gives you a private embeddings and reranking backend for RAG and search without paying per token.

## About Hosting Infinity Embeddings + Reranker API

Hosting Infinity is a single container with a single volume. This template pins the official CPU image `ghcr.io/michaelfeil/infinity_v3:cpu-3.0.1` (234 MB compressed), because the other tags need a GPU and Railway has none. It loads two small models side by side: `BAAI/bge-small-en-v1.5` for 384 dimension embeddings and `cross-encoder/ms-marco-MiniLM-L6-v2` for reranking. `PORT` is pinned to 7997 so Railway's healthcheck reaches `/health`, `INFINITY_HOSTNAME` is `::` so the public domain and other services in your project both reach the server, and the Hugging Face cache sits on a volume mounted at `/data`, so models download once instead of on every redeploy. A generated `INFINITY_API_KEY` protects every inference route by default, while `/health` and `/metrics` stay public for the platform.

## Common Use Cases

- **Private embeddings for RAG**: index and query with your own endpoint instead of a metered provider
- **Two stage retrieval**: fetch candidates with embeddings, then reorder them with the reranker on the same service
- **Drop in OpenAI embeddings replacement**: point any OpenAI SDK at `/v1` and change one base URL
- **Embeddings backend for other Railway services**: LiteLLM, Dify, Flowise, n8n, LangChain and LlamaIndex can call `http://infinity.railway.internal:7997/v1`

## Dependencies for Infinity Embeddings + Reranker API Hosting

- **None at runtime**: no database, no GPU, no external API keys
- **Hugging Face Hub** on first boot only, to download about 226 MB of model files into the volume
- **Memory**: under 1 GB with the default models
- **Storage**: a volume on `/data`, sized for your models

### Deployment Dependencies

- [Infinity repository](https://github.com/michaelfeil/infinity)
- [Infinity v3 quickstart](https://michaelfeil.eu/infinity/main/v3/quickstart/)
- [Supported models](https://github.com/michaelfeil/infinity/blob/v3.0.1/libs/infinity-v3/docs/supported-models.md)
- [Infinity v3 license](https://github.com/michaelfeil/infinity/blob/v3.0.1/libs/infinity-v3/LICENSING.md)

### Implementation Details

The image ENTRYPOINT is the `infinity_v3` binary and every setting is an `INFINITY_*` environment variable, so the template needs no custom start command. Key variables:

```env
INFINITY_MODEL_ID=BAAI/bge-small-en-v1.5;cross-encoder/ms-marco-MiniLM-L6-v2
PORT=7997
INFINITY_PORT=${{PORT}}
INFINITY_HOSTNAME=::
INFINITY_API_KEY=
INFINITY_HUGGINGFACE_HUB_CACHE=/data
INFINITY_MAX_BATCH_TOKENS=8192
```

**First steps after deploy**

1. Wait for `/health` to return 200. The first boot downloads both models before the port opens.
2. Copy `INFINITY_API_KEY` from the service variables.
3. Embed text. With two models loaded, name the model in every request:

```bash
curl -X POST https:///v1/embeddings \
  -H "Authorization: Bearer $INFINITY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "BAAI/bge-small-en-v1.5", "input": "Deploying on Railway is easy."}'
```

4. Rerank documents:

```bash
curl -X POST https:///rerank \
  -H "Authorization: Bearer $INFINITY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "cross-encoder/ms-marco-MiniLM-L6-v2", "query": "persist model weights", "texts": ["A cake recipe.", "Mount a volume for the model cache."]}'
```

5. Call `GET /v1/models` with the same header to see what is loaded. To change models, edit `INFINITY_MODEL_ID` (semicolon separated) and redeploy; the new weights are cached on the same volume. On the Free plan, serve a single model.

Infinity v3 uses the Infinity Restricted MIT License 1.0, a source available license that allows commercial use and hosting on Railway but not on 25 named providers. Keep the `cpu-` prefix when you upgrade the image tag.

## Why Deploy Infinity Embeddings + Reranker API on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying Infinity Embeddings + Reranker API on Railway, you get an authenticated, health checked embeddings and reranking endpoint with managed SSL, a persistent model cache and private networking to the rest of your project, with no per token bill and no GPU to rent.


## Similar templates

- [Chat Chat](https://railway.com/deploy/-WWW5r) — Chat Chat, your own unified chat and search to AI platform.
- [Weaviate](https://railway.com/deploy/weaviate-1) — Vector database with hybrid search and an API key-protected API
- [Möbius Personal AI OS](https://railway.com/deploy/mobius-v01-personal-ai-os) — Self-hosted AI agent that builds and runs apps for you, with memory.

Open this page in a browser: https://railway.com/deploy/infinity-embeddings-reranker-api
