---
title: "Deploy Text Embeddings Inference + Model Cache"
description: "Hugging Face TEI embeddings API with API key auth and model cache"
category: "AI/ML"
url: https://railway.com/deploy/text-embeddings-inference-model-cache
---

# Deploy Text Embeddings Inference + Model Cache

Hugging Face TEI embeddings API with API key auth and model cache

**[Deploy Text Embeddings Inference + Model Cache on Railway](https://railway.com/template/text-embeddings-inference-model-cache)**

Machine-readable deploy manifest (JSON, validated by TemplateCI): https://railway.com/deploy/text-embeddings-inference-model-cache/manifest.json

- **Creator:** Protemplate
- **Category:** AI/ML
- **Total deploys:** 1

## Template content

### Text Embeddings Inference https://avatars.githubusercontent.com/u/25720743?v=4

- **Image:** ghcr.io/huggingface/text-embeddings-inference:cpu-1.9.4
- **Health check:** /health
- **Public domain:** Yes

## Documentation

# Deploy and Host Text Embeddings Inference on Railway

Text Embeddings Inference (TEI) is Hugging Face's Rust inference server for embedding, reranking and sequence classification models. It turns any of the thousands of models tagged `text-embeddings-inference` on the Hugging Face Hub into a fast HTTP API with an OpenAI compatible `/v1/embeddings` route, token aware dynamic batching, Prometheus metrics and Swagger docs, so you can drop it in wherever you would otherwise pay per token for embeddings.

## About Hosting Text Embeddings Inference

Hosting TEI is a single container with a single volume. This template pins the official CPU image `ghcr.io/huggingface/text-embeddings-inference:cpu-1.9.4`, because the default tags are CUDA builds and Railway has no GPUs. `PORT` is pinned to 3000 so Railway's healthcheck reaches `/health`, `HOSTNAME` is `::` so the public domain and other services in your project both reach the server over IPv6 private networking, and `HUGGINGFACE_HUB_CACHE` points at a volume mounted on `/data` so the model is downloaded once instead of on every redeploy. The healthcheck window is 600 seconds because the port only opens after the weights are downloaded and loaded. A generated `API_KEY` protects every inference route by default, while `/health` and `/metrics` stay public so the platform can probe the service.

## Common Use Cases

- **Private embeddings for RAG**: index and query with your own endpoint instead of a metered provider
- **Drop-in OpenAI embeddings replacement**: point any OpenAI SDK at `/v1` and change one base URL
- **Reranking search results**: set `MODEL_ID` to a cross encoder such as `BAAI/bge-reranker-base` and call `/rerank`
- **Zero shot classification and scoring**: sequence classification models serve `/predict`
- **Embeddings backend for other Railway services**: LiteLLM, LibreChat, Dify, Flowise and n8n all accept an OpenAI compatible embeddings URL at `http://text-embeddings-inference.railway.internal:3000/v1`

## Dependencies for Text Embeddings Inference Hosting

- **None at runtime**: no database, no queue, no GPU, no external API keys
- **Hugging Face Hub** on first boot only, to download the model into the volume
- **Memory**: about 1 to 2 GB with the default model (Hobby plan or higher)
- **Storage**: a volume on `/data`, sized for your model (135 MB for the default)

### Deployment Dependencies

- [Text Embeddings Inference repository](https://github.com/huggingface/text-embeddings-inference)
- [Official documentation](https://huggingface.co/docs/text-embeddings-inference)
- [Supported models](https://huggingface.co/docs/text-embeddings-inference/supported_models)
- [BAAI/bge-small-en-v1.5 model card](https://huggingface.co/BAAI/bge-small-en-v1.5)

### Implementation Details

The image ENTRYPOINT is the `text-embeddings-router` binary and every setting has an environment variable, so the template needs no custom start command. Key variables:

```env
MODEL_ID=BAAI/bge-small-en-v1.5
PORT=3000
HOSTNAME=::
API_KEY=
HUGGINGFACE_HUB_CACHE=/data
MAX_BATCH_TOKENS=16384
MAX_CLIENT_BATCH_SIZE=32
```

**First steps after deploy**

1. Wait for `/health` to return 200. First boot downloads about 135 MB before the port opens.
2. Copy `API_KEY` from the service variables.
3. Call the API:

```bash
curl -X POST https:///v1/embeddings \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"input": "Deploying on Railway is easy.", "model": "tei"}'
```

4. Check `GET /info` with the same header to see the model, dtype and maximum input length.
5. To change models, set `MODEL_ID` to another Hub id and redeploy. The new weights are cached on the same volume.

Keep the `cpu-` prefix when you upgrade the image tag. `latest` and the GPU tags cannot start on Railway. If the container runs out of memory, lower `MAX_BATCH_TOKENS` before moving to a bigger plan.

## Why Deploy Text Embeddings Inference on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying Text Embeddings Inference on Railway, you get an authenticated, health checked, OpenAI compatible embeddings endpoint with managed SSL, a persistent model cache and private networking to the rest of your project, with no per token bill and no GPU to rent.


## Similar templates

- [Chat Chat](https://railway.com/deploy/-WWW5r) — Chat Chat, your own unified chat and search to AI platform.
- [stella](https://railway.com/deploy/stella) — Self-host stella with web, API, Postgres, Redis, and object storage.
- [Hermes Agent | OpenClaw Alternative with Dashboard](https://railway.com/deploy/hermes-agent-or-openclaw-alternative-wit) — Self-Hosted Hermes AI Agent for Telegram, Discord & Slack

Open this page in a browser: https://railway.com/deploy/text-embeddings-inference-model-cache