Deploy Text Embeddings Inference
Hugging Face Text Embeddings Inference 1.9 on CPU with an API key.
Just deployed
/data
Deploy and Host Text Embeddings Inference on Railway
Text Embeddings Inference (TEI) is Hugging Face's high-performance server for text embedding and reranker models. It serves models such as BGE, GTE, E5, Nomic and sentence-transformers through a simple REST API and an OpenAI-compatible embeddings endpoint, with dynamic batching and fast tokenization, which makes it a drop-in backend for RAG pipelines.
About Hosting Text Embeddings Inference
This template deploys TEI v1.9.4 using the CPU image, with BAAI/bge-small-en-v1.5 as the default model. The model is downloaded from Hugging Face on first start into a Railway volume, so restarts reuse the cache. Every request needs the generated API key as a bearer token. The service listens on IPv4 and IPv6, so other Railway services can use the private URL. The default model produces 384-dimension vectors and uses roughly 500 MB of memory. Change MODEL_ID for another embedding or reranker model, and size memory accordingly. Add replicas to scale throughput horizontally.
Common Use Cases
- Embeddings for RAG, semantic search and clustering without an external API
- An OpenAI-compatible
/v1/embeddingsendpoint for LangChain, LlamaIndex and others - Reranking search results with a cross-encoder model
Dependencies for Text Embeddings Inference Hosting
ghcr.io/huggingface/text-embeddings-inference:cpu-1.9.4(official image)- A Railway volume at
/datafor the model cache
Deployment Dependencies
Implementation Details
| Service | Image | Networking | Storage |
|---|---|---|---|
| tei | text-embeddings-inference:cpu-1.9.4 | public domain on 80; private IPv4/IPv6 | volume at /data |
curl "$TEI_URL/v1/embeddings" -H "Authorization: Bearer $API_KEY" -H "Content-Type: application/json" \
-d '{"input": ["hello world"], "model": "bge"}'
OpenAI SDKs work with base_url = "$TEI_URL/v1" and the API key.
| Variable | Default | Purpose |
|---|---|---|
MODEL_ID | BAAI/bge-small-en-v1.5 | Hugging Face model to serve |
API_KEY | generated | Required bearer token |
HOSTNAME | :: | Bind address; do not remove, Docker otherwise sets it to the container hostname |
HF_TOKEN | unset | Add for gated or private models |
Notes:
- The first start downloads the model; the health check allows up to 10 minutes.
- Larger models (for example
BAAI/bge-m3) need several GB of RAM.
This is a community-maintained deployment package and does not imply affiliation with or endorsement by the Text Embeddings Inference project or its maintainers.
Why Deploy Text Embeddings Inference on Railway?
Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.
By deploying Text Embeddings Inference on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.
Template Content
