Deploy LocalAI
OpenAI-compatible API to run open models on CPU, with a web UI
LocalAI
Just deployed
/models
Deploy and Host LocalAI on Railway
LocalAI is a self-hosted, OpenAI-compatible API for running open-source LLMs, image, audio and embedding models on your own hardware.
About Hosting LocalAI
This template runs the official CPU image localai/localai:latest with a persistent volume at /models, so downloaded models survive redeploys. Railway has no GPU, so use small quantized models. On first visit open the web UI, go to the model gallery and install a model, then call it at /v1/chat/completions like any OpenAI endpoint.
Common Use Cases
- Point OpenAI-compatible apps at your own endpoint
- Run small quantized models and embeddings without a third-party API
- Try models from the built-in gallery
Dependencies for LocalAI Hosting
- One service with a volume; no external database
Deployment Dependencies
Why Deploy LocalAI on Railway?
Railway gives you a public HTTPS URL, automatic restarts and no server to maintain. Models are large: pick small ones, as CPU inference on Railway is slow.
Template Content
LocalAI
localai/localai:latest
