--- title: "Deploy LocalAI | (Just Updated) Self-Hosted OpenAI API With The Model Baked In" description: "OpenAI-compatible LLM API. Model and backend baked in, ready on first boot." category: "AI/ML" url: https://railway.com/deploy/localai-or-just-updated-self-hosted-open --- # Deploy LocalAI | (Just Updated) Self-Hosted OpenAI API With The Model Baked In OpenAI-compatible LLM API. Model and backend baked in, ready on first boot. **[Deploy LocalAI | (Just Updated) Self-Hosted OpenAI API With The Model Baked In on Railway](https://railway.com/template/localai-or-just-updated-self-hosted-open)** - **Creator:** SuperSlowSloth - **Category:** AI/ML ## Template content ### localai - **Image:** ghcr.io/bon5co/localai-railway:v4.9.0 - **Health check:** /readyz - **Public domain:** Yes ## Documentation # Deploy and Host LocalAI on Railway LocalAI is the free, open-source drop-in replacement for the OpenAI API: it serves `/v1/chat/completions`, `/v1/embeddings`, `/v1/audio/*` and `/v1/images/*` from models that run on your own CPU, with no API key sold by anybody and no request leaving your project. This template runs LocalAI v4 as a single service with one volume, an inference backend and a chat model already installed in the image, and an API key enforced from the first request. ## About Hosting LocalAI LocalAI v4 ships a deliberately small base image. The inference backends live in a separate OCI gallery and the models live in a model gallery, so a stock container starts with neither: the API answers `backend not found: llama-cpp`, and the deployer has to install a backend and a model by hand before anything works. That is a reasonable design for a workstation and a poor one for a platform deploy, where the first thing a person does with the URL is send a request to it. Two further things need handling on Railway. A volume is a bind mount, so it hides whatever the image build wrote underneath its mount path — anything baked into the image has to be kept outside the mount and copied across on the first boot. And LocalAI does not read `$PORT`: it takes its listen address from `LOCALAI_ADDRESS`, while Railway's health check dials the port it injected, so the address has to be derived from the injected value before the server starts. This image handles all three, and refuses to start without an API key — LocalAI serves both inference and its web UI to anyone when no key is set, and the URL is public from the moment Railway assigns the domain. ## Why Deploy LocalAI on Railway? Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it. By deploying LocalAI on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway. Concretely, for this app: inference is a long-running, memory-resident process with a disk attached, which is exactly what Railway's always-on services and volumes provide, and the public domain plus TLS mean any OpenAI SDK can point at it by changing one base URL. ## Common Use Cases - Give an app, an agent framework or a coding tool an OpenAI-compatible endpoint whose per-token cost is a fixed monthly container instead of a metered bill. - Keep prompts and documents inside your own infrastructure, where no third-party provider sees them. - Serve embeddings, transcription or image generation from the same endpoint by installing further models from the built-in gallery. ## Dependencies for LocalAI Hosting - A Railway volume, mounted at `/data`. ### Deployment Dependencies - LocalAI documentation: https://localai.io/ - Upstream image: https://quay.io/repository/go-skynet/local-ai - Wrapper image source: https://github.com/bon5co/localai-railway ## What this template does that a stock deploy does not - **It answers on the first request.** The `cpu-llama-cpp` backend and the Apache-2.0 `qwen3-1.7b` chat model are installed at build time, and a warmup call loads the model once the server reports ready — so the first caller gets tokens, not a model download. - **The baked assets survive the volume.** Every LocalAI storage path is relocated under `/data`, and pristine copies kept outside the mount are seeded onto it on the first boot only, so a redeploy neither loses the model nor re-copies it. - **Nobody else can use your endpoint.** `LOCALAI_API_KEY` is generated per deploy and the container refuses to start if it is empty. Anonymous requests return 401. - **`$PORT` is honoured**, by exporting `LOCALAI_ADDRESS` from the injected port, so the deployment answers at its own domain and the health check passes. - **Threads are sized to the container.** llama.cpp otherwise sizes its pool from the host's core count, which a container always over-reports; this image reads the quota from `/sys/fs/cgroup/cpu.max`. - **The image is pinned** to LocalAI v4.9.0 and prebuilt. ## After deploying Point any OpenAI client at the deployment URL and use the generated `LOCALAI_API_KEY` from the service's variables: ``` curl $URL/v1/chat/completions \ -H "Authorization: Bearer $LOCALAI_API_KEY" \ -H 'Content-Type: application/json' \ -d '{"model":"qwen3-1.7b","messages":[{"role":"user","content":"hello"}]}' ``` Install further models from the gallery with `POST /models/apply`, or browse them in the web UI at the same URL. Larger models want more memory: raise the service's plan before pulling one. ## Similar templates - [Chat Chat](https://railway.com/deploy/-WWW5r) — Chat Chat, your own unified chat and search to AI platform. - [stella](https://railway.com/deploy/stella) — Self-host stella with web, API, Postgres, Redis, and object storage. - [Hermes Agent | OpenClaw Alternative with Dashboard](https://railway.com/deploy/hermes-agent-or-openclaw-alternative-wit) — Self-Hosted Hermes AI Agent for Telegram, Discord & Slack Open this page in a browser: https://railway.com/deploy/localai-or-just-updated-self-hosted-open