Deploy and Host FreeLLMAPI on Railway

FreeLLMAPI is an open-source LLM gateway that collapses the free tiers of dozens of providers — Google, Groq, Cerebras, Mistral, OpenRouter, GitHub Models, Cloudflare, Cohere — behind one OpenAI-compatible /v1 endpoint. A router picks an available model per request, fails over when a provider rate-limits, and tracks per-key usage so you stay inside every cap. This template runs it as a private gateway with keys on an encrypted volume and auth enforced before the domain goes live.

What This Template Deploys

Service	Purpose
`freellmapi`	Node gateway serving `/v1` chat, completions, embeddings and Anthropic-compatible `/v1/messages`. Public domain, health-checked.
Volume	Encrypted keys, per-key usage counters, cached model catalog.

One service, one volume. Everything stateful — provider keys and the usage ledger that keeps you under each cap — lives on the volume, which is why a redeploy without one means re-entering every key.

About Hosting

This is a localhost tool given a public URL. Almost every risk below follows from that one change.

A public gateway is an open proxy to your provider keys. Upstream assumes a local client and a local server. On Railway it gets an internet-reachable domain, and an unauthenticated instance lets anyone who finds the URL spend your quota — or reach the dashboard where keys are managed. Set the gateway key and lock the dashboard before the domain resolves. For clients that cannot send headers, use revocable URL tokens rather than disabling auth.

Free tiers come with terms, and stacking them is your liability. Upstream scopes this project to personal experimentation. Free tiers commonly restrict commercial use, resale and quota aggregation, and none of that changes because a gateway sits in front. Run it against keys you registered, for work you do yourself.

The encryption secret matters as much as the volume. Keys are stored encrypted, so a stable secret is required to read them back. Regenerate it on a later deploy and every stored key becomes undecryptable — the volume intact and useless.

The model catalog goes stale, and stale routing fails. Free-tier offerings shift weekly: models launch, get retired, quotas change without notice. The router stays current by pulling a signed catalog feed, which upstream sells as a paid subscription. Without it you route on whatever it last knew, and retired models surface as provider errors rather than an obvious stale-catalog message.

Failover hides errors, not latency. Free tiers are the lowest-priority queue at every provider. The router routes around a rate-limited provider but not a slow one, and tail latency on free inference is bad. Well suited to coding agents, batch jobs and experimentation; wrong behind a user-facing chat box.

Pin the fork and the tag. Several forks exist with different provider sets and very different capacity claims. Deploying latest means the gateway changes under you between redeploys. Pin a digest you tested.

Typical cost: ~$5–10/month for one small Node service and a volume at $10/GB/month RAM, $20/vCPU/month CPU and $0.15/GB/month volumes. Inference itself is the free tiers you already signed up for.

How It Compares

	FreeLLMAPI	LiteLLM	OpenRouter	Provider SDKs direct
Target	Free tiers, stacked	Any provider	Hosted marketplace	One provider each
Hosting	Self-hosted	Self-hosted	Vendor	None needed
Cost	Infrastructure only	Infrastructure only	Per token plus margin	Provider pricing

The honest edge: if you spend real money on inference and need budgets, spend tracking and per-team virtual keys, LiteLLM is the more complete gateway. OpenRouter is simpler still if you would rather pay than operate anything. FreeLLMAPI fits the narrow, genuinely useful case of stitching a dozen free tiers into enough capacity to run coding agents all day without a bill — and it is the only one tracking per-provider caps for you.

Deploy in Under 5 Minutes

Click Deploy and pick a workspace. The gateway comes up with a volume attached and its encryption secret generated once.
Before anything else, confirm the gateway API key is set and the dashboard requires auth. The domain is public the moment it resolves.
Open the dashboard and add provider keys for the free tiers you hold — one is enough to start; routing improves with each.
Point any OpenAI client at https://your-domain/v1 with your gateway key. Claude Code and the Anthropic SDKs use /v1/messages instead.
Send one request and check the dashboard attributes it to a provider and counts it against that key's budget.

Verify before you rely on it: open your domain in a private window with no credentials. If you reach the dashboard or complete a request, the gateway is open to the internet — fix that before adding another key.

Common Use Cases

Coding agents without a bill — run Claude Code, Codex, Aider or Continue against pooled free tiers, with failover when one caps out mid-task.
Prototyping across models — compare a dozen model families through one base_url without maintaining a dozen SDKs.
Batch and background work — summarisation, classification and enrichment where latency does not matter and cost does.
One endpoint for local tooling — point editors, notebooks and scripts at one gateway instead of scattering keys across machines.

Configuration

Variable	Required	Description
Gateway API key	Generated	Required on every request. Without it, the endpoint is open to anyone with the URL.
Encryption secret	Generated	Encrypts stored provider keys. Set once; rotating it invalidates every saved key.
`PORT`	Pre-set	Application port, matched to the Railway target port.
Provider keys	Added in UI	Your own free-tier keys, entered in the dashboard rather than as variables.
Storage volume	Pre-set	Persistent volume for encrypted keys, usage counters and the cached catalog.

Do not expose the dashboard unauthenticated, even briefly. It manages keys to every provider account you have connected, and a public URL is discoverable the moment it exists.

Never rotate the encryption secret on a populated volume. Stored provider keys are encrypted with it and cannot be recovered — you re-add every one by hand.

Dependencies for FreeLLMAPI Hosting

Railway account — ~$5–10/month for one small service plus a volume.
Bundled services — none. The gateway runs as a single Node process with no external database.
Volume — required. Holds encrypted keys, usage counters and the cached catalog; without it a redeploy wipes every key.
Optional — the upstream paid catalog feed for live model and quota updates, and the MCP endpoint for agent tooling.

Deployment Dependencies

Implementation Details

The gateway runs from the upstream image pinned to a tested digest rather than latest, on a Railway domain with a health check so a failed boot is caught instead of silently serving errors. It is a single Node process with no database: the volume holds the encrypted key store, per-key usage counters, and a cached model catalog. The container is replaceable; the volume is the thing worth protecting.

The wire surface is broader than the name suggests. Alongside POST /v1/chat/completions and GET /v1/models, the router implements the Responses API current Codex builds require, a legacy completions shim for editor ghost-text, and Anthropic's /v1/messages so Claude Code runs against the same pool. Each is a translating layer over one router, so failover and quota accounting behave identically whichever format a client speaks.

Treat this as a private service with a public address. The gateway key, the authenticated dashboard and revocable per-client tokens are what stand between an internet-reachable URL and a dozen provider accounts. For backups, snapshot the volume and keep the encryption secret separately: a restored volume without it is a file full of unreadable keys.

Frequently Asked Questions

Is this safe to expose publicly? Only with the gateway key enforced and the dashboard authenticated. It was designed for localhost, so treat the public domain as the thing you are securing against.

Can I use this commercially? Upstream scopes the project to personal experimentation, and free tiers commonly restrict commercial use and quota aggregation. Read each provider's terms; a gateway does not change them.

What happens when a provider rate-limits me? The router marks it unavailable and fails over to the next with headroom, tracking usage per key so it knows which have budget left.

Why are responses sometimes slow? Free tiers are the lowest-priority queue everywhere. Failover routes around errors, not latency. Use this for agents and batch work, not user-facing chat.

Will my keys survive a redeploy? Yes, with the volume attached and the encryption secret unchanged. Lose either and you re-enter every key.

Why Deploy FreeLLMAPI on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying FreeLLMAPI on Railway you get the gateway wired the way a public deployment needs — auth enforced before the domain resolves, an encryption secret generated once and held stable, keys and usage counters on a volume that survives redeploys, and a pinned image rather than a moving tag.