Deploy Firecrawl · Quick Start

Deploy and Host Firecrawl on Railway

Firecrawl is an open-source API that turns websites into clean markdown and structured data for LLM applications. It scrapes single pages, crawls whole sites, maps a site's URLs and searches the web. With an LLM provider connected, it also extracts structured JSON. This template runs the official self-hosted stack from the Firecrawl repository and puts an API key in front of it, so your instance is not open to the internet.

About Hosting Firecrawl

• The upstream self-hosted stack. It runs the same services as Firecrawl's docker-compose.yaml: api, worker, playwright-service (headless Chromium), nuq-postgres (the Postgres job queue), redis and rabbitmq. A small Caddy gateway sits in front of them.
• Official images on moving tags. ghcr.io/firecrawl/firecrawl:latest, ghcr.io/firecrawl/playwright-service:latest, ghcr.io/firecrawl/nuq-postgres:latest, redis:alpine, rabbitmq:3-management and caddy:latest. A redeploy picks up new releases.
• An API key is required. Self-hosted Firecrawl has no authentication of its own unless you run Supabase (USE_DB_AUTHENTICATION=false upstream). The gateway is the only public service. It answers any request without Authorization: Bearer {FIRECRAWL_API_KEY} with 401. The key is generated when you deploy.
• The queue admin UI is locked. Bull Board is reachable only at /admin/{BULL_AUTH_KEY}/queues, and the 32-character key is generated at deploy time. Every other /admin path returns 404, even when the request carries the API key.
• No default passwords. The Postgres, Redis and RabbitMQ passwords are generated. None of the data services has a public domain or a TCP proxy. All services talk over Railway's private network.
• Durable. Volumes hold Postgres (queue and crawl state), Redis and RabbitMQ data. Crawl results, keys and the admin UI survive a redeploy of every service.
• A complete worker. The worker service runs the same processes as upstream's all-in-one runner: the queue worker, the extract worker, five NuQ workers, the prefetch worker and the reconciler. If any one of them exits, the service restarts.
• Healthchecks. The gateway is checked at /healthz, the API at /v0/health/liveness, the worker at /health and Playwright at /health.
• LLM keys are optional. Scrape, crawl, map, batch scrape and search work without any key. Extract, the JSON format and summaries need an OpenAI-compatible key or an Ollama server.

Getting started

1. Click Deploy. Leave the LLM fields empty if you only need scraping and crawling. The first boot takes a few minutes while the images are pulled.
2. Open the gateway service, then Variables. Copy FIRECRAWL_API_URL (your public URL) and FIRECRAWL_API_KEY (your key).
3. Check that the stack is up. This request needs no key:

   curl https://YOUR_DOMAIN/health
   # {"status":"ok"}
   

4. Scrape a page:

   curl -X POST https://YOUR_DOMAIN/v2/scrape \
     -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{"url":"https://example.com","formats":["markdown"]}'
   

5. Crawl a site. Crawls run on the worker. Start one, then poll it with the id it returns:

   curl -X POST https://YOUR_DOMAIN/v2/crawl \
     -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{"url":"https://example.com","limit":10}'
   
   curl https://YOUR_DOMAIN/v2/crawl/CRAWL_ID -H "Authorization: Bearer $FIRECRAWL_API_KEY"
   

6. Use the official SDKs. Point them at your URL and key:

   # pip install firecrawl-py
   from firecrawl import Firecrawl
   
   app = Firecrawl(api_key="fc-...", api_url="https://YOUR_DOMAIN")
   doc = app.scrape("https://example.com", formats=["markdown"])
   print(doc.markdown)
   

   // npm install @mendable/firecrawl-js
   import Firecrawl from "@mendable/firecrawl-js";
   
   const app = new Firecrawl({ apiKey: "fc-...", apiUrl: "https://YOUR_DOMAIN" });
   const doc = await app.scrape("https://example.com", { formats: ["markdown"] });
   console.log(doc.markdown);
   

7. Optional: open the queue admin UI at https://YOUR_DOMAIN/admin/{BULL_AUTH_KEY}/queues. The BULL_AUTH_KEY is on the api service.
8. Optional: to turn on extract, set OPENAI_API_KEY on the api service, or set OLLAMA_BASE_URL and MODEL_NAME. The worker reads both values from the api service. Then redeploy api and worker.

Common Use Cases

• Turning web pages and documentation sites into markdown for retrieval-augmented generation (RAG) pipelines and knowledge bases.
• Giving AI agents a scraping and search backend. The official SDKs and the Firecrawl MCP server accept a self-hosted API URL and key.
• Crawling a site on a schedule and storing the pages, without per-page credits. You pay only for the Railway resources you use.
• Getting a site's URL list from its sitemap with /v2/map before you crawl it selectively.
• Extracting structured JSON from pages with your own OpenAI key or your own Ollama models.

Dependencies for Firecrawl Hosting

• Firecrawl API and worker (ghcr.io/firecrawl/firecrawl)
• Playwright rendering service (ghcr.io/firecrawl/playwright-service)
• NuQ Postgres, Postgres 17 with pg_cron (ghcr.io/firecrawl/nuq-postgres)
• Redis and RabbitMQ
• Caddy, as the authenticating gateway

Deployment Dependencies

• Firecrawl source: https://github.com/firecrawl/firecrawl
• Self-hosting guide: https://github.com/firecrawl/firecrawl/blob/main/SELF_HOST.md
• API reference: https://docs.firecrawl.dev/api-reference/introduction
• Container images: https://github.com/orgs/firecrawl/packages
• Firecrawl MCP server: https://github.com/firecrawl/firecrawl-mcp-server
• Caddy: https://caddyserver.com/docs/caddyfile
• Bull Board (the queue UI): https://github.com/felixmosh/bull-board
• Railway private networking: https://docs.railway.com/reference/private-networking

Implementation Details

How clients authenticate. Every request to the public URL must carry the header Authorization: Bearer {FIRECRAWL_API_KEY}. This is the same header the Firecrawl SDKs send when you pass api_key / apiKey. The key must match exactly. A missing or wrong key gets 401 with a JSON error. Three things need no key: GET /health (the API's liveness check, for uptime monitors), GET /healthz (the gateway's own check) and CORS preflight OPTIONS requests. There is one shared key. This template does not provide per-user keys, usage tracking or credits.

Admin routes. The queue UI is at /admin/{BULL_AUTH_KEY}/queues. The /admin/ops/* routes need Authorization: Bearer {BULL_AUTH_KEY}. All other /admin paths return 404.

Upgrade notes

• The images track :latest. To upgrade, redeploy api, worker and playwright-service. Upstream sometimes changes the process layout. Before you upgrade an instance you depend on, check SELF_HOST.md and docker-compose.yaml in the release.
• The worker's start command runs the same processes as upstream's runner (queue-worker, extract-worker, nuq-worker × NUQ_WORKER_COUNT, nuq-prefetch-worker, nuq-reconciler-worker). If a release adds a process, add it to the start command.
• nuq-postgres:latest runs Postgres 17 today. If upstream moves it to a new major version, the existing data directory will not start. Before redeploying nuq-postgres, pin the tag, or dump and restore the data. The queue data is mostly short-lived job state.
• Postgres starts with max_wal_size=1GB and min_wal_size=80MB. These override the 16GB/4GB values that upstream's schema script sets, which could fill a small volume.
• To rotate the API key, change FIRECRAWL_API_KEY on the gateway and redeploy only the gateway. To rotate the admin key, change BULL_AUTH_KEY on the api and redeploy api and gateway. Keep both keys to letters, digits and dashes, because the gateway config uses them as literal values.
• Search uses Firecrawl's built-in DuckDuckGo fallback. To use your own SearXNG instance, set SEARXNG_ENDPOINT on the api.

Why Deploy Firecrawl on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying Firecrawl on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.