Deploy and Host Firecrawl with Railway

Firecrawl is the open-source web scraping and crawling API behind firecrawl.dev. Give it a URL and it returns clean markdown, HTML, links, page metadata, or LLM-extracted JSON, with a job queue for whole-site crawls. It is the data-ingestion layer for RAG pipelines, agents, and search indexes.

About Hosting Firecrawl

Self-hosting Firecrawl means running its API and background workers next to four supporting services: a Playwright (headless Chromium) renderer, Redis, RabbitMQ, and a Postgres database with pg_cron that holds the NuQ job queue. This template deploys the same five services as the official docker-compose.yaml: the pinned ghcr.io/firecrawl/firecrawl:2.11.322 image runs the API and all workers through the upstream harness, and the official playwright-service and nuq-postgres images are used as-is. Every connection string is wired over Railway's private IPv6 network with explicit ports, passwords are generated at deploy time, the API binds dual-stack so healthchecks pass, and Redis and Postgres get volumes.

Common Use Cases

LLM-ready markdown for RAG and fine-tuning datasets from documentation sites, blogs, and knowledge bases
Site crawls that follow links and return every page as structured data via /v1/crawl
Agent tooling: give an AI agent a scrape/crawl/map tool without paying per page
Structured extraction with /v1/extract when you add an OpenAI-compatible key
Search-backed retrieval with /v1/search when paired with a SearXNG instance on the same private network

Dependencies for Firecrawl Hosting

Playwright service (included): renders JavaScript-heavy pages
Redis (included): caching, locks, rate limiting
RabbitMQ (included): job notifications and webhook queues
NuQ Postgres (included): Postgres 17 + pg_cron with the queue schema, persisted on a volume
OpenAI-compatible model (optional): only for /v1/extract, the json format, and other LLM features

Deployment Dependencies

Implementation Details

First steps after the deploy turns green:

Open the Firecrawl service and wait for /v0/health/readiness to return 200 (first boot pulls a large image and starts about ten worker processes).
Run a scrape against your Railway domain:

curl -X POST https:///v1/scrape \
  -H 'Content-Type: application/json' \
  -d '{"url": "https://example.com", "formats": ["markdown"]}'

Expected shape: {"success": true, "data": {"markdown": "# Example Domain ...", "metadata": {"statusCode": 200}}}. /v2/scrape, /v1/crawl, and /v1/map are available too. SDKs require a non-empty key; any string works because the template runs with USE_DB_AUTHENTICATION=false.

Key variables (all pre-wired; change only if you know why):

PORT=3002
HOST=::
USE_DB_AUTHENTICATION=false
REDIS_URL=${{Redis.REDIS_URL}}
PLAYWRIGHT_MICROSERVICE_URL=http://${{Playwright.RAILWAY_PRIVATE_DOMAIN}}:${{Playwright.PORT}}/scrape
POSTGRES_HOST=${{Postgres.RAILWAY_PRIVATE_DOMAIN}}
NUQ_RABBITMQ_URL=${{RabbitMQ.RABBITMQ_PRIVATE_URL}}
NUQ_WORKER_COUNT=5
BULL_AUTH_KEY=${{secret(32)}}
OPENAI_API_KEY=            # optional, enables /v1/extract and json format

Self-hosted Firecrawl has no built-in authentication: anyone with the public URL can use it. Either remove the public domain and call it from your other Railway services at http://${{Firecrawl.RAILWAY_PRIVATE_DOMAIN}}:${{Firecrawl.PORT}}, or place an authenticating proxy in front. Screenshots, page actions, and agent/browser features need Firecrawl's Fire-engine and are not part of the open-source stack. Lower NUQ_WORKER_COUNT and Playwright's MAX_CONCURRENT_PAGES on small plans.

Why Deploy Firecrawl on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying Firecrawl on Railway, you get the complete official stack (API, workers, Playwright, Redis, RabbitMQ, Postgres) with generated secrets, healthchecks, persistent volumes, managed SSL, and private networking to the rest of your project, ready to feed your agents and RAG pipelines.