Deploy and Host Crawl4AI on Railway

Crawl4AI is an open-source, LLM-friendly web crawling and scraping framework. It extracts clean markdown, structured data, and screenshots from any webpage via a REST API. Built with async browser automation, it supports LLM-powered content extraction, streaming results, and includes a built-in monitoring dashboard, interactive playground, and MCP integration.

About Hosting Crawl4AI

Deploying Crawl4AI requires running a containerized service that bundles a FastAPI server, headless Chromium browser, and Redis — all managed by supervisord. The official Docker image handles this out of the box. The service exposes a REST API on port 11235 for crawling, markdown extraction, screenshots, PDFs, and LLM-powered content extraction. To use LLM extraction features, you'll need at least one LLM provider API key (OpenAI, Anthropic, Groq, etc.), though basic crawling and markdown extraction work without any API keys.

Common Use Cases

RAG pipelines — Crawl websites and extract clean, LLM-ready markdown for retrieval-augmented generation systems
Structured data extraction — Use LLM-powered extraction to pull structured JSON from unstructured web pages (product listings, job boards, articles)
Web monitoring and research — Automate periodic crawling of web pages with screenshot capture, PDF generation, and content change detection
MCP server for AI agents — Connect Crawl4AI as an MCP tool to Claude Code or other AI agents for live web access via the /mcp/sse endpoint

Dependencies for Crawl4AI Hosting

Docker image: unclecode/crawl4ai:latest (official, multi-arch AMD64/ARM64)
LLM API key (optional): At least one provider key (OpenAI, Anthropic, Groq, DeepSeek, Gemini, Together, or Mistral) for LLM-powered extraction features

Deployment Dependencies

Implementation Details

After deployment, the following endpoints are available:

GET  /health          # Health check
GET  /playground      # Interactive web UI to test crawling
GET  /dashboard       # Real-time monitoring dashboard
POST /crawl           # Synchronous crawl
POST /crawl/stream    # Streaming crawl results
POST /crawl/job       # Async job submission
POST /html            # HTML extraction
POST /screenshot      # Screenshot capture
POST /pdf             # PDF generation
GET  /mcp/sse         # MCP server (SSE transport)

To connect as an MCP server to Claude Code:

claude mcp add --transport sse crawl4ai https://your-railway-domain.up.railway.app/mcp/sse

Why Deploy Crawl4AI on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying Crawl4AI on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.