---
title: "Deploy Deploy PageSift Web Scraper API"
description: "Firecrawl-compatible web scraper and document parser for AI"
category: "AI/ML"
url: https://railway.com/deploy/deploy-pagesift-web-scraper-api
---

# Deploy Deploy PageSift Web Scraper API

Firecrawl-compatible web scraper and document parser for AI

**[Deploy Deploy PageSift Web Scraper API on Railway](https://railway.com/template/deploy-pagesift-web-scraper-api)**

Machine-readable deploy manifest (JSON, validated by TemplateCI): https://railway.com/deploy/deploy-pagesift-web-scraper-api/manifest.json

- **Creator:** onepush
- **Category:** AI/ML

## Template content

### PageSift

- **Image:** ghcr.io/onepush-dev/pagesift:1.2.1
- **Health check:** /healthz
- **Public domain:** Yes

## Documentation

# Deploy and Host self hosted PageSift (Web Scraper API) on Railway

This template deploys PageSift, an open-source web scraper API, as one Rust service on Railway with a persistent /data volume.

## About Hosting PageSift open-source software on Railway (self hosted PageSift template)

Your PageSift API key, scraped pages and crawl state stay in your own Railway project rather than on a shared hosted service.

## Why Deploy PageSift, the open-source web scraper API on Railway (Railway Free Trial)

Firecrawl's hosted API charges per page and routes your data through its servers.

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying PageSift on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.

### Railway vs Other Hosting Providers and VPS for PageSift self hosting

Generic VPS hosting means you run the server, TLS and backups yourself, while Railway wires up a volume and domain from one template.

| Provider | What You Get with Railway | What You Get with the Other Provider |
|---|---|---|
| DigitalOcean | A one-click template with a public domain, a persistent /data volume and a generated API key. | Full droplet control, but you manage Docker, TLS and upgrades. |
| AWS | A single managed service with usage-based billing and no VPC or IAM setup. | Deep networking and storage control, with more configuration. |
| Hetzner | The same template flow with no server to provision or patch after deploy. | Low-cost bare servers, but you own hardening, proxy and backups. |

## Common Use Cases for hosted PageSift

- **Markdown extraction for LLM context:** Point `/v2/scrape` or `/r/` at a page and get clean Markdown back, with main-content detection that drops navigation and boilerplate.
- **Firecrawl v2 compatible clients:** PageSift implements a documented subset of the Firecrawl v2 HTTP API. Point a Firecrawl SDK at your Railway domain and `PAGESIFT_API_KEYS` value, then test the methods you use.
- **Jina Reader style URLs:** Prefix any URL with `/r/` and call it with a Bearer key, which suits prototypes and agents that already fetch pages this way.
- **Document parsing with optional OCR:** Upload PDFs, Office files, e-books or email to `/v2/parse`, with OCR for scanned pages.
- **Persistent crawl jobs:** Run multi-page crawls with state kept in the `/data` volume, so progress survives a restart.

## Dependencies for PageSift Docker hosted on Railway

This template runs one PageSift service built from the onepush-dev image, with no separate Redis or database to wire up.

### Deployment Dependencies for Managed PageSift Service (one Rust service with bundled Lightpanda and /data volume, no separate Redis or database service)

The service listens on port 8080 behind Railway's public domain and checks health at /healthz. Two variables come from the template:

- **PAGESIFT_API_KEYS:** a generated secret sent as the Bearer token.
- **PAGESIFT_PUBLIC_URL:** built from your Railway domain and used in links such as crawl job status URLs.

PAGESIFT_LLM_API_KEY (for /v2/agent) and PAGESIFT_SEARCH_API_KEY are optional, not generated, and you add them yourself if needed.

### Implementation Details for PageSift (onepush-dev/pagesift:1.2.1 wraps unchanged upstream image with volume ownership and IPv6 listen fixes)

The image wraps the upstream ghcr.io/yolow92/pagesift:1.2.1 build unchanged with a small start script. Upgrades mean bumping the pinned tag.

## How does PageSift compare against other tools

### PageSift vs Firecrawl (Firecrawl Alternative)

- **Compatibility:** PageSift implements a documented subset of the Firecrawl v1 and v2 HTTP API, so existing Firecrawl SDK clients can point at your instance by changing the API URL. Behavior is not guaranteed to be identical in every edge case.
- **Deployment shape:** PageSift runs as one Rust service with a bundled browser and a volume, while a typical Firecrawl self-host involves several cooperating services.
- **Where Firecrawl wins:** Firecrawl has a larger hosted product, more integrations and a bigger community, which matters if you want a managed cloud rather than your own instance.

### PageSift vs Jina Reader (Jina Reader Alternative)

- **URL style:** PageSift accepts the Jina Reader style `/r/` path with a Bearer key, so simple read-a-page calls can move over with little change.
- **Scope:** PageSift also handles crawling, PDF, Office and e-book input and structured extraction on the same instance, which Jina Reader's hosted reader does not focus on.
- **Where Jina Reader wins:** Jina Reader is a hosted service with no infrastructure for you to run, which is simpler if you only need occasional page reads.

### PageSift vs Crawl4AI (Crawl4AI Alternative)

- **Shape:** Crawl4AI is a Python library you embed in your own code, while PageSift is a standalone HTTP service you call from any language.
- **Documents:** PageSift parses PDF, scanned pages with OCR, Office files and archives as well as web pages, so mixed input can go through one endpoint.
- **Where Crawl4AI wins:** if you want a Python-native crawler you control inside your own pipeline, Crawl4AI fits that model more naturally.

### PageSift with Docker vs Railway (Self Hosting Comparison)

- **Docker:** you run the upstream image yourself, manage the volume, TLS proxy and restarts, and keep full control of the host.
- **Railway:** the template sets up the service, persistent volume, public domain and health check, and generates the API key for you.
- **Trade-off:** Railway handles the wiring, while Docker on your own machine gives you more control over networking and placement.

## How to use PageSift on Railway?

Work through these steps after the deploy finishes.

1. Copy the generated PAGESIFT_API_KEYS value from your Railway service variables. Scraping, parsing and crawling endpoints need it; /healthz and /metrics do not.
2. Check that the service is up by calling the health endpoint, which should return ok.
3. Scrape a page with the Jina Reader style URL, sending your key as a Bearer token. The response is clean Markdown.
4. Point a Firecrawl SDK at your deployment by setting the API URL to the bare https://your-domain and the API key to your generated key.
5. /metrics is public and needs no key. The MCP HTTP process binds to loopback only and is not exposed by this template.

```bash
curl -s https://your-domain/healthz
curl -s https://your-domain/r/https://example.com \
  -H "Authorization: Bearer YOUR_PAGESIFT_API_KEY"
```

## How to self host PageSift on other VPS Services (PageSift self hosting guide)

PageSift runs as one container, so any VPS with Docker works. Put your own TLS proxy in front, since the container speaks plain HTTP.

### Clone the Repository

Grab the upstream source to read or build the code.

```bash
git clone --branch v1.2.1 https://github.com/YOLOW92/pagesift.git
cd pagesift
```

### Install Dependencies

On the host you only need Docker. Nothing else is required for HTTP scraping, PDF and Office parsing.

```bash
docker --version
```

### Configure Environment Variables

Generate a real key and keep it out of your shell history. Clients send it as a Bearer token.

```bash
export PAGESIFT_API_KEYS="$(openssl rand -hex 24)"
```

### Start the PageSift Application

Run the upstream image with a named volume for /data, binding the HTTP port to loopback only. Put your own TLS and authentication reverse proxy in front of 127.0.0.1:3002.

```bash
docker run -d -p 127.0.0.1:3002:3002 -v pagesift-data:/data \
  -e PAGESIFT_API_KEYS="$PAGESIFT_API_KEYS" \
  ghcr.io/yolow92/pagesift:1.2.1
curl -s https://your-domain/healthz
```

Railway handles the domain, TLS, volume and restarts for you. OCR is considerably heavier than plain HTML or Office parsing, so size the host for your load.

## Official Pricing of PageSift (PageSift pricing)

PageSift is open-source software licensed under AGPL-3.0, and a commercial license is available separately from the upstream project for uses that AGPL does not cover.

## PageSift hosted vs self hosted comparison (Pricing, features, costs, and more)

PageSift is open source under AGPL-3.0, with a commercial license available separately for appropriate uses.

### Monthly cost of self hosting PageSift on Railway

Railway bills on usage. Crawl jobs and browser-backed pages raise memory use, so check the dashboard after your first real workload.

### System Requirements for Hosting PageSift on a VPS

PageSift runs as one Rust service with a bundled Lightpanda browser and a /data volume. Keep one replica, since crawl state lives on that volume.

## Frequently Asked Questions (FAQs)

### What key do I use to call my PageSift instance?

Send one of the keys you set in PAGESIFT_API_KEYS as a Bearer token in the Authorization header.

### Where is the /v2/scrape endpoint?

It is served on your instance's public domain, for example POST https://your-domain/v2/scrape with a JSON body containing the url and the formats you want.

### Does PageSift bypass every blocker or paywall?

No. Pages behind blockers, logins or paywalls may return incomplete content or warnings, and PageSift does not claim to bypass them.

### Do I need AI API keys for basic scraping?

No. Basic scraping and document parsing work without model keys. PAGESIFT_LLM_API_KEY is only needed for /v2/agent, which sends page content to your configured model provider.

### Is the MCP server publicly reachable?

No. The MCP HTTP process binds to loopback only, so use local stdio MCP or the public HTTP API instead.

### What is stored in /data?

The /data volume holds crawl job state and other local data. Keep one replica, because local state is not shared across replicas.


## Similar templates

- [Chat Chat](https://railway.com/deploy/-WWW5r) — Chat Chat, your own unified chat and search to AI platform.
- [Weaviate](https://railway.com/deploy/weaviate-1) — Vector database with hybrid search and an API key-protected API
- [Möbius v0.1 Personal AI OS](https://railway.com/deploy/mobius-v01-personal-ai-os) — Self-hosted AI agent that builds and runs apps for you, with memory.

Open this page in a browser: https://railway.com/deploy/deploy-pagesift-web-scraper-api