---
title: "Deploy WebSurfer [Updated Sep '26]"
description: "WebSurfer [Sep '26] (Open-Source LLM Web Fetching API) Self Host"
category: "Other"
url: https://railway.com/deploy/websurfer
---

# Deploy WebSurfer [Updated Sep '26]

WebSurfer [Sep '26] (Open-Source LLM Web Fetching API) Self Host

**[Deploy WebSurfer [Updated Sep '26] on Railway](https://railway.com/template/websurfer)**

- **Creator:** shinyduo's Projects
- **Category:** Other

## Template content

### Redis https://cdn.sanity.io/images/sy1jschh/production/0ce0bfdcfbdbf69662b1116671f97c2dd788b655-157x157.svg

- **Image:** redis:8.2.1
- **Start command:** `/bin/sh -c "rm -rf $RAILWAY_VOLUME_MOUNT_PATH/lost+found/ && exec docker-entrypoint.sh redis-server --requirepass $REDIS_PASSWORD --save 60 1 --dir $RAILWAY_VOLUME_MOUNT_PATH"`

### joeychilson/websurfer:latest

- **Image:** ghcr.io/joeychilson/websurfer:latest
- **Public domain:** Yes

## Documentation

# Deploy and Host WebSurfer with One Click on Railway

WebSurfer is an open-source, high-performance API designed to help Large Language Models (LLMs) surf the web. Built in Go, it fetches and processes web content with token-based content limiting, Redis caching, and API key authentication — making it easy to give your AI agents reliable, controlled access to live web data. It is a self-hosted alternative to proprietary web scraping APIs and LLM browsing tools.

With Railway, deploying WebSurfer becomes a true one-click experience. No server provisioning, no Redis setup, no reverse proxy configuration — just deploy and start fetching.

## About Hosting WebSurfer on Railway (Self Hosting WebSurfer on Railway)

Self hosting WebSurfer traditionally means:

* Provisioning a server with Go installed or Docker configured
* Setting up and managing a Redis instance for caching
* Configuring API keys, ports, and environment variables
* Managing networking between WebSurfer and Redis
* Handling service restarts, updates, and uptime monitoring

WebSurfer on Railway removes this burden completely. Railway automatically provisions both the WebSurfer container and a Redis instance, wires them together with internal networking, manages environment variables, and keeps both services running.

## Why Deploy WebSurfer on Railway

Deploying WebSurfer on Railway is ideal for developers building AI agents, RAG pipelines, or LLM-powered tools that need live web access.

You get:

* Automatic provisioning of WebSurfer + Redis
* Internal networking between services
* Secure environment variable management
* One-click deployment with zero configuration
* Built-in logs, metrics, and restart policies

### Railway vs DigitalOcean

On DigitalOcean, you must create a droplet, install Docker, run a Redis container, configure networking between services, and manage firewall rules. 

### Railway vs Linode

Linode provides raw servers where you are responsible for OS updates, Docker installation, Redis management, and security patches. Railway abstracts these operational tasks and provides a ready-to-use WebSurfer instance with Redis instantly.

### Railway vs Hetzner

Hetzner offers excellent performance and low costs but assumes strong sysadmin skills for Docker, Redis, and multi-service orchestration. Railway prioritizes ease, speed, and reliability, making WebSurfer ideal for rapid AI infrastructure deployment.

## Common Use Cases

1. LLM Web Browsing: Give AI agents the ability to fetch and read live web pages with token-limited responses optimized for LLM context windows.

2. RAG Pipeline Data Ingestion: Fetch web content for Retrieval-Augmented Generation workflows, with caching to avoid redundant requests.

3. AI Research Assistants: Power research tools that pull real-time information from the web for summarization, fact-checking, or analysis.

4. Content Monitoring: Periodically fetch and process web pages to detect changes, track competitors, or monitor news sources.

5. API Gateway for Web Content: Provide a centralized, authenticated API endpoint that multiple AI services can use to access web content safely.

## Dependencies for WebSurfer Hosted on Railway

WebSurfer has a minimal dependency footprint.

Core dependencies include:

* Runtime: Go binary (compiled in Docker)
* Cache: Redis for response caching and performance
* Networking: HTTP server on configurable port
* Configuration: YAML config file for site rules, rate limits, and cache TTLs

### Deployment Dependencies

When deployed via Railway, WebSurfer automatically provisions:

* WebSurfer container from the GitHub repo
* Redis instance with internal networking
* Environment variable binding for `REDIS_URL`
* Port binding on 8080

Useful links:
* [WebSurfer on GitHub](https://github.com/joeychilson/websurfer)

### Implementation Details

Key environment variables:

* `ADDR` — Server address (default `:8080`)
* `REDIS_URL` — Redis connection string (auto-configured by Railway)
* `CONFIG_FILE` — Path to configuration YAML
* `LOG_LEVEL` — Logging verbosity (debug, info, warn, error)

API endpoints:

* `POST /v1/fetch` — Fetch and process a URL. Requires Bearer token or `X-API-Key` header. Accepts `url` and `max_tokens` parameters.
* `GET /health` — Health check endpoint (no authentication required).

Example request:
```bash
curl -X POST https://your-websurfer.railway.app/v1/fetch \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "max_tokens": 4000}'
```

Response includes: status code, page title, token estimate, content, cache state, and pagination metadata.

## How Does WebSurfer Compare to Other Web Fetching Tools

### WebSurfer vs Firecrawl

Firecrawl is a full-featured web scraping and crawling platform with JavaScript rendering, sitemap crawling, and structured extraction. WebSurfer is intentionally simpler — focused specifically on LLM-optimized content fetching with token limiting, making it lighter and faster for AI agent use cases.

### WebSurfer vs Jina Reader API

Jina Reader is a hosted service that converts URLs to LLM-friendly text. WebSurfer gives you the same capability but fully self-hosted, with no API rate limits imposed by a third party and full control over caching and configuration.

### WebSurfer vs Browserless

Browserless provides headless Chrome as a service for full browser rendering. WebSurfer is lighter weight — it fetches and processes content without a full browser, making it faster and cheaper to run for content that does not require JavaScript rendering.

## How to Use WebSurfer

1. Deploy: Click the Railway deploy button to launch WebSurfer and Redis with a single click.

2. Set API Key: Configure your API key environment variable in the Railway dashboard to secure the fetch endpoint.

3. Get URL: Copy the Railway-generated public URL for your WebSurfer instance.

4. Fetch Content: Send POST requests to `/v1/fetch` with a URL and `max_tokens` parameter to retrieve LLM-optimized web content.

## How to Self Host WebSurfer on Other VPS

To self host WebSurfer on a traditional VPS:

1. Provision a server with Docker installed
2. Start a Redis instance:
```bash
docker run -d --name redis -p 6379:6379 redis:7-alpine
```
3. Clone the repository:
```bash
git clone https://github.com/joeychilson/websurfer.git
cd websurfer
```
4. Build and run the container:
```bash
docker build -t websurfer .
docker run -d \
  --name websurfer \
  -p 8080:8080 \
  -e REDIS_URL=redis://redis:6379 \
  -e ADDR=:8080 \
  --link redis \
  websurfer
```
5. Configure firewall rules to allow port 8080
6. Set up SSL/TLS termination with a reverse proxy
7. Configure API key authentication

With Railway and the WebSurfer template, all of this is handled automatically.

## Features of WebSurfer

* High-performance URL fetching optimized for LLM consumption
* Token-based content limiting to fit LLM context windows
* Redis caching for fast repeated requests and reduced bandwidth
* API key authentication via Bearer token or X-API-Key header
* Rate limiting and retry mechanisms for reliable fetching
* Site-specific configuration rules via YAML
* Lightweight Go binary with minimal resource usage

## Official Pricing of WebSurfer

WebSurfer is completely free and open source. There are no licensing fees, usage limits, or premium tiers. You only pay for the infrastructure running it.


### System Requirements

* CPU: 1 vCPU (minimum)
* RAM: 256 MB–512 MB for WebSurfer, 256 MB+ for Redis
* Storage: Minimal (Redis uses memory-based caching)
* Port: 8080 (API server)

## FAQs

**What is WebSurfer?**
WebSurfer is an open-source API built in Go that helps LLMs fetch and process web content. It provides token-limited responses, Redis caching, and API key authentication for AI agent web access.

**Is WebSurfer free and open source?**
Yes. WebSurfer is fully open source under a permissive license with no fees. You only pay for the server or platform where you host it.

**How do I self host WebSurfer?**
You can deploy WebSurfer with one click on Railway using the template, or run it manually via Docker on any VPS alongside a Redis instance.

**Is WebSurfer suitable for production use?**
Yes. WebSurfer includes API key authentication, rate limiting, retry mechanisms, and Redis caching — making it production-ready for AI workflows.

**How much does it cost to host WebSurfer on Railway?**
Typically $5–$10 per month for the WebSurfer container and Redis instance. Railway bills based on actual resource consumption.

**Does WebSurfer render JavaScript?**
No. WebSurfer fetches and processes HTML content without a headless browser. For JavaScript-heavy sites, consider pairing it with a rendering service.

**Can I use WebSurfer with any LLM?**
Yes. WebSurfer is LLM-agnostic — it returns processed web content via a standard REST API that any LLM framework can consume.

**What is the max_tokens parameter?**
It controls how much content WebSurfer returns from a fetched page, measured in estimated LLM tokens. This helps you stay within your model's context window limits.

**Does WebSurfer support authentication for fetched sites?**
WebSurfer supports custom headers and site-specific configuration via `config.yaml`, which can include custom user agents and request parameters.

**What is the best open-source alternative to Jina Reader?**
WebSurfer is a strong self-hosted alternative to Jina Reader, offering the same LLM-optimized web fetching with full control over caching, rate limits, and configuration.

## Ready to Deploy WebSurfer?

Click below to Deploy WebSurfer on Railway and give your AI agents reliable web access in minutes.

## Why Deploy WebSurfer on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.


## Similar templates

- [Rocky Linux](https://railway.com/deploy/rocky-linux) — Hosted Rocky Linux 9 workspace with SSH and persistent storage. 🚀
- [Foundry Virtual Tabletop](https://railway.com/deploy/X5tR6G) — A Self-Hosted & Modern Roleplaying Platform
- [Letta Code Remote](https://railway.com/deploy/letta-code-remote) — Run a Letta Code agent 24/7. No inbound ports, just deploy.

Open this page in a browser: https://railway.com/deploy/websurfer