Deploy and Host Prefect with a Worker on Railway

Prefect is a Python orchestration framework for building data pipelines. Two decorators — @flow and @task — turn an ordinary script into a production workflow with scheduling, retries, caching, logging and a UI that shows you every run.

This template deploys the whole loop: a self-hosted Prefect server, its PostgreSQL database, and a worker that is already running and already attached to a work pool. Deployments you declare start executing on schedule immediately — there is no separate machine to keep awake and no prefect worker start running on your laptop.

Versions. Prefect is pinned to the 3.8 line (prefecthq/prefect:3.8-python3.12, currently 3.8.2) on both the server and the worker, and PostgreSQL to 18. Pinning means a deploy today and a deploy next month give you the same Prefect minor — the same API, the same UI, the same database schema — while patch releases still arrive automatically.

About Hosting Prefect with a Worker

Self-hosting Prefect properly means running three things, not one:

The server — the REST API and the web UI. It stores deployments, schedules, flow runs and logs, and decides when work is due.
PostgreSQL — where all of that state lives. Prefect defaults to SQLite, which is fine on a laptop and unsuitable for a deployed server.
A worker — the process that actually executes your flows. It polls a work pool, claims scheduled runs, and runs them.

The third is the one people miss. A Prefect server with no worker is a control plane over an empty queue: you can schedule a flow for 3am, and at 3am nothing happens. The run simply sits in a Late state. Most "I self-hosted Prefect and nothing runs" confusion is exactly this.

This template provisions all three and wires them together. The database is attached over Railway's private network rather than the public internet. The server gets a domain and is password-protected with a unique random password generated at deploy time. The worker waits for the server to finish its migrations, ensures the work pool exists, registers whatever you declared, and starts polling.

Why Deploy Prefect on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying Prefect on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.

Specifically for Prefect: all three pieces are long-running processes, one with a persistent volume — exactly the shape Railway handles well and a laptop does not. They sit in one project, on one bill, with per-service logs and metrics, and the worker rebuilds straight from your fork of the repository whenever you push.

Common Use Cases

Scheduled ETL / ELT — pull from an API or database on a cron, transform, load, with retries and alerting when a step fails.
Data quality and reporting jobs — nightly checks, dashboard source refreshes, warehouse maintenance.
ML pipelines — recurring feature builds, batch scoring, retraining runs that need to be observable and re-runnable.
Event-driven automation — webhook or event triggers that fan out into dependent tasks with real failure handling.
Replacing cron and shell scripts — the same jobs, but with run history, parameters, retries, logs and a UI.

Dependencies for Prefect Hosting

PostgreSQL — provisioned by this template.
A work pool and worker — provisioned by this template.
Your flow code — kept in your fork of the worker repository.

Deployment Dependencies

Prefect documentation — https://docs.prefect.io/v3/get-started/index
Writing flows and tasks — https://docs.prefect.io/v3/develop
Deployment configuration (prefect.yaml) — https://docs.prefect.io/v3/deploy
Security settings — https://docs.prefect.io/v3/advanced/security-settings
Worker image source — https://github.com/yuting1214/Prefect-PostgreSQL

Implementation Details

Replacing the example with your own flows — entirely in a browser.

A deployed template attaches directly to the original template repository, which you cannot push to. Railway's Eject hands you your own copy. After that, everything is a file edit on github.com — no terminal, no local Python, no git client.

In Railway, open the worker service and go to Settings.
Under Source, find Upstream Repo and click Eject.
Choose the GitHub organization or account for your copy.
Click Eject service. Railway mirrors the repository into your account and repoints the service at it.
Open your new repository on github.com and click flows/example_stars.py.
Click the pencil icon, replace the code with your flow, and commit.
Click flows/prefect.yaml and edit the deployments: entry so entrypoint points at your flow, with the schedule you want.

Railway sees the push, rebuilds the worker, and the worker re-registers your deployments on boot. Add more flows by adding more files under flows/ and one deployments: entry each.

The example flow runs from the moment you deploy — you only need to eject when you want to change it.

Adding a dependency. Flow runs execute inside the worker, so anything a flow imports must be listed in pyproject.toml. Add it with uv add pandas, which updates pyproject.toml and uv.lock together, then commit both. The build uses uv sync --frozen and will fail on a stale lockfile rather than quietly install a different version, so the two files must be committed as a pair.

Keeping up with template updates. Once ejected, Railway watches the original template and opens a pull request branch in your copy when it improves, so you can review changes before taking them.

Prefer to keep flows in your own repository instead? Skip ejecting and register deployments straight from your machine — the worker fetches the code at run time:

prefect config set PREFECT_API_URL="https://YOUR-SERVER.up.railway.app/api"
prefect config set PREFECT_API_AUTH_STRING="admin:YOUR-PASSWORD"

from prefect import flow

flow.from_source(
    source="https://github.com/YOUR-USER/YOUR-REPO.git",
    entrypoint="etl.py:daily",
).deploy(name="nightly", work_pool_name="Process", cron="0 3 * * *")

This needs no fork, but your flows can then only import packages already present in the worker image, because you are not rebuilding it.

Signing in. The server is password-protected by default with a unique random password generated at deploy time. Find it in the Railway dashboard: open the prefect-server service, go to Variables, and read PREFECT_SERVER_API_AUTH_STRING. It has the form admin:PASSWORD. The browser prompts for it the first time you open the UI.

To drive the deployed server from your own machine:

prefect config set PREFECT_API_URL="https://YOUR-SERVER.up.railway.app/api"
prefect config set PREFECT_API_AUTH_STRING="admin:YOUR-PASSWORD"

Scaling execution. Flow runs execute as subprocesses inside the worker container, so they share its CPU and memory. For heavier workloads, scale the worker service up in Railway, or run additional workers against the same pool.