Deploy Dagster
Dagster data orchestrator: webserver, daemon, code server, Postgres, S3
dagster-daemon
Just deployed
dagster-code
Just deployed
/data
Just deployed
/var/lib/postgresql/data
dagster-webserver
Just deployed
dagster-storage
Bucket
Just deployed
Deploy and Host Dagster on Railway
Dagster is an open-source (Apache-2.0) data orchestrator for building, scheduling and observing data pipelines. You define assets (tables, files, models) in Python, and Dagster runs them, tracks their lineage and materialization history, and runs schedules and sensors. This template deploys Dagster 1.13.30.
About Hosting Dagster
A production Dagster deployment is several cooperating processes. This template runs them as separate Railway services, in the layout Dagster's docs recommend. dagster-webserver serves the UI and GraphQL API on the public domain, behind a basic-auth login, because Dagster OSS has none. dagster-daemon runs schedules, sensors, automation, the run queue and run monitoring. dagster-code is the gRPC code server that loads your definitions and executes runs. A Railway PostgreSQL database stores runs, events and schedules, and a Railway Bucket stores the captured step logs. The services find each other over the private network and start in dependency order. The webserver applies schema migrations on every start.
Common Use Cases
- Scheduling and monitoring ELT jobs, dbt projects and warehouse refreshes
- Building a catalog of data assets with lineage, checks and run history
- Running Python data and ML pipelines on a schedule or from sensors
Dependencies for Dagster Hosting
- PostgreSQL (Railway PostgreSQL) for runs, event logs, schedules and the asset catalog
- S3-compatible object storage (Railway Bucket) for captured step stdout/stderr
Deployment Dependencies
- Dagster: https://github.com/dagster-io/dagster (release 1.13.30)
- Official image: https://hub.docker.com/r/dagster/dagster-k8s (pinned
1.13.30@sha256:97501f44…) - Dagster deployment docs: https://docs.dagster.io/deployment/oss
- Template repo: https://github.com/aalfath/dagster-railway-template
Implementation Details
After deploy: open the dagster-webserver domain and log in as admin with the password in the webserver's DAGSTER_PASSWORD variable (generated at deploy). An example code location railway is loaded, with two assets, an asset check, a job and an hourly schedule that starts stopped. Click Materialize all to run it.
Services:
dagster-webserver: UI and GraphQL API, Caddy basic auth, health check/healthzdagster-daemon: schedules, sensors, run queue (DAGSTER_MAX_CONCURRENT_RUNS, default 4), run monitoringdagster-code: gRPC code server on port 4000 (private), runs execute here; volume/dataholds asset valuesPostgres: Dagster storage, on a volumedagster-storage: bucket for step logs, viewable in the UI
Your own code: fork the template repo, put your package under code/project/ and extra packages in code/requirements.txt, point the dagster-code service's source at your fork, and set DAGSTER_MODULE (for example my_project.definitions).
Updates: every service builds from the template repo, so Railway shows an "update available" notice when we release a new Dagster version. Read the repo's CHANGELOG.md before you apply it.
Why Deploy Dagster on Railway?
Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.
By deploying Dagster on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.
Template Content
dagster-daemon
aalfath/dagster-railway-templatedagster-code
aalfath/dagster-railway-templatedagster-webserver
aalfath/dagster-railway-templatedagster-storage
Bucket