---
title: "Deploy Apache Airflow"
description: "Schedule and monitor data pipelines written in Python"
category: "Automation"
url: https://railway.com/deploy/airflow
---

# Deploy Apache Airflow

Schedule and monitor data pipelines written in Python

**[Deploy Apache Airflow on Railway](https://railway.com/template/airflow)**

- **Creator:** A3A
- **Category:** Automation

## Template content

### airflow-apiserver https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/apache-airflow.svg

- **Source:** https://github.com/gridalpha/airflow-railway
- **Start command:** `/railway-entrypoint.sh api-server`
- **Health check:** /api/v2/monitor/health
- **Public domain:** Yes

### Redis https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/redis.svg

- **Image:** redis:8.2
- **Start command:** `/bin/sh -c "rm -rf $RAILWAY_VOLUME_MOUNT_PATH/lost+found/ && exec docker-entrypoint.sh redis-server --requirepass $REDIS_PASSWORD --save 60 1 --dir $RAILWAY_VOLUME_MOUNT_PATH"`

### airflow-dag-processor https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/apache-airflow.svg

- **Source:** https://github.com/gridalpha/airflow-railway
- **Start command:** `/railway-entrypoint.sh dag-processor`
- **Health check:** /healthz

### airflow-triggerer https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/apache-airflow.svg

- **Source:** https://github.com/gridalpha/airflow-railway
- **Start command:** `/railway-entrypoint.sh triggerer`
- **Health check:** /healthz

### Postgres https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/postgresql.svg

- **Image:** ghcr.io/railwayapp-templates/postgres-ssl:18

### airflow-scheduler https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/apache-airflow.svg

- **Source:** https://github.com/gridalpha/airflow-railway
- **Start command:** `/railway-entrypoint.sh scheduler`
- **Health check:** /health

### airflow-worker https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/apache-airflow.svg

- **Source:** https://github.com/gridalpha/airflow-railway
- **Start command:** `/railway-entrypoint.sh worker`
- **Health check:** /healthz

## Buckets

- **airflow-logs**

## Documentation

![Apache Airflow logo](https://upload.wikimedia.org/wikipedia/commons/7/71/AirflowLogo.svg)

# Deploy and Host Apache Airflow on Railway

Apache Airflow is the workflow orchestrator most data teams standardise on. You write pipelines as Python — a DAG of tasks with dependencies, retries and schedules — and Airflow runs them in order, on time, across as many machines as you give it. Because DAGs are ordinary Python files they live in version control and can call any library you already use. Self-host Airflow and you get the scheduler, REST API and React UI with no per-task pricing and no data leaving your infrastructure.

Deploy Airflow on Railway and you get the production CeleryExecutor topology, not a single container that quietly drops work. Five services run from one image: an API server serving the UI, REST API and task-execution endpoint; a scheduler; a DAG processor; a triggerer; and a Celery worker. Managed PostgreSQL holds the metadata, managed Redis is the broker, and an object storage bucket keeps task logs so any service can read a log any worker wrote. Only the API server is public.

![Diagram of the five Airflow services with Postgres, Redis and a bucket](https://res.cloudinary.com/rroe4rtk/image/upload/v1788035103/apache-airflow-architecture.png)

## Getting Started with Apache Airflow on Railway

Open the API server's public URL and you land on the Airflow sign-in page. The first administrator is created on first boot from `AIRFLOW_ADMIN_USERNAME` and `AIRFLOW_ADMIN_PASSWORD` on the `airflow-apiserver` service — read that password from the service's Variables tab, sign in, and change it from the user menu. The home page's Health panel should show MetaDatabase, Scheduler, Triggerer and Dag Processor green within a minute or two, confirming every service found the database and the broker. Click **Dags** to find three example pipelines, paused by default. Toggle `railway_sales_etl` on and press play: it builds a synthetic day of orders, rolls them up by region and prints the totals. Open the run, click `transform` and then **Logs** — that log comes from the bucket, proving the worker, broker and remote logging are wired correctly. `railway_parallel_fanout` then shows task mapping across the worker tier.

![Airflow home page with all four components reporting healthy](https://res.cloudinary.com/rroe4rtk/image/upload/v1788035104/apache-airflow-home-health.png)
![Airflow Dags list showing three pipelines and their recent runs](https://res.cloudinary.com/rroe4rtk/image/upload/v1788035106/apache-airflow-dags-list.png)
![Dag overview with the mapped fan-out run duration chart](https://res.cloudinary.com/rroe4rtk/image/upload/v1788035107/apache-airflow-dag-overview.png)
![Task log read back from object storage showing regional totals](https://res.cloudinary.com/rroe4rtk/image/upload/v1788035108/apache-airflow-task-log.png)

## About Hosting Apache Airflow

Airflow separates *what* runs from *where* it runs: DAGs describe dependencies and schedules, the scheduler decides which task instances are ready, and the executor hands them to workers. Running all of that in one process is fine on a laptop and a liability in production, because one crash takes scheduling and execution down together. This template splits the roles as Airflow documents them.

- **Pipelines as Python code** — dynamic DAGs, TaskFlow decorators, task mapping, typed XComs
- **Real scheduling** — cron, data intervals, timetables, catchup, backfills, dependency-aware retries
- **Deferrable tasks** — long waits move to the triggerer, so waiting an hour costs no worker slot
- **Hundreds of providers** — AWS, GCP, Azure, Snowflake, dbt, Kubernetes, HTTP and SQL
- **REST API, CLI and role-based access** — Admin, Op, User, Viewer and Public roles

Each supporting service has a job. **PostgreSQL** stores DAG definitions, run history, task state, connections and variables, and is also the Celery result backend. **Redis** is the broker the scheduler pushes queued tasks onto and workers pull from. The **bucket** holds task logs, which matters because a Railway volume attaches to exactly one service. The **DAG processor** parses Python in its own process so a broken file cannot stall scheduling; the **triggerer** runs the loop deferrable operators wait on.

## Why Deploy Apache Airflow on Railway

Railway removes the infrastructure work self-hosting Airflow usually implies.

- Managed PostgreSQL, Redis and object storage provisioned with the template
- Five services, private networking and TLS with no YAML to write
- Push to the source repository and every service rebuilds together
- Scale the worker on its own when pipelines get heavier
- No Kubernetes cluster, Helm chart or certificates

## Common Use Cases for Self-Hosted Apache Airflow

- **ELT and warehouse loading** — pull from APIs and databases on a schedule, then run dbt models in order
- **Machine learning pipelines** — scheduled retraining, feature generation and batch scoring, with retries
- **Report and dashboard refresh** — recompute aggregates overnight so BI tools open instantly
- **Operational automation** — reconciliation, data-quality checks and backups that need audit history

## Dependencies for Apache Airflow

- `apache/airflow:3.3.1`, extended by a small source repository, [gridalpha/airflow-railway](https://github.com/gridalpha/airflow-railway), which adds the boot-time configuration Railway needs and bakes in the example DAGs
- PostgreSQL 18 (Railway managed) — metadata database and Celery result backend
- Redis 8 (Railway managed) — Celery broker
- Railway object storage bucket — remote task logs

### Environment Variables Reference

| Variable | Purpose |
|---|---|
| `AIRFLOW_SECRET_SEED` | Seed the Fernet key, task-execution JWT secret and API secret key are derived from; identical on all five Airflow services |
| `AIRFLOW_ADMIN_USERNAME` | Administrator created on first boot |
| `AIRFLOW_ADMIN_PASSWORD` | That administrator's password; ignored once it exists |
| `DATABASE_URL` | PostgreSQL connection string |
| `REDIS_URL` | Celery broker URL |
| `AIRFLOW_LOGS_*` | Bucket name, endpoint, region and keys for remote logging |
| `AIRFLOW_DAGS_GIT_REPO_URL` | Optional: adds a versioned git DAG bundle |
| `AIRFLOW__CELERY__WORKER_CONCURRENCY` | Tasks one worker runs at once (default 4) |

Any `AIRFLOW__SECTION__KEY` variable you set overrides what the image would otherwise compute, so the whole Airflow configuration reference stays available.

### Deployment Dependencies

- [Apache Airflow on GitHub](https://github.com/apache/airflow)
- [Official documentation](https://airflow.apache.org/docs/apache-airflow/stable/)
- [apache/airflow on Docker Hub](https://hub.docker.com/r/apache/airflow)
- [Provider packages](https://airflow.apache.org/docs/apache-airflow-providers/)

## Hardware Requirements for Self-Hosting Apache Airflow

Airflow's own guidance is at least 4 GB of RAM and 2 CPUs for the cluster as a whole; the worker is the service to grow.

| Resource | Minimum | Recommended |
|---|---|---|
| CPU | 2 vCPU total | 4+ vCPU, most on the worker |
| RAM | 4 GB total | 8 GB+, about 1 GB per service |
| Storage | Database only | Grows with run history; logs go to the bucket |
| Runtime | Python 3.13, PostgreSQL 12+, Redis 5+ | Managed PostgreSQL 18 and Redis 8 |

Memory is dominated by DAG parsing and by what your tasks do, so heavy work belongs in a service the task calls, not the worker itself.

## Self-Hosting Apache Airflow with Docker

The Airflow project publishes a Compose file for the same topology. These shell commands start it locally:

```
curl -LfO https://airflow.apache.org/docs/apache-airflow/3.3.1/docker-compose.yaml
mkdir -p ./dags ./logs ./plugins ./config
echo "AIRFLOW_UID=$(id -u)" > .env
docker compose up -d
```

Adding Python dependencies means extending the image rather than installing at container start:

```
FROM apache/airflow:3.3.1
COPY requirements.txt /tmp/requirements.txt
RUN pip install --no-cache-dir -r /tmp/requirements.txt
COPY dags/ /opt/airflow/dags/
```

On Railway, fork the source repository, add your packages and DAGs, and push: every service rebuilds from the same commit, keeping the five roles on matching versions.

## How Much Does Apache Airflow Cost to Self-Host?

Airflow is free and open source under the Apache 2.0 licence, with no paid tier, seat limits or task quotas. Managed alternatives such as Astronomer, Cloud Composer and Amazon MWAA start around $300 a month for a small environment. Self-hosting on Railway costs only the compute, database and storage used.

## FAQ

**What is Apache Airflow?**
An open-source platform for authoring, scheduling and monitoring workflows. Pipelines are Python code — directed acyclic graphs of tasks with dependencies — and Airflow schedules them, retries failures and records the history in a web UI.

**Why does the template include PostgreSQL, Redis and object storage?**
PostgreSQL is the metadata database Airflow requires beyond local testing, Redis is the broker moving queued tasks to workers, and the bucket holds task logs so the API server can show logs written on another service.

**How do I add my own DAGs to self-hosted Apache Airflow?**
Fork the source repository, drop your Python files into `dags/` and push. Or set `AIRFLOW_DAGS_GIT_REPO_URL` on all five Airflow services to add your git repository as a second, versioned DAG bundle without rebuilding.

**How do I scale Apache Airflow workers on Railway?**
Raise `AIRFLOW__CELERY__WORKER_CONCURRENCY` to run more tasks per container, or increase the worker's replica count. Keep the scheduler, triggerer and DAG processor at one replica each unless configured for high availability.


## Similar templates

- [N8N Main + Worker](https://railway.com/deploy/n8n-main-worker) — Deploy and Host N8N with Inactive worker.
- [Evolution API with n8n](https://railway.com/deploy/evolution-api-with-n8n) — Build a WhatsApp automation platform with Evolution API, n8n & Postgres.
- [Postgres Backup](https://railway.com/deploy/postgres-s3-backups) — Cron-based PostgreSQL backup to bucket storage

Open this page in a browser: https://railway.com/deploy/airflow