Deploy Argilla
Build AI datasets with your team: Argilla 2.8 with Postgres, ES, Redis
Elasticsearch
Just deployed
/usr/share/elasticsearch/data
Just deployed
Redis
Just deployed
/data
Just deployed
Just deployed
/var/lib/postgresql/data
Deploy and Host Argilla on Railway
Argilla is an open-source (Apache-2.0) collaboration tool for AI engineers and domain experts who build datasets for AI models. Teams load records with its Python SDK, then label, rank and review them in a web UI: text classification, span tagging, preference and rating data for fine-tuning LLMs, and evaluation sets.
About Hosting Argilla
Argilla 2.x is a FastAPI server with a bundled web UI. It keeps users, datasets and annotations in PostgreSQL, indexes records in Elasticsearch for search and filtering, and runs imports, exports and webhooks through a Redis-backed background worker. This template deploys all five: Argilla 2.8.0 and its worker (official image pinned by version and digest), Elasticsearch 8.19 with security on and a volume, and Railway's Postgres and Redis with volumes. The owner account admin is created on the first start with a generated password and API key, so there is nothing to fill in. The first deploy takes about 3 to 5 minutes while Elasticsearch starts.
Common Use Cases
- Collecting human feedback on LLM outputs (ratings, rankings, corrections) for SFT, DPO or RLHF datasets
- Labeling text for classification, named-entity and span tasks with a team of annotators
- Reviewing model predictions and building evaluation sets, then pushing them to the Hugging Face Hub
Dependencies for Argilla Hosting
- PostgreSQL (Railway's Postgres, included) for users, datasets and annotations
- Elasticsearch 8 (included, built from the template repo) for search, and Redis (Railway's Redis, included) for background jobs
Deployment Dependencies
- Argilla server image: argilla/argilla-server v2.8.0 (release notes)
- Elasticsearch image: elastic/elasticsearch 8.19.25
- Template repo, Dockerfiles and changelog: aalfath/argilla-railway-template
- Argilla documentation: docs.argilla.io
Implementation Details
After the deploy turns green, open the Argilla service's domain and log in as admin with the PASSWORD from the Argilla service's Variables tab. Use the API_KEY from the same tab in Python:
import argilla as rg
client = rg.Argilla(api_url="https://YOUR-DOMAIN", api_key="YOUR-API-KEY")
Add annotators with rg.User(username="...", password="...", role="annotator").create() and give them a workspace. Elasticsearch has no public domain and answers only its health check without a password. Optional settings (Hugging Face OAuth sign-in, HF_HUB_DISABLE_TELEMETRY=1 to turn off anonymous telemetry) are listed in the repo README.
Updates: the Argilla, Worker and Elasticsearch services build from the template's GitHub repo, so Railway shows an "update available" notice when a new version is merged. Read the repo's CHANGELOG and back up Postgres before applying one, because Argilla migrates its schema on start. Argilla's authors put the project in maintenance mode in 2025: bug fixes only, no new features.
Why Deploy Argilla on Railway?
Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.
By deploying Argilla on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.
Template Content
Elasticsearch
aalfath/argilla-railway-templateRedis
redis:8.2
