--- title: "Deploy Grafana 13 LGTM Observability Stack" description: "Logs, metrics and traces with Grafana, Loki, Tempo and Prometheus." category: "Observability" url: https://railway.com/deploy/grafana-13-lgtm-observability-stack --- # Deploy Grafana 13 LGTM Observability Stack Logs, metrics and traces with Grafana, Loki, Tempo and Prometheus. **[Deploy Grafana 13 LGTM Observability Stack on Railway](https://railway.com/template/grafana-13-lgtm-observability-stack)** Machine-readable deploy manifest (JSON, validated by TemplateCI): https://railway.com/deploy/grafana-13-lgtm-observability-stack/manifest.json - **Creator:** bento - **Category:** Observability - **Total deploys:** 1 ## Template content ### tempo https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/tempo.svg - **Source:** baranberkay96/grafana-lgtm-railway - **Health check:** /ready ### alloy https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/alloy.svg - **Source:** baranberkay96/grafana-lgtm-railway - **Health check:** /-/ready - **Public domain:** Yes ### loki https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/loki.svg - **Source:** baranberkay96/grafana-lgtm-railway - **Health check:** /ready ### prometheus https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/prometheus.svg - **Source:** baranberkay96/grafana-lgtm-railway - **Health check:** /-/ready ### grafana https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/grafana.svg - **Source:** baranberkay96/grafana-lgtm-railway - **Health check:** /api/health - **Public domain:** Yes ## Buckets - **LokiBucket** - **TempoBucket** ## Documentation # Deploy and Host Grafana LGTM with Railway [![Deploy on Railway](https://railway.com/button.svg)](https://railway.com/deploy/grafana-13-lgtm-observability-stack?referralCode=20H68q) LGTM is Grafana's open-source observability stack: Loki for logs, Grafana for dashboards, Tempo for traces and Mimir for metrics, here replaced by single-node Prometheus. This community template deploys Grafana, Loki, Tempo and Prometheus plus Grafana Alloy as a single OpenTelemetry endpoint, with logs and traces stored in Railway Buckets and the datasources linked to each other. ## About Hosting Grafana LGTM A self-hosted LGTM stack needs more than four containers: Loki and Tempo need object storage and retention, Prometheus needs a way to receive pushed metrics, applications need one place to send OpenTelemetry data, and Grafana needs datasources that link a slow trace to its logs and metrics. This template wires that up on Railway. Alloy receives OTLP from your services over the private network, or from anywhere through a token-protected public endpoint, and routes traces to Tempo, logs to Loki and metrics to Prometheus. Loki and Tempo keep their data in Railway Buckets, Tempo generates request-rate and service-graph metrics from your traces, and Grafana opens with everything provisioned. ## Common Use Cases - Collect traces, logs and metrics from apps on Railway with any OpenTelemetry SDK, using one endpoint. - Jump from a slow or failing trace to the logs and request metrics of that service, and back. - See service dependencies and error rates in a service graph built from traces. - Scrape `/metrics` endpoints of your Railway services and build dashboards and alerts on them. - Receive telemetry from apps hosted outside Railway through the token-protected OTLP endpoint. ## Dependencies for Grafana LGTM Hosting - Grafana `13.2.3` - Loki `3.7.8` (single binary) with a Railway Bucket - Tempo `3.1.0` (monolithic mode) with a Railway Bucket - Prometheus `v3.15.0` - Grafana Alloy `v1.20.1` (OpenTelemetry collector) ### Deployment Dependencies - Grafana documentation: https://grafana.com/docs/grafana/latest/ - Loki documentation: https://grafana.com/docs/loki/latest/ - Tempo documentation: https://grafana.com/docs/tempo/latest/ - Prometheus documentation: https://prometheus.io/docs/introduction/overview/ - Grafana Alloy documentation: https://grafana.com/docs/alloy/latest/ - OpenTelemetry SDK configuration: https://opentelemetry.io/docs/languages/sdk-configuration/otlp-exporter/ - Railway Buckets: https://docs.railway.com/storage-buckets ### Implementation Details | Service | Role | Public | Storage | |---|---|---|---| | grafana | Dashboards, Explore, alerting; datasources provisioned and linked | Yes | Volume (SQLite) | | alloy | OTLP endpoint: traces to Tempo, logs to Loki, metrics to Prometheus | Yes, port 4320, bearer token | – | | loki | Logs | No | Volume (WAL) + `LokiBucket` | | tempo | Traces, span metrics and service graphs | No | Volume (WAL) + `TempoBucket` | | prometheus | Metrics: remote write, OTLP, scraping | No | Volume (TSDB) | **First login** 1. Deploy the template. No input is required. 2. Open the `grafana` public domain and log in as `admin` with `GF_SECURITY_ADMIN_PASSWORD` from the `grafana` variables (applied on first start; change it later in Grafana). 3. Open Explore and pick Prometheus, Loki or Tempo. The stack's own metrics are there from the start. **Sending data from services in this project** Add to your service: - `OTEL_EXPORTER_OTLP_ENDPOINT` = `${{alloy.OTEL_EXPORTER_OTLP_ENDPOINT}}` (OTLP/HTTP on port 4318; gRPC is on port 4317) - `OTEL_EXPORTER_OTLP_PROTOCOL` = `http/protobuf` - `OTEL_SERVICE_NAME` = `{your-service-name}` **Sending data from outside Railway** - `OTEL_EXPORTER_OTLP_ENDPOINT` = `https://{alloy-public-domain}` (OTLP/HTTP only) - `OTEL_EXPORTER_OTLP_HEADERS` = `Authorization=Bearer%20{OTLP_PUBLIC_TOKEN}` (most SDKs decode `%20` to a space; if yours does not, use a literal space) Requests without the token are rejected with 401. Rotate the token by changing `OTLP_PUBLIC_TOKEN` on `alloy` and redeploying. **Other ways in** - Prometheus scraping: set `EXTRA_SCRAPE_TARGETS` on `prometheus` to a comma-separated list such as `${{api.RAILWAY_PRIVATE_DOMAIN}}:8080` (services exposing `/metrics`). - Prometheus remote write: `${{prometheus.PROMETHEUS_REMOTE_WRITE_URL}}`. - Loki push API for log shippers (Promtail-compatible clients, Vector, Fluent Bit): `${{loki.LOKI_PUSH_URL}}`. **Retention:** Loki `LOKI_RETENTION_PERIOD` (31 days), Tempo `TEMPO_RETENTION` (14 days), Prometheus `PROMETHEUS_RETENTION_TIME` (15 days) and `PROMETHEUS_RETENTION_SIZE` (4 GB). **Scaling:** this is a single-node stack. Scale vertically: give Loki and Tempo more memory as ingest grows, and Prometheus more memory and volume as series grow. Alloy is stateless; raise `ALLOY_MEMORY_LIMIT` with its memory. Grafana uses SQLite, so keep one replica. **Pinning and upgrades:** each version is pinned in `services/{service}/Dockerfile`. To upgrade, change the `FROM` tag in that Dockerfile, read the release notes (Loki and Tempo major versions can change configuration), and redeploy the service. Data in the buckets and volumes is kept. **Resources:** about 0.6 GB RAM idle (around $7/month). A setup with 5 to 15 instrumented services typically uses about 2.5 GB RAM and 30 GB of bucket storage (around $35/month). Hobby plan works; Pro once Prometheus needs more than 5 GB of volume. ### Why Deploy Grafana LGTM on Railway? Railway runs Grafana, Loki, Tempo, Prometheus and Alloy as one project with private networking to the services you monitor, volumes and S3 buckets for storage, generated secrets and usage-based billing. Your telemetry stays in your own project, and instrumenting a service is a matter of referencing one variable. ## Similar templates - [Rootprint](https://railway.com/deploy/rootprint-1) — Open-source logs and traces with full-text search on object-storage. - [Pyroscope profiling](https://railway.com/deploy/pyroscope-profiling) — Protected continuous profiling with durable Pyroscope storage. - [SigOnly](https://railway.com/deploy/sigonly) — Deploy SigNoz with a working demo app & config in one click Open this page in a browser: https://railway.com/deploy/grafana-13-lgtm-observability-stack