---
title: "Deploy Speaker Diarization | (Just Updated) Who Spoke When API, Nemotron on CPU, API Key Pre-Set"
description: "Nemotron speaker diarization API: who spoke when, up to 8 speakers"
category: "AI/ML"
url: https://railway.com/deploy/speaker-diarization-or-just-updated-who-
---

# Deploy Speaker Diarization | (Just Updated) Who Spoke When API, Nemotron on CPU, API Key Pre-Set

Nemotron speaker diarization API: who spoke when, up to 8 speakers

**[Deploy Speaker Diarization | (Just Updated) Who Spoke When API, Nemotron on CPU, API Key Pre-Set on Railway](https://railway.com/template/speaker-diarization-or-just-updated-who-)**

Machine-readable deploy manifest (JSON, validated by TemplateCI): https://railway.com/deploy/speaker-diarization-or-just-updated-who-/manifest.json

- **Creator:** SuperSlowSloth
- **Category:** AI/ML

## Template content

### diarization

- **Image:** ghcr.io/bon5co/diarization-railway:1.0.0@sha256:ef621f73f71a011e5ab83ed089ab88be7b49726a2edc1cb6be7e892671267468
- **Health check:** /health
- **Public domain:** Yes

## Documentation

# Deploy and Host Speaker Diarization API (Nemotron-3-Diarization) on Railway

Speaker diarization answers "who spoke when": give it a recording and it returns time-stamped segments labelled speaker_0, speaker_1 and so on, for up to eight voices. This template serves NVIDIA Nemotron-3-Diarization, a 100M-parameter open model, behind one authenticated JSON API on CPU. The weights are already inside the image, an API key is generated for you at deploy, and no GPU is needed.

## About Hosting Speaker Diarization

This template runs `nvidia/Nemotron-3-Diarization` with `transformers` 5.24.0 and PyTorch on CPU behind a small FastAPI server. The native `transformers` classes are used, so no remote model code and no NeMo toolkit. `POST /v1/diarize` takes audio as a multipart `file` or as a base64 `audio` field in a JSON body, in any format ffmpeg reads (wav, mp3, m4a, ogg, flac, opus, mp4), and returns `segments` with `speaker`, `speaker_id`, `start` and `end` in seconds, plus `num_speakers` and `duration_seconds`. Speakers are numbered by first appearance. Segments of different speakers may overlap, because the model detects overlapped speech. Add `rttm=true` to also get an RTTM string for scoring tools, and `merge_gap` (seconds) to join one speaker's neighbouring segments. `mode` defaults to `offline`, which gives the whole recording to the model; `low_latency`, `very_low_latency` and `ultra_low_latency` run the step-by-step streaming path of the same model, which is far slower on CPU. `GET /v1/models` describes the model.

The weights are baked into the image, so the first request is answered from disk and no model download hides inside a request that Railway's edge would abandon after five minutes. The service sizes its PyTorch thread pool from the container's own CPU quota instead of the host's core count, handles one request at a time so memory stays bounded, and refuses a recording longer than 1800 seconds or a file above 100 MB.

**Measured on Railway, 8 vCPU:** a 97.6 s conversation was diarized in **1.2 s** (81x real time); a 586 s recording in **10.2 s** (58x); a 1758 s recording in **28.5 s** (62x), the whole request answered in 28.9 s. The default `offline` mode is the one to use. The streaming `low_latency` mode took 25.4 s on the 97.6 s clip, 3.9x real time, about 20 times slower for the same audio. The model loads in about 5 s after the container starts, and the process peak resident memory read **0.8 GB** after boot, **1.3 GB** after the 586 s file and **2.3 GB** after the 1758 s file (`peak_rss_mb` on `/health`), so memory grows with recording length. Accuracy was not benchmarked here: the clip gave six speakers where the number actually present was not checked, so treat `num_speakers` as the model's estimate. Short back-channel utterances and heavy overlap are the usual weak spots of any diarizer.

### Agent contract

- Public URL: `https://RAILWAY_PUBLIC_DOMAIN` (the generated domain of the `diarization` service)
- Private URL: `http://diarization.railway.internal:8080`
- Credentials: the `API_KEY` variable of the `diarization` service
- Auth: send `Authorization: Bearer $API_KEY` on every route except `GET /health`; a call without the key or with a wrong one answers `401`
- Health: `GET /health` returns 200 once the model is loaded (503 while it loads)

```bash
# needs: RAILWAY_PUBLIC_DOMAIN, API_KEY (a variable of the diarization service); also curl.
# signup: none - the API has no user accounts, only the API_KEY bearer
R="https://$RAILWAY_PUBLIC_DOMAIN"
n=0; until [ "$(curl -s -o /dev/null -w '%{http_code}' "$R/health")" = 200 ]; do n=$((n+1)); [ "$n" -le 60 ] || break; sleep 5; done
curl -s -o /dev/null -w "health: %{http_code}\n" "$R/health"
# a 98 s recording of a conversation (public test clip, Hugging Face hf-internal-testing)
curl -sL -o /tmp/meeting.mp3 https://huggingface.co/datasets/hf-internal-testing/dummy-audio-samples/resolve/main/diarization_example.mp3
curl -s -o /dev/null -w "write without key: %{http_code}\n" -X POST "$R/v1/diarize" -F file=@/tmp/meeting.mp3
curl -s -o /dev/null -w "write wrong key: %{http_code}\n" -X POST "$R/v1/diarize" -H "Authorization: Bearer not-the-key" -F file=@/tmp/meeting.mp3
curl -s -X POST "$R/v1/diarize" -H "Authorization: Bearer $API_KEY" -F file=@/tmp/meeting.mp3 \
  | grep -o '"num_speakers":[0-9]*' | head -1
```

A fresh deploy prints `health: 200`, `write without key: 401`, `write wrong key: 401` and a `"num_speakers"` count for the test clip. To diarize your own file and get an RTTM:

```bash
curl -X POST "https://$RAILWAY_PUBLIC_DOMAIN/v1/diarize" -H "Authorization: Bearer $API_KEY" -F file=@call.wav -F rttm=true -F merge_gap=0.5
```

An unreadable file answers `422` with the reason, an unknown `mode` answers `422`, and a recording over the limit answers `413`. The API never fetches a URL for you: it only reads the bytes you send, so it cannot be pointed at an address inside your network.

## Why Deploy Speaker Diarization API on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it.

By deploying Speaker Diarization on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.

- **Who spoke when, in one request** — speaker labels with start and end times, ready to merge with a transcript.
- **Your recordings stay in your container** — calls and meetings are read by your own service, not sent to a hosted diarization API.
- **No GPU bill and no per-minute fee** — about an hour of audio per minute of compute on 8 vCPU, and you pay for the container.
- **Open weights, commercial use** — the OpenMDW-1.1 licence grants use without restriction and puts no restriction on outputs.
- **Private by default** — a generated bearer key guards every route but the health check, and the container refuses to start without one.

## Common Use Cases

- **Meeting and call transcripts** — pair the segments with a speech-to-text service to label each line with its speaker.
- **Podcast and interview editing** — find each guest's talk time and cut by speaker.
- **Speaker tool for AI agents** — give an LLM agent one authenticated request that says how many people talk in a recording and when.

## Dependencies for Speaker Diarization Hosting

- A service with **at least 4 GB of RAM** for recordings up to the 1800 s limit (peak resident memory measured at 2.3 GB after a 1758 s file).
- No volume: the service keeps no state and the model is in the image.

### Deployment Dependencies

- [nvidia/Nemotron-3-Diarization](https://huggingface.co/nvidia/Nemotron-3-Diarization) — the model, a streaming Sortformer with up to 8 speakers
- [huggingface/transformers](https://github.com/huggingface/transformers) — the inference library, Apache-2.0, version 5.24.0
- [bon5co/diarization-railway](https://github.com/bon5co/diarization-railway) — the Railway wrapper image this template deploys, Apache-2.0
- Weights licence: the Hugging Face model card of `nvidia/Nemotron-3-Diarization` declares `license: openmdw-1.1` and says "Use of this model is governed by the OpenMDW License Agreement, version 1.1". The agreement grants "permission ... free of charge, to deal in the Model Materials without restriction", requires that "if you distribute any portion of the Model Materials, you shall retain in your distribution (1) a copy of this agreement, and (2) all copyright notices and other notices of origin", and states that it "does not impose any restrictions or obligations with respect to any use, modification, or sharing of any outputs generated by using the Model Materials". The image keeps the agreement next to the weights. Attribution: Nemotron-3-Diarization, copyright NVIDIA Corporation, OpenMDW-1.1.


## Similar templates

- [Chat Chat](https://railway.com/deploy/-WWW5r) — Chat Chat, your own unified chat and search to AI platform.
- [Weaviate](https://railway.com/deploy/weaviate-1) — Vector database with hybrid search and an API key-protected API
- [Möbius Personal AI OS](https://railway.com/deploy/mobius-v01-personal-ai-os) — Self-hosted AI agent that builds and runs apps for you, with memory.

Open this page in a browser: https://railway.com/deploy/speaker-diarization-or-just-updated-who-
