Deploy and Host Agent Ranking Score on Railway

Measure precision at K for retrieved context.

About Hosting Agent Ranking Score

This is a stateless JSON tool. Deploy it, copy TOOL_TOKEN from service variables and send it as an Authorization: Bearer header on POST /run. The /health endpoint is public and returns readiness only. Inputs are limited to 64 KiB. The application does not store requests or call a model.

Why Deploy Agent Ranking Score

Call a deterministic operation from an agent, workflow or application without an LLM API key.

Common Use Cases

Measure precision at K for retrieved context.
Insert a deterministic check or transformation in a workflow.
Repeat the documented example in a regression check.

Dependencies for Agent Ranking Score

Python 3.13 container and a Railway project.
An HTTP client with your generated bearer token.

Example input for POST /run

{"k": 2, "retrieved": ["a", "b"], "relevant": ["a"]}

Expected response

{"result": 0.5}

The single process is intended for small internal workloads. Set concurrency and load limits upstream before serving high volume traffic.