Deploy Litellm-Novita-ai

Deploy and Host Litellm-Novita-ai on Railway

Litellm-Novita-ai is a self-hosted, OpenAI-compatible AI gateway that routes requests exclusively to open-source models hosted by Novita AI. It combines LiteLLM with Redis caching and PostgreSQL persistence to provide centralized credentials, virtual API keys, usage tracking, budgets, rate limits, and lower-cost model access through one endpoint.

About Hosting Litellm-Novita-ai

Hosting Litellm-Novita-ai on Railway deploys a complete three-service stack. LiteLLM provides the public API gateway and administration interface, Redis caches repeated model responses, and PostgreSQL stores virtual keys, users, budgets,
configuration, and usage logs. Get a Novita AI API key: https://novita.ai/?ref=mzblm2z&utm_source=affiliate

Common Use Cases

• Provide one OpenAI-compatible endpoint for multiple Novita-hosted models.
• Reduce latency and API costs by caching repeated requests in Redis.
• Create virtual API keys for separate applications, users, or teams.
• Apply budgets, expiration dates, and rate limits to API keys.
• Track model usage and spending in PostgreSQL.
• Connect OpenAI-compatible applications to Novita AI models.
• Keep provider credentials hidden from client applications.

Dependencies for Litellm-Novita-ai Hosting

• A Novita AI account and API key.
• LiteLLM Proxy for the OpenAI-compatible gateway.
• Redis for response caching.
• PostgreSQL for persistent keys, budgets, and usage logs.

Deployment Instructions

1. Deploy the Railway template.
2. Enter your NOVITA_API_KEY when prompted.
3. Confirm that the project contains:
   • LiteLLM Gateway
   • Redis
   • PostgreSQL
4. Confirm that the Gateway has these variables:
   • REDIS_URL=${{Redis.REDIS_URL}}
   • DATABASE_URL=${{Postgres.DATABASE_URL}}
5. Open the admin interface at https://YOUR-GATEWAY.up.railway.app/ui.

Getting Started

Configure your application with:

• OPENAI_BASE_URL=https://YOUR-GATEWAY.up.railway.app/v1
• OPENAI_API_KEY=YOUR_LITELLM_KEY

Configuration

• NOVITA_API_KEY — Novita AI provider credential.
• LITELLM_MASTER_KEY — gateway administrator credential.
• LITELLM_SALT_KEY — encryption key for stored credentials.

Security

Only the LiteLLM Gateway should receive a public domain. Redis and PostgreSQL should remain on Railway's private network.

Applications should use LiteLLM virtual keys rather than the master key. Virtual keys can have independent budgets, rate limits, expiration dates, model restrictions, and revocation controls.

Why Deploy Litellm-Novita-ai on Railway?

Railway is a singular platform to deploy your infrastructure stack. Railway will host your infrastructure so you don't have to deal with configuration, while allowing you to vertically and horizontally scale it. By deploying Litellm-Novita-ai on Railway, you are one step closer to supporting a complete full-stack application with minimal burden. Host your servers, databases, AI agents, and more on Railway.