hugging-face-transformer

A Hugging Face Transformer Railway Template

Deploy hugging-face-transformer

transformer-fastapi

JigsawStack/huggingface-transformers-railway-template

Just deployed

Hugging Face Transformer Example

This example starts up a FastAPI server which runs the Hugging Face Transformers.

This example runs an embedding model on the CPU which works great with railway as resources can scale per requests.

Note: This won't work for GPU work flows and will crash.

⬆️ Deploy

Deploy on Railway

or

Install the Railway CLI, then run railway up

✨ Features

  • Transformers
  • FastAPI
  • Hypercorn
  • Python 3.11

Uses Nixpacks to deploy on railway which runs Python 3.11 by default.

💁‍♀️ How to use locally

  • Clone locally and install packages with pip using pip install -r requirements.txt
  • Run locally using hypercorn main:app --reload

🧩 Background

The team at JigsawStack is launching an embedding model and we're experimenting with different infrastructure for scalable and affordable CPU usage. Check out the embedding model here

Common issues

  • When deployed on Railway, your instance region might be deployed to a metal region which is currently in BETA and seem to be a lot slower than non-beta regions. Switch to US West (Oregon, USA) for the best performance on testing. Metal regions don't have volume attachments which could be the issue for caching data.
  • You can attach a volume if you switching models in a single instance, this would allow for better caching and faster switches
  • If you need a specific python version, you can set NIXPACKS_PYTHON_VERSION in the variables tab to the desired version

Template Content

More templates in this category

View Template

Chat Chat

Chat Chat, your own unified chat and search to AI platform.


View Template

openui

Deploy OpenUI: AI-powered UI generation with GitHub OAuth and OpenAI API.


View Template

firecrawl

firecrawl api server + worker without auth, works with dify