hugging-face-transformer
A Hugging Face Transformer Railway Template
transformer-fastapi
JigsawStack/huggingface-transformers-railway-template
Just deployed
Hugging Face Transformer Example
This example starts up a FastAPI server which runs the Hugging Face Transformers.
This example runs an embedding model on the CPU which works great with railway as resources can scale per requests.
Note: This won't work for GPU work flows and will crash.
⬆️ Deploy
or
Install the Railway CLI, then run railway up
✨ Features
- Transformers
- FastAPI
- Hypercorn
- Python 3.11
Uses Nixpacks to deploy on railway which runs Python 3.11 by default.
💁♀️ How to use locally
- Clone locally and install packages with pip using
pip install -r requirements.txt
- Run locally using
hypercorn main:app --reload
🧩 Background
The team at JigsawStack is launching an embedding model and we're experimenting with different infrastructure for scalable and affordable CPU usage. Check out the embedding model here
Common issues
- When deployed on Railway, your instance region might be deployed to a metal region which is currently in BETA and seem to be a lot slower than non-beta regions. Switch to US West (Oregon, USA) for the best performance on testing. Metal regions don't have volume attachments which could be the issue for caching data.
- You can attach a volume if you switching models in a single instance, this would allow for better caching and faster switches
- If you need a specific python version, you can set
NIXPACKS_PYTHON_VERSION
in the variables tab to the desired version
Template Content
transformer-fastapi
JigsawStack/huggingface-transformers-railway-template