Deploy Sentence Transformers Model

Given any sentence transformer model, this template will deploy a quick and easy FastAPI server to serve embeddings for that model.

Setup requires you to choose which model to run, and choose the number of workers. Be careful using models that are too large, since a) they'll be expensive since the weights will be loaded into memory N times (i.e. depending on number of workers you chose during setup), and b) they'll be running on CPU so inference isn't overly quick compared to GPUs.

Usage is simple, just post {"texts":["hello","world"]} to /embed on the exposed server, and you'll get back the embeddings in JSON.