Jaydip Sanghani's Projects

gpt-vLLM

model name, check https://docs.vllm.ai/en/stable/models/supported_models.html for more details

Sets the amount of RAM (in GiB) allocated to key-value (KV) cache for CPU inference, modify to get better performance

Start Redis process as root to allow for writing to the volume

Public URL to connect to Redis, needed for the Data panel.

URL to connect to Redis over the private network.

secret key for authentication, so use something hard to guess