Veloxml Deploy

Deploy open-source LLMs directly to your own AWS/GCP account with one command. Zero Docker, zero Kubernetes, scale-to-zero.

Beta: Ready for use. But go easy on us, there may be a few kinks.

This repo is still under heavy development and the documentation is evolving. You're welcome to try it, but expect some breaking changes. Watch "releases" of this repo to receive a notification when we are ready for Beta. And give us a star if you like it!

This is a CLI and deployment engine that allows you to deploy open-source LLMs directly to your own cloud account (AWS/GCP) with a single command.

Your data, prompts, and model weights never leave your own AWS/GCP account. Zero third-party servers, and zero SOC2 or HIPAA compliance headaches

You don't have to pay a $50k-$100k enterprise paywall just to deploy inside your private VPC. VeloxML gives you that exact serverless experience natively in your account on day one

Zero framework lock-in. Modal forces you to rewrite your code with proprietary decorators ( @modal.function )

The beauty of deploying directly to your own cloud account is that your proprietary data, customer queries, and model weights never leave your security perimeter. Zero third-party compliance reviews (SOC2/HIPAA) needed.