Self-Serve Model Deployment
Upload a trained model through an admin panel and get a predict endpoint, with version and performance tracking, so ML engineers can ship classifiers without DevOps help.
Problem
ML engineers on a client team could train classifiers, but they needed DevOps help to ship them.
What I built
For a client engagement, I built a platform where an ML engineer uploads a trained model through an admin panel and gets a predict endpoint, with version and performance tracking. The client version isn't public.
The platform detects the framework without executing the artifact. It loads a model only inside an isolated worker, because unpickling can run code. Each dependency set gets one runtime image.
The live version is a database pointer, so promotion and rollback are near-instant. Before promotion, a parity check compares in-container predictions with the engineer's local ones.
Stack: FastAPI, React with TypeScript, Postgres, S3, Redis, and a container registry.
admin panel (React + TypeScript)
│ upload model + local predictions
▼
FastAPI ── detect framework, no execution ── artifact to S3
│
▼
isolated worker (one runtime image per dependency set)
│ load model · parity check vs local predictions
▼
Postgres: live-version pointer ──▶ predict endpointDecisions and tradeoffs
A UI platform over a CLI or a scaffold, because the users are ML engineers, not DevOps people.
Promotion by pointer instead of baking each version into an image. The tradeoff: a new dependency set still needs a rolling restart.
Scope limited to scikit-learn, xgboost, and lightgbm.
What's next
Drift monitoring was deferred out of the first scope.