Week 2 · Day 7 of 10
Run it in production: Docker, logs, cost and latency
Students can package their app with Docker, log every model call, and make it faster or cheaper with numbers to prove it.
1LearnDocker, logs, cost and latency
Packaging the app with Docker so it runs on any cloud host, not only Vercel. Logging every model call (tokens, cost, latency, errors) as structured lines. Reading the logs to find the slow and costly paths, then fixing them: streaming so the first words show in about a second, prompt caching, and sending easy requests to a cheaper model, with day 5's evals checking that quality holds.
How you make it faster and cheaper
- Docker
Check your understanding
What does packaging your app with Docker buy you?
Your app feels slow and costs more than expected. Where do you start?
A model is four times cheaper. When should you switch to it?
2BuildRun it in Docker, then make it cheaper
Package your app with Docker and run it, log every model call, then cut the cost or latency of your slowest route and prove with your evals that quality held.
Done when: your app runs in Docker, logs every call, and you can say what one request costs.
3CapstoneCost and latency, before and after
A cost per request and a latency number, before and after.
4HomeworkCost it at 10, 100, 1,000 users
Write a one-page cost estimate: what your app costs per month at 10, 100 and 1,000 users.
For teachers: the day 7 teaching kit.