Cold starts, honestly

The catch

Salespeople skip this slide, so here it is. When an event arrives and no copy of your function is warm, AWS must create one: provision a micro-VM, load your code, start the runtime. That pause is a cold start, and the request that triggered it waits through all of it: commonly 100 to 1000 ms, worse for big deployment packages and heavy runtimes, and it happens again per concurrent copy when traffic jumps.

After the run, AWS keeps the copy warm for a while, so the next event skips the pause. Under steady traffic most invocations are warm, and cold starts hide in the slowest fraction of requests, which is exactly where users notice them. Engineers measure that slow fraction with percentiles: sort all response times, and p99 is the value that 99% of requests come in under, leaving only the slowest 1% above it. A cold start barely moves the average, but it shows up instantly at p99, which is why latency budgets are usually written as percentiles, not averages.

Honest guidance: for background jobs and pipelines like lesson 7-2's thumbnails, cold starts are irrelevant. For latency-sensitive user-facing APIs, they are a real cost you weigh against never patching a server, and mitigations (like paying for pre-warmed capacity) exist but cancel some of the savings.