The bottleneck
Traffic grows. At some point one resource on the box hits its limit, and every request queues behind it. That resource is the bottleneck: the narrowest point in the pipe. Fixing anything other than the bottleneck changes nothing, which is why finding it comes before fixing it.
On a single server, the usual suspects in the order they tend to fail:
- Database time. Tables grow, queries that scanned 1,000 rows now scan 10 million
- CPU. The processor is busy 100% of the time, requests wait for a turn
- Memory (RAM). The app or database needs more working space than exists
- Connections. The server can only hold so many open network connections
In practice the database is the first and most common bottleneck, and units 3 through 5 are devoted to it.
Code exercise · python
Run this capacity check. We take the average RPS from lesson 1-1, apply a 10x peak factor, and compare against what the box can handle.
Where the time goes
To find a bottleneck, measure where each request spends its time. A typical breakdown for one request on our box:
| Step | Time |
|---|---|
| App code (parse, logic, JSON) | 5 ms |
| Database query | 20 ms |
The database takes 80% of the time. If the server handles requests one at a time, the maximum throughput is 1000 ms ÷ 25 ms = 40 requests per second. Real servers overlap requests, but the ratio still tells you what to fix first: making app code twice as fast saves 5 ms, making the query twice as fast saves 10 ms.
This measuring habit has a name you will meet again in unit 8: observability.
Code exercise · python
Your turn. Given the timing breakdown, print the total time per request in ms, the database share as a whole percent (like 80%), and the max requests per second using integer division of 1000 by the total.
Quiz
Requests take 25 ms: 5 ms of app code and 20 ms of database time. You rewrite the app code so it runs in 1 ms. What happens to the site under heavy load?
Problem
A one-box app averages 4 requests per second. Its busiest hour runs at 8 times the average. The box can handle 30 requests per second. During the busiest hour, how many requests per second arrive? (Answer with the number.)