One box runs everything

One server runs everything

Every system you have heard of, Google, Netflix, WhatsApp, started as one program on one computer. This course grows that one computer into a planet-scale system, one problem at a time.

You already hold the prerequisites. From How the Internet Works you know a browser sends an HTTP request and gets back a response. From backend basics you know a server is a program listening on a port. From SQL you know a database stores rows in tables.

Our starting point, and the anchor picture for the whole course, is the simplest real architecture:

• One machine in a data center (engineers call it a box)
• The app process: your backend code, running on that box
• The database: running on the same box

Plenty of profitable products run exactly this. Never add complexity before the simple version breaks.

The life of one request

When a user taps Load feed:

1. The browser sends GET /feed across the internet to your box, using DNS and TCP exactly as in How the Internet Works
2. The app process runs your handler code
3. The handler queries the database: SELECT * FROM posts WHERE ...
4. The database reads rows from disk and returns them
5. The app turns rows into JSON and responds

Total time is a few milliseconds of app code plus tens of milliseconds of database and network time. One box can do this thousands of times a minute without strain.

Notice that steps 3 and 4 are where most of the time goes, which will matter for the whole rest of the course. The app code is fast because it is arithmetic and string building, and the database is slower because it touches disk.

How much traffic is that in real units? The number system designers care about is requests per second, written RPS, and it is the number every capacity decision starts from.

A traffic calculator

This converts users per day into average requests per second, which is the basic unit of load.

users = 2000
requests_per_user = 40
requests_per_day = users * requests_per_user
seconds_per_day = 24 * 60 * 60
print("Requests per day:", requests_per_day)
print("Average per second:", round(requests_per_day / seconds_per_day, 2))

Output

Requests per day: 80000
Average per second: 0.93

Eighty thousand requests a day sounds like a lot and works out to under one per second. That gap between the daily total and the per-second rate is why RPS is the unit designers use, since the daily number makes small systems sound busy.

seconds_per_day is written as 24 * 60 * 60 rather than 86400 on purpose. The arithmetic is visible, so a reader can check it without trusting a magic number.

Changing users to 20000 would push the average into double digits, and that is the scale where one box starts to need attention. The relationship is linear, so ten times the users is ten times the RPS.

Note what this number is not, which is the load the server actually sees. It is an average across a full day, including the hours when almost nobody is awake.

The same math at 50,000 users

Fifty thousand users making 20 requests each per day is a million requests, and the per-second figure is still small.

users = 50000
requests_per_user = 20
requests_per_day = users * requests_per_user
seconds_per_day = 24 * 60 * 60
print("Requests per day:", requests_per_day)
print("Average per second:", round(requests_per_day / seconds_per_day, 2))

Output

Requests per day: 1000000
Average per second: 11.57

A million requests a day is 11.57 per second, and that is worth committing to memory as a reference point. It is the single most useful conversion in back-of-envelope estimation, and unit 9 leans on it constantly.

round(x, 2) keeps the output readable, and the precision is honest about what these numbers are. An estimate built from a guessed request count does not deserve six decimal places.

The two runs together show the shape of the problem. Twenty-five times the users produced twelve times the load here, because each user was assumed to make half as many requests, so both factors matter.

Eleven requests per second is still comfortable for one box. This is the number to keep in mind whenever someone describes a system as handling a million requests a day, since it sounds impressive and is not.

Why an app under 1 RPS crashes at 8 pm

Because averages hide peaks, and traffic bunches into busy hours that are many times the average.

Traffic is never spread evenly. People use a consumer app in the evening, a work tool during business hours, and a food delivery service at lunch and dinner, so the load follows human schedules.

A site averaging 0.93 RPS might see 10 to 20 RPS during its busiest hour, which is a tenfold or twentyfold difference from the number the daily total suggested. The box that looked idle on paper is the same box that falls over at dinner time.

Capacity planning always uses peak load, never average load. A server sized for the average is a server that fails every day at the same time, and the failure looks mysterious precisely because the average metric looks fine.

Remember this, because every estimate in this course multiplies the average by a peak factor. The factor is a guess, usually somewhere between 2 and 10, and stating it out loud is part of doing the estimate honestly.