Latency, bandwidth, and round trips

Why fast servers still feel slow

You now know that opening a TCP connection costs a full back-and-forth before any data moves. This lesson names the costs, because they explain a mystery every engineer eventually hits: a server that responds in 2 ms can still feel slow from the other side of the world, and no amount of server power fixes it. Understanding why is how you avoid optimizing the wrong thing.

Two different measurements get mixed up constantly:

• Latency is the time one message takes to reach the other machine, set mostly by physical distance and the number of routers on the way. Measured in milliseconds (ms).
• Bandwidth is the volume a connection can carry per second, like the width of a pipe. Measured in bits or bytes per second.

A round trip is one message over and one reply back, and its duration is the RTT (round-trip time). Typical RTTs are a few ms within a city, around 70 ms across a continent, and 150–250 ms between continents.

Here is the punchline. Before a browser receives the first byte of a page, it spends several sequential round trips: the DNS lookup, the TCP handshake from lesson 3-2, the TLS handshake from unit 7, and finally the request itself. Each one must finish before the next starts, so the waiting adds up multiplicatively with distance.

The same server at three distances

This computes the minimum time before the first byte arrives, for one 4-round-trip setup at three distances.

rtts = {"same city": 5, "cross country": 70, "other continent": 200}

round_trips = 4  # DNS + TCP handshake + TLS handshake + the request itself

for place, rtt in rtts.items():
    print(place + ":", round_trips * rtt, "ms before the first byte")

Output

same city: 20 ms before the first byte
cross country: 280 ms before the first byte
other continent: 800 ms before the first byte

Nothing about the server changed between those three lines. Only the distance did, and the result moved by a factor of 40.

The multiplication is the point. Because the four round trips are sequential, distance is not added once, it is added four times, so a slow link punishes a chatty setup much harder than a fast one.

That is also the ceiling on any server-side optimization. If travel alone costs 800 ms, shaving 2 ms of processing changes nothing a user can perceive, which is what the next block makes concrete.

Why a 2 ms server takes 800 ms in Sydney

Because setting up the conversation takes several sequential round trips, and each one costs the full 200 ms of distance.

DNS, the TCP handshake, the TLS handshake, and the request itself each need a full round trip, one after another. Four trips at 200 ms is 800 ms of pure travel time, and the server's 2 ms of processing barely registers in the total.

No server upgrade improves this, because the bottleneck is the speed of light through fiber plus the routers along the way. Latency is a property of the path rather than the machine.

Which is why moving content closer to users, the CDN idea in unit 8, often beats making servers faster. Cutting the RTT from 200 ms to 20 ms saves 720 ms in one move, and no code change can match that.

When bandwidth matters, and when it does not

A useful mental formula for one download:

total time ≈ (setup round trips × RTT) + (size ÷ bandwidth)

The first term is latency's share and the second is bandwidth's. For a small request, say an API call returning 2 KB of data, the transfer part is nearly zero and latency is everything. For a 2 GB video, the transfer term dominates and bandwidth is what you would pay to improve. Diagnosing "slow" starts with asking which term you are actually in.

This is also why HTTP clients reuse connections. Keep-alive (the default in HTTP/1.1) keeps the TCP and TLS connection open after a response so the next request to the same server skips both handshakes and pays only one round trip. Browsers go further and issue independent requests in parallel rather than queueing them.

Watching setup and transfer trade places

Three downloads over a connection with 100 ms RTT, 3 setup round trips, and 10 KB per millisecond of bandwidth, which is 10 MB/s.

rtt_ms = 100
setup = 3 * rtt_ms       # TCP + TLS + request round trips before the first byte
kb_per_ms = 10           # bandwidth: 10 MB/s is 10 KB per millisecond

for size_kb in [10, 100, 10000]:
    transfer = size_kb // kb_per_ms
    total = setup + transfer
    print(size_kb, "KB:", total, "ms (setup", setup, "+ transfer", str(transfer) + ")")

Output

10 KB: 301 ms (setup 300 + transfer 1)
100 KB: 310 ms (setup 300 + transfer 10)
10000 KB: 1300 ms (setup 300 + transfer 1000)

Reading the numbers

• transfer = size_kb // kb_per_ms uses integer division, which keeps the result a whole number of milliseconds.
• str(transfer) + ")" glues the closing parenthesis on without a space, since print would otherwise insert one between arguments.
• The 10 KB and 100 KB downloads cost almost the same, 301 ms against 310 ms, because setup dominates small transfers. Ten times the data added 3 percent to the time.
• Only the 10 MB download is really paying for bandwidth, where transfer is 1000 ms against 300 ms of setup. Buying a wider pipe helps that line and does nothing for the first two, which is the whole diagnostic value of splitting the formula in two.

Five sequential API calls at 200 ms RTT

The minimum wait is 1000 ms, and the fix is to issue the five requests in parallel, or batch them into one.

Sequential calls stack their round trips, so 5 times 200 ms is 1000 ms minimum regardless of how fast the server computes. The server could answer instantly and the page would still take a second.

Parallel requests overlap the waiting, so the total approaches a single RTT instead of five. Batching the five questions into one request gets there exactly, at the cost of a less granular API.

Counting round trips is one of the highest-leverage habits in web performance work. It is a number you can read off the code by looking for await inside a loop, without measuring anything.