Course outline · 0% complete

0/30 lessons0%

Course overview →

Closing down, TIME_WAIT, and when UDP wins

lesson 7-4 · ~13 min · 22/30

Closing takes four segments, not three

TCP is full duplex, meaning the two directions are independent streams. So closing is not one event, it is each side announcing that it has finished sending. That takes four segments.

  1. Client sends FIN. "I have no more data." It can still receive.
  2. Server sends ACK of that FIN. The client's direction is now closed.
  3. Server sends its own FIN, once its application has finished writing.
  4. Client sends ACK. Both directions are closed.

The gap between steps 2 and 3 can be long, and that is legitimate. It is called a half close, and it is what makes cat file | ssh host command work: the local side signals end of input while still reading the remote output.

There is also the abrupt option. RST (reset) tears the connection down immediately with no exchange, and it is sent when a segment arrives for a connection that does not exist, when a listening socket refuses, or when an application closes with unread data still buffered. RST is not part of an orderly close, it is an abort, and a capture full of RSTs usually means an application is closing sockets rudely or a middlebox is killing idle connections.

The state machine

Every TCP socket is in exactly one state, and ss prints it. Learning the list turns confusing output into a diagnosis.

StateMeaningWho is usually here
LISTENwaiting for SYNsa server socket
SYN-SENTsent a SYN, waiting for SYN-ACKa connecting client
SYN-RECEIVEDgot a SYN, sent SYN-ACK, waiting for ACKa server mid handshake
ESTABLISHEDdata may flow both waysa working connection
FIN-WAIT-1sent FIN, waiting for its ackthe side that closed first
FIN-WAIT-2our FIN acked, waiting for their FINthe same side
CLOSE-WAITgot their FIN, our application has not closed yetsee below
LAST-ACKsent our FIN after theirs, waiting for the final ackthe passive side
TIME-WAITwaiting out 2×MSS before forgetting the connectionthe active closer
CLOSEDno connection existsthe resting state

Two of these rows are diagnoses in themselves.

Many sockets stuck in CLOSE-WAIT is an application bug, not a network problem. The peer said FIN, the kernel acked it, and your program never called close on the socket. The file descriptors leak until the process hits its limit.

Many sockets in SYN-RECEIVED on a server can mean a SYN flood, where an attacker sends SYNs and never completes handshakes to fill the accept queue. The standard mitigation, SYN cookies, encodes the needed state in the ISN itself so the server can stay stateless until the third segment arrives.

active closerpassive closer1. FIN2. ACK3. FIN4. ACKFIN-WAIT-2CLOSE-WAITLAST-ACKFIN-WAIT-1TIME-WAITthe passive side is CLOSED at once, the active side waits 2×MSL
The four way close. The side that closes first lands in TIME_WAIT and holds the port pair for twice the maximum segment lifetime.

Why TIME_WAIT exists, and why it bites

The side that closes first sits in TIME-WAIT for twice the MSL (maximum segment lifetime), commonly 60 seconds total on Linux. During that time the four tuple cannot be reused. Two solid reasons:

1. The final ACK may be lost. If it is, the peer retransmits its FIN, and somebody has to be there to ack it again. A socket that vanished instantly would answer with RST, and the peer would log an error on a connection that completed perfectly. 2. Old segments may still be wandering. A delayed duplicate from the closed connection must not be accepted into a brand new connection that happens to reuse the same ports. Waiting two segment lifetimes guarantees the strays have expired.

The cost appears on busy clients and proxies, because TIME-WAIT consumes an ephemeral port for a minute. Do the arithmetic on a default Linux range:

ephemeral range 32768 to 60999 = 28,232 ports
TIME_WAIT       60 seconds
max new connections to ONE destination
  = 28232 ÷ 60 = about 470 per second

Past that, connect fails with "cannot assign requested address" while the machine looks completely idle. The correct fixes, in order of preference:

- Reuse connections instead of opening one per request. HTTP keep alive and database connection pools exist for exactly this. - Let the server close first, so the server side holds TIME-WAIT and the busy client does not. - Widen the ephemeral range or enable timestamp based reuse, which are real tunings but treat the symptom. - SO_REUSEADDR is a different thing and worth not confusing: it lets a listening socket bind while an old connection in TIME-WAIT still exists, which is why a restarted server no longer complains "address already in use".

When UDP is the right answer

TCP's guarantees cost a handshake, a close, buffering, and head of line blocking, meaning one lost segment stalls delivery of everything behind it. For some workloads that trade is wrong.

UseWhy UDP fits
DNSone small question, one small answer, retry is cheaper than a handshake
DHCPthe client has no address yet, so no connection is possible
NTPa timestamp exchange, where retransmitting a stale time is useless
Voice and videoa late frame is worthless, so skip it and keep going
Game stateonly the newest position matters, resending an old one is noise
QUIC and HTTP/3reliability is implemented above UDP, on the application's terms

The voice case makes the principle clear. If a 20 ms audio frame is lost, TCP would stall the stream, retransmit it, and deliver it 60 ms late, by which time the moment has passed and the delay is audible. UDP delivers the next frame on time and the codec conceals the gap. Timeliness beats completeness.

QUIC deserves its own paragraph, because it looks like a contradiction: a reliable, encrypted, congestion controlled transport built on top of unreliable UDP. It exists because TCP is implemented in the kernel and, worse, is inspected by middleboxes that break anything unfamiliar, so improving TCP itself proved nearly impossible. QUIC moves the whole transport into user space over UDP port 443 and gains four things:

1. A shorter handshake. The transport and TLS handshakes are combined, so a new connection costs one round trip instead of TCP plus TLS, and a resumed one can send data immediately. 2. Independent streams. Several streams share one connection, and a loss in one does not stall the others, removing TCP's head of line blocking. 3. Connection IDs instead of four tuples. A connection survives your address changing, so switching from Wi-Fi to mobile does not break the download. 4. Encrypted transport headers, so middleboxes cannot inspect or ossify them.

So UDP's lack of features is the feature. It gives an application the raw delivery service and lets it build precisely the guarantees it needs, which is Lesson 1-3's layering argument playing out one more time.