Design a chat app

Interview 2: Design a chat app

Step 1, requirements. Functional: one-to-one messages, delivered instantly when the recipient is online, stored and shown when they come back. Non-functional: low latency (chat feels dead beyond ~1 second), no lost messages, 50 million users online at peak. Out of scope after asking: group chat, video.

Step 2, estimation. 50M online users sending 40 messages a day each: about 23,000 messages per second average. Each message ~200 bytes, so 2B messages/day ≈ 400 GB/day of message storage. That storage number, unlike the URL shortener's, says sharding from day one (unit 5), keyed by conversation so a chat's history lives together.

The new problem. HTTP is request-response: servers answer, they never call first. But chat needs the server to push Bob's message to Ada the instant it arrives. The tool is a websocket: a connection the client opens and both sides keep open, letting the server send messages down at any time.

Steps 4 and 5, the message path

Websockets break lesson 2-3's rule: a connection is state, pinned to one gateway. We handle it honestly:

1. Ada's phone holds a websocket to gateway 17, Bob's to gateway 62. A Redis registry (lesson 2-3's shared store) maps user → gateway
2. Ada sends a message. Gateway 17 hands it to a stateless chat service, which writes it to the sharded message store, the source of truth, before anything else
3. The chat service looks up Bob, finds gateway 62, and forwards the message there for push delivery
4. If Bob is offline, the message simply waits in storage. When he reconnects, his client asks for everything after the last message ID it has

Deep dives interviewers reach for: delivery receipts are just tiny messages flowing backward. Ordering within a chat comes from sequence numbers assigned by the shard that owns the conversation (one writer per conversation, the lesson 4-2 trick). Duplicate sends on retry are killed by idempotency keys (lesson 6-2).