What was happening when the comment vanished
Her reload read from a replica that had not yet replayed the write.
This is replication lag from lesson 4-3. The write reached the leader, and the read hit a follower still catching up, so both nodes were behaving correctly.
The comment reappearing a second later is the diagnostic detail. A bug that loses data does not un-lose it, so data that arrives late points at replication rather than at a defect.
This unit gives that phenomenon its proper name and its rules, since the replica was eventually consistent with the leader. Naming it matters because the same behavior appears in caches, queues, and distributed databases, and it is the same trade every time.
Eventual consistency
A replicated system is strongly consistent if every read, anywhere, returns the latest write, as if there were only one copy. It is eventually consistent if replicas are allowed to be temporarily out of date, with the guarantee that once writes stop, all copies converge to the same value.
Eventual consistency is not sloppiness, it is a deliberate trade. To be strongly consistent, a write must reach every replica (or a majority, next lesson) before it is confirmed, making writes slower and impossible during network trouble. To be eventually consistent, a write can be confirmed by one node and spread in the background, staying fast and available.
Most of the real world runs eventually consistent and nobody notices: a like count that is 2 seconds behind harms no one. The skill is naming the data where staleness does harm, money, inventory, permissions, and paying the consistency cost only there.
Replicas converging
Three replicas each hold a value and a version number, a write lands on A only, then a sync spreads the newest value.
replicas = {"A": ("likes=0", 0), "B": ("likes=0", 0), "C": ("likes=0", 0)}
replicas["A"] = ("likes=1", 1)
print("right after the write:")
for name in sorted(replicas):
print(" ", name, "->", replicas[name][0])
newest = max(replicas.values(), key=lambda pair: pair[1])
for name in replicas:
replicas[name] = newest
print("after replicas sync:")
for name in sorted(replicas):
print(" ", name, "->", replicas[name][0])Output
right after the write: A -> likes=1 B -> likes=0 C -> likes=0 after replicas sync: A -> likes=1 B -> likes=1 C -> likes=1
The first block is the inconsistent window and the second is convergence. Eventual consistency is precisely the promise that the second block happens, without a promise about how long the first one lasts.
A read during that window gets either 0 or 1 depending on which replica answers, and both are answers the system genuinely holds. That is the whole user-visible consequence, and it is why the disappearing comment looked like a bug.
Version numbers are how replicas know which value is newest, and max(..., key=lambda pair: pair[1]) is comparing on them. Without a version, a replica receiving likes=0 from a peer could not tell whether that value is older or newer than its own.
Real systems use fancier clocks, and the idea is the same. Vector clocks and Lamport timestamps exist because a single counter cannot order writes that happened on different nodes at once, which is the conflict problem from lesson 4-2 in a new guise.
The background sync has a name worth knowing, which is anti-entropy. Replicas periodically compare state and fix disagreements, so convergence does not depend on any single message arriving.
Note the shape of the guarantee once more. Once writes stop, all copies agree, and while writes continue there may always be some replica that is slightly behind, which is fine for a like count and not for a balance.
The CAP intuition
Now the famous theorem, minus the fog. Replicas talk over a network, and networks sometimes break so that node groups cannot reach each other: a partition. During a partition, a replica receiving a read has exactly two options:
- Answer from what it has, possibly stale. The system stays available but gives up consistency
- Refuse or wait until it can check with the others. The system stays consistent but gives up availability
That is the CAP theorem: when a Partition happens, choose Consistency or Availability. You cannot dodge the choice, because partitions are not optional, the network decides when they happen.
Banking systems choose C: better to show an error than a wrong balance. Social feeds, DNS, and shopping carts usually choose A: better slightly stale than down. Real systems choose per feature, not per company.
What the shopping cart design chose
Availability over consistency, with a merge step to converge afterward.
Both halves keep answering, so the cart is available through the partition at the cost of temporarily divergent copies. A user shopping during a network split never sees an error, which is the business outcome the design was chosen for.
The merge, meaning the union of both carts, restores agreement once the partition heals. Union is the right operation because it errs toward keeping items, and an extra item in a cart is a smaller harm than a missing one.
The worst case was judged acceptable, and it is worth stating exactly. A removed item can reappear, because one half saw the removal and the other did not, and the union brings it back.
That trade is a product decision rather than a technical one. Amazon reasoned that a reappearing item costs a moment of confusion while an unavailable cart costs a sale, and the arithmetic favored availability.
This is the classic Amazon Dynamo example, and it is availability-first design done consciously. The lesson to take is the word consciously, since the failure mode was named, evaluated, and accepted rather than discovered in production.
Classifying a system that rejects writes during a partition
The answer is CP.
It is consistent under partitions, at the price of availability. Rejecting the write is the system refusing to proceed without confirmation, which sacrifices availability precisely to preserve a single agreed-upon balance.
The other choice, AP, would keep accepting writes on both sides and reconcile later. That is fine for carts and likes and not for money, since two sides independently approving withdrawals against the same balance creates money that does not exist.
Note what CP costs in practice, because it is not free. An error shown to a user during a partition is a failed transaction, a support ticket, and possibly a lost customer, and choosing CP means deciding that outcome beats a wrong balance.
Interviewers love the question of which parts of your design are CP and which are AP, and per-feature answers are the strong ones. The same product usually wants CP for payments and AP for its feed, so a single-letter answer for a whole system is a weaker answer.
The next lesson is about how a CP system actually decides. Refusing to answer alone is easy, and agreeing with a majority of peers is the interesting part, and that is what quorums provide.