Course outline · 0% complete

0/29 lessons0%

Course overview →

Quorums and consensus, gently

lesson 7-2 · ~11 min · 21/29

Consistency by counting

There is a middle path between one leader does everything and replicas drift freely: make reads and writes each check in with several replicas and count responses.

With N replicas, pick two numbers:

  • W: a write is confirmed only after W replicas store it
  • R: a read asks R replicas and takes the newest answer (using version numbers, as in lesson 7-1)

The magic rule: if W + R > N, any R replicas you read must overlap any W replicas that stored the latest write, so at least one fresh copy is always in your read set. That is a quorum.

With N = 3, the popular setting W = 2, R = 2 gives 2 + 2 > 3: strong-ish reads that also survive one dead replica. Setting W = 1, R = 1 is fastest but 1 + 1 ≤ 3, so stale reads are possible. Consistency becomes a dial, not a switch.

N1N2N3W = 2 nodes must store the writeR = 2 nodes must answer the readoverlapW + R > N, here 2 + 2 > 3
With three replicas, writing to two and reading from two forces at least one node to appear in both sets, so a read cannot miss the write.

A quorum checker

The classic N=3, W=2, R=2 configuration, tested against the rule.

N, W, R = 3, 2, 2
print("copies:", N, "| write waits for:", W, "| read asks:", R)
if W + R > N:
    print("W + R > N: every read overlaps the latest write")
else:
    print("W + R <= N: a read can miss the latest write")

Output

copies: 3 | write waits for: 2 | read asks: 2
W + R > N: every read overlaps the latest write

The overlap argument is worth walking through concretely. With three replicas, any two that stored the write and any two that answer the read must share at least one member, because two disjoint pairs would need four replicas.

That shared member is the one holding the fresh value, and the version numbers from lesson 7-1 are what let the reader recognize it. Overlap alone is not enough without a way to tell which of the two answers is newer.

With W = 3 and R = 1 you still have a quorum, this time with slow writes and fast reads. That configuration suits data written rarely and read constantly, which is a real pattern for configuration and feature flags.

NWRW+R>NCharacter
322yesbalanced, survives one failure
331yesfast reads, writes need every node
313yesfast writes, reads need every node
311nofastest, stale reads possible

Note that W = 3 gives up failure tolerance for writes, since one dead replica means no write can be confirmed. W = 2 with N = 3 is popular precisely because it satisfies the rule and survives a failure, which is why it is the default in Cassandra and DynamoDB.

The speed-first configuration

N=3 with W=1 and R=1, checked against the same rule.

N, W, R = 3, 1, 1
print("copies:", N, "| write waits for:", W, "| read asks:", R)
if W + R > N:
    print("W + R > N: every read overlaps the latest write")
else:
    print("W + R <= N: a read can miss the latest write")

Output

copies: 3 | write waits for: 1 | read asks: 1
W + R <= N: a read can miss the latest write

1 + 1 = 2, which is not greater than 3, so the guarantee is gone. A write stored on replica A only and a read answered by replica B produces a stale answer, and both operations were confirmed as successful.

This is the fastest possible configuration, and it is also the eventually consistent one from lesson 7-1. Nothing is broken here, since W=1, R=1 is a deliberate choice for data where speed matters more than freshness.

Note that stale is possible rather than certain. Most reads with this setting still land on an up-to-date replica, which is what makes the failure mode intermittent and easy to miss in testing.

The point of the two blocks together is that consistency is a dial rather than a switch. The same cluster, the same data, and the same code can be strongly consistent or eventually consistent depending on two numbers in a client configuration.

That is also why these settings are usually per-query rather than per-cluster. A session lookup and a balance check can run against the same replicas with different R values, which is the per-feature reasoning from lesson 7-1 expressed as configuration.

Consensus, in one paragraph

One question remains from unit 4: when the leader dies, who picks the new leader? If two nodes each declare themselves leader (a split brain), both accept writes and the data forks.

Consensus protocols like Raft and Paxos solve this with the same counting idea: a node may only become leader after winning votes from a majority of the cluster. Two competing leaders would each need a majority, and two majorities of one cluster always share at least one member, who only votes once. So split brain is arithmetically impossible.

That is why clusters run an odd number of coordinators (3 or 5): majorities stay well defined and one node can die without freezing elections. The deep details fill textbooks, but majority voting prevents split brain is the sentence to carry into interviews.

Which side of a 3-2 partition elects a leader

Only the group of 3, since 3 is a majority of 5.

A majority of 5 requires 3 votes. The group of 3 has them, elects a leader, and continues accepting writes.

The group of 2 cannot reach 3 votes, so it refuses writes until the partition heals, then catches up by replaying what it missed. It refuses even though its nodes are healthy and reachable from each other, which is the part that feels wrong and is exactly right.

Notice this is a CP choice from lesson 7-1, since the minority side gives up availability to keep the data consistent. Clients talking to those two nodes see errors, which is the cost paid to guarantee that no two leaders exist.

The arithmetic is what makes split brain impossible rather than merely unlikely. Two majorities of the same cluster must share a member, and that member votes once, so a second leader cannot be elected no matter how the network fails.

This is also why cluster sizes are odd. A 4-node cluster split 2-2 has no majority on either side, so the whole cluster stops accepting writes, and adding a fifth node buys real availability while adding a fourth to three buys none.

The smallest W for N=5 and R=2

W + R > N requires W + 2 > 5, so W = 4.

Any 2 replicas you read must then overlap the 4 that stored the newest write in at least one member. Four out of five leaves only one replica possibly missing the value, so a pair of readers cannot both land on it.

The price shows the three-way trade. Reads stay fast because they ask only 2 replicas, and writes wait for 4 of 5 machines, so the write is as slow as the fourth-slowest replica.

The availability consequence is the sharper cost. If two replicas are down, writes stop entirely, and a configuration that cannot tolerate two failures out of five is fragile for a cluster that size.

NRMinimum WWrite failure tolerance
515none
5241 node
5332 nodes
5423 nodes

Read that table for where the balance sits, and W = R = 3 is the usual answer. It satisfies the rule, tolerates two failures on both paths, and treats reads and writes symmetrically.

Tuning W and R against N is how these systems dial between read speed, write speed, and safety. Being able to derive the minimum W from the rule, rather than recalling a default, is what the question is really testing.