Invalidation: the hard part

Stale data

Ada updates her display name. The database now says the new name, but the cache still holds the old one, and cache-aside will happily serve it. That cached copy is stale, and deciding when to refresh or delete cached copies is called cache invalidation. It is famously one of the hardest problems in practice because every option trades freshness against cost.

The two standard tools:

1. TTL (time to live). Every cache entry expires automatically after a fixed time, say 60 seconds. Simple and robust. Worst case, users see data that is 60 seconds old
2. Explicit invalidation. When your code writes to the database, it also deletes the matching cache key. The next read misses and fetches the fresh value

Most real systems use both: explicit invalidation for correctness, plus a TTL as a safety net for the deletes your code forgets.

A TTL simulation

Each cache entry stores its value plus an expiry time, and a fake clock is passed in so runs are deterministic.

TTL = 30
cache = {}

def get(key, now):
    if key in cache:
        value, expires_at = cache[key]
        if now < expires_at:
            print("t=" + str(now) + "s cache hit:", value)
            return value
        print("t=" + str(now) + "s entry expired")
    print("t=" + str(now) + "s reading database, caching for " + str(TTL) + "s")
    cache[key] = ("Ada", now + TTL)
    return "Ada"

get("user:1", 0)
get("user:1", 10)
get("user:1", 45)

Output

t=0s reading database, caching for 30s
t=10s cache hit: Ada
t=45s entry expired
t=45s reading database, caching for 30s

The entry cached at t=0 expires at t=30, so the read at t=45 misses. Expiry is stored as an absolute deadline rather than a countdown, which is what lets a single comparison decide.

if now < expires_at is the whole TTL mechanism, and note that expiry is checked on read rather than enforced by a timer. Nothing sweeps the cache at t=30, and the entry simply stops being usable, which is how Redis behaves too.

The t=45 read produces two lines, meaning the expiry notice and then the database read. An expired entry takes the same path as a missing one, so there is no separate refresh code to write.

Passing now as a parameter instead of calling the clock is the injectable-dependency habit, and it is why this output is reproducible. Testing time-based behavior against the real clock means either waiting 30 seconds or getting flaky results.

Note the guarantee a TTL gives, and it is a bound rather than a promise of freshness. Data can be up to 30 seconds stale here, and it can also be fresh, so the TTL sets a worst case rather than an expected case.

What the next read does after a delete-on-write

A cache miss, so it reads the fresh name from the database and refills the cache.

Deleting the key forces the next read down the cache-aside miss path from lesson 3-2, which fetches the fresh value and re-caches it. No special refresh logic is needed, because the miss path already does exactly the right thing.

That is why delete-on-write works, since it converts staleness into one cheap miss. One request pays 10 ms extra and every request after it is both fast and correct.

Deleting rather than updating the cache is the deliberate choice here. Writing the new value into the cache directly seems more efficient and introduces a race, because two concurrent writers can leave the cache holding the older of the two values, while deleting leaves the database as the single source of the answer.

The TTL stays on as the safety net for any write path that forgets the delete. New code, a background job, or a manual database fix can all change data without going through your delete, and the TTL bounds how long that mistake lasts.

Choosing a TTL, and what caching buys

How long should a TTL be? Ask: how stale can this data be before someone is harmed?

• Account balance: seconds, or do not cache it at all
• A profile page: a minute is fine
• A country list: hours

The reward for getting this right is measured in latency. Average read latency with a cache is a weighted blend:

average = hit_rate × cache_latency + miss_rate × database_latency

With a 1 ms cache, a 10 ms database, and an 80% hit rate, reads average 2.8 ms instead of 10 ms. You will compute this yourself now, and this exact formula shows up in system design interviews.

Computing average read latency

The weighted blend, plus the speedup it represents.

hit_rate = 0.8
cache_ms = 1
db_ms = 10
average_ms = hit_rate * cache_ms + (1 - hit_rate) * db_ms
print("Average read latency (ms):", round(average_ms, 1))
print("Speedup vs no cache:", str(round(db_ms / average_ms, 1)) + "x")

Output

Average read latency (ms): 2.8
Speedup vs no cache: 3.6x

The 2.8 ms is 0.8 ms of cache time plus 2.0 ms of database time, and the split is worth noticing. Even at an 80% hit rate, most of the average latency comes from the 20% of requests that miss.

That is the general shape of caching math, meaning the misses dominate the average. Pushing the hit rate to 95% gives 1.45 ms, and to 99% gives 1.09 ms, so the remaining gains all come from eliminating misses rather than speeding up hits.

db_ms / average_ms expresses the improvement as a multiple, which is the form people quote. Saying 3.6x is more useful in a design discussion than saying 7.2 ms saved, since the multiple holds even if the absolute numbers change.

This exact formula shows up in system design interviews, so it is worth being able to produce on a whiteboard. It is also the honest answer to how much a cache helps, since quoting the 1 ms cache latency alone would describe only the hits.

Worst-case staleness with a 5-minute TTL

Up to 5 minutes, because the stale copy was cached just before the update and survives almost its entire TTL.

The stale entry lives until its TTL expires, and nothing about the avatar update touches it. The database is correct immediately, and the cache is the thing serving the wrong answer.

The timing is what makes this the worst case. Cached a moment before the change means nearly the full TTL remains, and the same update one second before expiry would be visible almost instantly, so the user-visible delay is unpredictable within the window.

That unpredictability is its own problem, and it is why users report caching bugs as intermittent. The same action looks instant sometimes and takes minutes other times, which makes it hard to reproduce and easy to dismiss.

If that is unacceptable, add explicit invalidation by deleting the cache key inside the avatar-update code path, and keep the TTL as the safety net. That combination of delete on write plus TTL backstop is the workhorse of production caching.

Note that the user's own experience matters most here. Seeing your own avatar unchanged after uploading it reads as a broken upload, which is why the write path invalidating immediately is worth the extra line even when other users could wait.