Eviction: choosing what to forget
A cache lives in RAM, and RAM is the most expensive storage you run: a cache holds gigabytes while the database holds terabytes. So a real cache is always too small for the data, and once it is full, storing anything new means deleting something old. That deletion is eviction, and the rule for picking the victim is the eviction policy. Every production Redis runs with a policy configured, and a bad choice quietly wrecks the hit rate you learned to treasure in lesson 3-1, with no error anywhere to see.
The default policy nearly everywhere is LRU, least recently used: evict the entry that has gone unread the longest. The justification is a measured property of real traffic, not a hunch: data accessed a moment ago is far more likely to be accessed again soon (engineers call this temporal locality), so the entry idle longest is the safest one to sacrifice.
The pleasant consequence: an LRU cache automatically keeps whatever is currently popular, with no configuration telling it what popular means.
Code exercise · python
Run this LRU cache with room for only 2 entries. On a hit, the entry is re-inserted, moving it to the back of the line (most recently used). On a miss with a full cache, the front of the line (least recently used) is evicted. Python dicts remember insertion order, which keeps the simulation short.
Code exercise · python
Your turn: run the request list through an LRU cache with capacity 3 and count hits, misses, and evictions. On a hit, re-insert the key to mark it recently used (cache[key] = cache.pop(key)). On a miss with a full cache, evict the oldest entry (next(iter(cache))) before inserting. Print the three totals exactly as shown.
When eviction is the bug
Watch one number alongside hit rate: the eviction rate. If entries are routinely evicted before anyone reads them a second time, the cache is smaller than its working set, the set of data traffic actually touches in a window of time. The symptom is a hit rate that stays low no matter how correct the code is, exactly like the 1-hit run you just simulated. The fixes, in order of preference: cache smaller values (store IDs, not whole rendered objects), stop caching low-value data, or pay for more RAM.
One more production trap ties this unit together. If a popular, slow-to-compute entry is evicted (or expires), every concurrent request misses at once and stampedes the database recomputing the same value. Teams call it a cache stampede, and the standard cure is to let only one request recompute while the others briefly wait for its result instead of piling on.
Quiz
A cache with room for 10,000 entries serves traffic that touches about 1 million distinct keys per hour, spread evenly. Why is the hit rate terrible?