Caching & Databases · lesson 1 of 8 · about 22 min
Why a small fast copy of the data saves a slow database, how to measure whether it works, what to throw out when it fills, how writes keep it honest, and how it fails. Tap any for a plain-language card.
The pain
A shop's product pages get 90 visits. Most visits are for the same few products (the phone, the laptop). Each visit asks the database, which reads from disk and takes ~25 ms.
Run the visits without a . Count how many times the database was asked. Then switch the cache on and run the exact same visits.
Same visitors, same products. The only change: the answer to “what is the phone?” is remembered for next time.
Warmer colour = more visits. Press Send visits.
The maths
The share of reads the cache answers is the . Everything else falls through to the database. So at 10,000 reads per second:
database load = reads × (1 − hit ratio)
That is why teams chase 99% and 99.9%: each extra “nine” is ten times less work for the database, while the average latency barely changes. Drag the slider, and also make the database slower.
Going from 90% to 99% sounds like +9%. For the database it is a tenfold cut.
The database handles 1,000 reads per second. Push toward 99% and watch it drop.
Should I cache this?
Cache the wrong data and you add a layer that is mostly misses, can show old values, and gives you one more thing to break. Answer five questions about your data and see what the score says. Or load a real example.
Interviewers love this: “would you cache that?” The honest answer is sometimes “no, here is why”.
Strong case. Lots of repeats, tolerant of age, worth the saving.
What to throw out
A cache is smaller than the data behind it, so when it is full, each new key forces an old one out. The rule that chooses is the :
LRU drops the key unused the longest. LFU drops the key used the fewest times. FIFO drops the oldest arrival, ignoring use.
No policy wins every time. Pick a traffic pattern, then watch how each policy copes. The bars below always compare all three on the same requests.
Try the Big scan with LRU. Then try Hot + one-hit wonders with LFU. The right policy depends on the traffic.
A few keys get most requests (like real traffic).
Skewed traffic is what caches love. Best here: LFU at 50%. All three do decently because the hot keys keep coming back.
Who updates the cache?
Reading is easy: look in the cache, fall back to the database. The hard part is writing. When data changes, do you update the cache, the database, or both, and in what order?
Pick a pattern, then use the buttons: write a new value, read it back, make another service edit the database, expire the entry, or crash the cache. Look for the one that loses data and the ones that go stale.
There is no free lunch: faster writes cost safety, and fresher reads cost slower writes.
The app manages the cache itself (lazy loading)
The value is 1 in the database; the cache is empty. Try a write, then a read.
Caches everywhere
You met caches inside the CPU. The same idea repeats at every layer of a web system: the browser, a , the app's own memory, a shared Redis, the database's , and finally the disk.
Each layer answers what it can and passes the rest down. Out of 1,000 requests, see how many are caught at each layer. Switch layers on and off.
Every layer you add catches a share and spares everything below it. But each is also another copy that can go stale.
200 requests reach the database, 20 touch the disk. Add more layers above and watch those numbers drop.
Who lives by this
Almost every large site is a database hidden behind several layers of caches. Four examples.
web → memcache (regional pools) → MySQL
Facebook's memcache paper (NSDI 2013) describes leases that stop a thundering herd when a hot key expires, and gutter pools that absorb load when cache servers fail. The cache is a core part of the system, not an add-on.
app → EVCache (memcached, replicated across zones) → source of truth
Netflix built EVCache on memcached, with copies spread across availability zones, so losing a zone does not mean a cold cache and a database stampede.
tweet → fan-out into per-user timelines held in Redis
Twitter has described keeping each user's home timeline in an in-memory Redis cache so that opening the app is a quick lookup, not a heavy database query.
user → nearby edge server → origin only on a miss
A CDN is a cache with hundreds of locations. Static files and even API responses are served from a city near the user. Origin servers only see the misses.
Interview angle
Five questions, each worth 20 XP. Cache questions are about trade-offs: freshness, loss, and what happens when the cache fails.
Always say what happens on a miss and what happens when the cache is empty or gone. That is where designs break.
A cache has a 99% hit ratio and the database used to handle 10,000 reads per second. Roughly how many reach the database now?
Tip: press 1 to 3 to answer, Enter for the next one.
Break It
500 popular keys were all cached at the same moment with the same 30-second . Traffic is 2,000 reads per second. The database can serve about 600 per second.
At second 30 every key expires together. Every request that misses goes to the database, which slows down, which makes refreshes slower, which causes more misses. That is a .
Run it with a plain TTL, then add the fixes one by one.
Jitter spreads the expiry times. A lock lets only one request per key refresh while others wait or get the old value.
Press Run. Red bars are seconds when the database is over its limit.