A cache keeps a copy of data in a faster place so you do not repeat slow work. It is one of the cheapest ways to cut latency and database load. The hard part is deciding how data gets in, how it stays correct and what to throw out when space runs out.
How data gets in and out
| Strategy | How it works | Good for | Watch out for |
|---|---|---|---|
| Cache-aside | The app checks the cache, on a miss reads the database and fills the cache | Read-heavy data, the most common default | A miss is slow. Data can go stale |
| Read-through | The cache itself loads from the database on a miss | Simpler app code | Needs a cache that supports it |
| Write-through | Every write goes to the cache and the database together | Data you read right after writing | Slower writes |
| Write-back | Write to the cache first, save to the database later | Very high write volume | Data can be lost if the cache fails before saving |
| Write-around | Write to the database only, cache on read | Data written once and rarely read | First read is a miss |
Eviction: what to remove when full
- LRU (least recently used). Remove what was used longest ago. A common default. You can build one yourself: see LRU Cache.
- LFU (least frequently used). Remove what is used least often.
- FIFO. Remove the oldest entry, whether or not it is still popular.
- TTL (time to live). Each entry expires after a set time. This also limits how stale data can get.
Problems to plan for
- Stale data. The cache and the database disagree. Use a short TTL, or delete or update the cache entry when you write.
- Cache stampede. A popular key expires and thousands of requests hit the database at once. Fixes: lock so only one request refills it, spread the expiry times, or refresh before it expires.
- Missing keys. Requests for data that does not exist always miss and always reach the database. Cache the "not found" answer for a short time, or check a Bloom filter first.
- Hot keys. One key gets most of the traffic and overloads one cache node. Keep a local copy in each server or split the key.
Where caches live
In the browser, on a CDN near the user, in your application memory, in a shared cache such as Redis, and inside the database itself. Each layer is a trade between speed, size and freshness.
Try these live in the caching lesson, where you can change cache size and policy and watch the hit rate move.