Memory & Storage · lesson 2 of 4 · about 15 min
CPU cache, RAM, SSD, spinning disk, the cloud: why each exists, why some are fast and others slow, what vanishes when the power goes, and how to choose. Tap any for a plain-language card.
The pain
Your page needs some data. Where that data lives decides how long you wait. A single read looks harmless, but real pages make dozens of them.
Drag the slider to change how many reads one page load needs. Bars are on a log scale: each step right is ten times slower. Anything past the gold line is a delay the user feels.
Then flip to : pretend one nanosecond lasts one second.
RAM and the hard disk are both just “storage” in your head. On this clock one is minutes and the other is months.
20 reads from RAM take 2 µs. The same 20 reads from a spinning disk take 200 ms, long enough for the user to notice.
The ladder
Computers do not have one kind of memory. They stack several, each trading speed against size and price. The top is tiny, blazing fast and costly per byte. The bottom is huge, cheap and slow.
Tap a floor to read its numbers. Each floor is a for the one below it: the CPU asks L1 first, then L2, L3, RAM, and only then goes down to disk.
Now press Pull the plug and see which floors .
Wider floor means more capacity. Notice how the ones that vanish on power loss are exactly the fast ones.
Each bit is a microscopic capacitor that leaks, so the chip rewrites it thousands of times a second. Fast and big, but it forgets everything the moment power drops.
Tap any floor for its latency, size, price and whether it survives a power cut.
Why caches work
The CPU never fetches a single number. It fetches a : 64 bytes, here shown as one row of 8 numbers. Read one, and its seven neighbours arrive with it.
That helps only if your next read is a neighbour (). Sum this grid row by row, then column by column, and watch the cache.
Teal is a hit (about 1 ns), coral a miss (a trip to RAM, about 100 ns). The cache here has only a few slots, like the real L1.
Same numbers, same sum, same code size. Only the order changed, and the column walk can be around 100 times slower per read.
CPU cache (4 lines)
for r in 0..8:
for c in 0..8:
total += grid[r][c] # along a rowPick an order and press Run.
Speed costs money
You have 2 TB of data on SSD. RAM is about 63 times pricier per GB. How much should you buy?
It depends on skew: do all items get read equally, or do a few popular ones get most of the traffic? Real traffic is skewed (a few viral videos, a few best-selling products).
Watch the and the average read time as you move both sliders.
Try 64 GB and push skew to 95%: about 3% of the data in RAM serves roughly 84% of reads. Then drag skew to 0 and watch the benefit collapse.
Where reads are served from
54% hit ratio, but the 46% of misses still dominate: average 46.5 µs. Add RAM or find what is hot.
Which tier?
Seven real data problems, four places to put them: RAM, SSD, HDD or a cold archive. Pick one for each. The reasoning is the part interviewers want to hear.
Ask three questions every time: how often is it read, how fast must it answer, and can I afford to lose it?
Login session tokens, checked on every single request
Tip: press 1 to 4 to answer, Enter for the next one.
Who lives by this
The numbers on this page shape the architecture of every large system. A few places where it shows.
L1 1 ns → RAM 100 ns → SSD 100 µs → disk 10 ms
The "latency numbers every programmer should know" list was popularised by Google's Jeff Dean in talks around 2009. Hardware got faster since, but the ratios between the levels barely moved, and the ratios are what you design around.
web server → memcache (RAM) → MySQL (disk)
Facebook described running huge fleets of memcached servers in front of MySQL so most reads never touch a disk at all. The paper is called "Scaling Memcache at Facebook" (NSDI 2013).
app → Redis (RAM) → answer in well under a millisecond
Redis keeps its data set in memory, which is why teams use it for sessions, leaderboards and rate limits. It can also write to disk in the background so a restart does not wipe everything.
popular titles → flash · rarely watched titles → HDD
Netflix places its own caching servers inside internet providers. Their storage mixes flash and spinning disks, because a few titles are watched constantly and the long tail is cheap to keep on disk.
Interview angle
Four questions, each worth 20 XP. They test whether you know the ratios, not just the names.
Quote ratios, not adjectives. “SSD is about a thousand times slower than RAM” beats “SSD is slower”.
A RAM read takes about 100 ns and an SSD random read about 100 µs. Roughly how many times slower is the SSD?
Tip: press 1 to 3 to answer, Enter for the next one.
Break It
Your server has 16 GB of RAM. The data your app actively uses, its , keeps growing. While it fits, every read is a fast RAM hit.
The moment it does not fit, the OS starts : pushing pages to disk and faulting them back. Choose where the swap lives, then grow the data.
It does not degrade gently. Past the line you hit a cliff, and the cliff is taller on a spinning disk.
Press the button to grow the working set from 4 GB to 64 GB.