Networking & Traffic · lesson 4 of 7 · about 28 min
Why a CDN exists (distance, not bandwidth), how much of your traffic it can keep off the origin, how cache keys quietly destroy hit ratios, how to keep content fresh, what happens when a hot object expires, and what goes wrong when a personal page is cached. Tap any for a plain-language card.
Why a CDN
The table uses real great-circle distances between six cities, a fibre speed of 200 km per millisecond, a route that is 1.5× longer than the arc (illustrative) and 5 ms of equipment. “Direct” is 4 round trips straight to the origin; “with CDN” mixes edge hits (a nearby edge, about 50 km away) with misses that also cross to the origin.
Put the origin in New York and look at Sydney. Then raise the hit ratio from 50% to 99%.
With the origin in New York, the farthest user waits 979 ms for 4 round trips. With a 90% edge hit ratio they wait about 72 ms: hits are served from 6 ms away, and only the misses pay the long trip.
Distances are great-circle from city coordinates; fibre speed 200 km/ms, route inflation 1.5× and equipment 5 ms are illustrative round numbers.
Offload
The numbers come from a real Zipf distribution over 100,000 objects: the share of requests served by the hottest fraction of objects. Pick how much each edge can hold, whether a shield tier exists, and your total traffic.
At 1% edge capacity with skew 1.0, the edge alone catches about 60%. Switch on a 5% shield and compare the origin load.
Of 50K requests a second, the edges answer 62%; the origin sees 19K a second (38.1%). Too much is getting through: make objects cacheable, lengthen TTLs, or add a shield.
Computed from Zipf weights over 100,000 objects, assuming each cache keeps the hottest objects. Real caches only approximate that.
The cache key
The fix is to normalise the key: drop parameters that do not change the content, ignore cookies on public pages, and use Vary only for headers that really change the response (such as Accept-Encoding). The simulation sends 20,000 requests for 400 pages, each with one of 12 tracking values and one of 25 cookie values, to an unbounded cache under three key rules.
Start with the whole URL and cookie, then drop the tracking parameters, and then the cookie.
every ?utm_source=… and every cookie value is a different key
With the whole URL and cookie in the key, 400 pages become 17,111 cache entries and only 14% of requests hit. The CDN is there, but it is barely helping.
20,000 seeded requests (skewed toward popular pages). Hit ratio of an unbounded cache = 1 − distinct keys ÷ requests.
Freshness
The graph shows what share of users still see the old version after a change at time zero. The “look at” slider reads it off at a chosen moment.
Pick a 1-hour TTL and look 2 minutes after the change: most users still see the old file. Then switch to versioned names.
With a 1 h TTL, 97% of users still see the old version 2 min after the change, and it takes the whole 1 h to clear. Shortening the TTL fixes that and multiplies the load on the origin.
TTL: caches expire uniformly, so stale share = 1 − t ÷ TTL. Purge delay and TTL values are your inputs; the 300-edge fleet is illustrative.
When a hot object expires
Two defences. Request collapsing (also called coalescing): the edge lets one request fetch from the origin and makes the others wait for that answer. A shield tier: all edges ask one shared cache instead of the origin, so the origin sees at most one fetch, not one per edge. Together they turn thousands of requests into one.
Start with neither and see how many requests hit the origin at one expiry. Then add each, then both.
While one fetch is in flight (400 ms), each of 100 edges receives about 8 more requests for the same object. At this setting the origin sees 800 requests for one object instead of 800: nothing is protecting it.
Requests that arrive during one origin fetch = rate per edge × fetch time; collapsing makes it 1 per edge, a shield makes it 1 in total.
What should the CDN cache?
The right cache policy depends on how fast the content changes and who it is for. Each answer is worth 20 XP.
Ask: does it change? and is it the same for everyone?
A JavaScript bundle with a content hash in its file name (app.4f9c2.js).
Tip: press 1 to 3 to answer, Enter for the next one.
Who lives by this
Four real networks and what they have published about how they work.
user → nearest Akamai server → origin only on a miss
Akamai, founded in 1998 by MIT researchers including authors of the consistent hashing paper, built the first large CDN: thousands of servers placed inside ISP networks so that content is served from a few network hops away rather than from the publisher's data centre.
one IP address announced from every city → nearest data centre
Cloudflare announces the same IP addresses from its data centres in hundreds of cities (its site lists well over 300), so internet routing sends each visitor to a nearby one, and a failed site simply stops announcing. The same network also caches, terminates TLS and absorbs attacks.
Netflix appliance in the ISP → subscriber
Netflix runs its own CDN, Open Connect, and offers ISPs free appliances to install in their networks. Popular titles are pre-loaded overnight during quiet hours, so most streaming traffic is served from inside the viewer's own ISP and never crosses the wider internet.
publish → API purge by key → all edges
Fastly's documentation describes instant purging by URL or by surrogate key (a tag you attach to many objects) and advertises purges taking effect worldwide in around 150 milliseconds, which lets sites cache news and e-commerce pages for a long time and still update them at once.
Interview angle
Six questions, each worth 20 XP. CDNs appear in almost every design that serves images, video or a global audience.
Say what is cached, for how long, how it is invalidated, and what is never cached.
Why does a CDN make a site faster for distant users?
Tip: press 1 to 3 to answer, Enter for the next one.
Break it
The defences are all about telling the cache what it may share: Cache-Control: private or no-store on anything personal, keys that include the session where caching is allowed at all, Vary for headers that change the response, and never letting a CDN rule cache by file extension alone. The simulation sends 1,000 requests for /account from 200 different users through a shared cache under three policies and counts the responses that belong to someone else.
Start with URL only and look at the leak. Then add the cookie to the key, then switch to private.
With the URL as the only key, the first visitor's page was stored and 995 of 1,000 requests got another user's account: names, addresses, whatever the page holds. The cache hit ratio looks wonderful (100%) and the site is leaking data.
1,000 seeded requests from 200 users. A response leaks when the cached copy was created for a different user.