Playbook / General system design / Design a distributed cache / global content delivery layer

Design a distributed cache / global content delivery layer

Expected question

"Design a distributed cache / global CDN layer. How do you cache content at the edge, invalidate consistently, and reduce origin load?"

Variant forms

Same design, different framing:

  • "Design a CDN for static assets serving 10M QPS globally."
  • "How do you cache API responses at the edge without serving stale user-specific data?"
  • "Design cache invalidation when a product price changes worldwide in <60 seconds."
  • "Our origin melted during a viral event — architect multi-tier cache (browser, CDN, origin shield)."
  • "Design consistent hashing for a distributed memcached cluster with minimal remapping on node add."
  • "How do you cache personalized pages for logged-in users safely?"
  • "Design geo-routing to the nearest healthy PoP with failover."

Where this actually gets asked — with two attribution corrections

Mixed sourcing, with two specific corrections worth disclosing rather than smoothing over. Microsoft is the best-sourced of the six for this topic — DesignGurus frames "Design a Distributed Caching Solution" as a Microsoft question referencing Azure Cache for Redis, and a Microsoft-tagged Glassdoor question set includes general distributed-cache framing — moderate confidence, secondary-aggregator sourcing rather than a raw verified quote. A real Google Glassdoor question does exist, but it's "Implement an LRU cache" — that's a coding-round data- structure question, not a distributed-systems-round question, and should not be conflated with this entry's scope. The one solid, specific Glassdoor citation found for "Design a distributed cache" is attributed to Amazon (SDE-3) — not one of this repo's six companies — worth citing as evidence the question circulates in FAANG-style loops broadly, but not claimable as one of the six without further verification. What's unusually strong here is the real-system grounding: Meta's own paper, "Scaling Memcache at Facebook" (Nishtala et al., NSDI 2013), is a Facebook-authored USENIX paper — Meta's TAO paper (USENIX ATC 2013) is also directly relevant for the caching-tier-over-database pattern.

Executive summary

30-second thesis

I'd default a multi-tier cache — process L1, regional Redis/Memcache with consistent hashing, CDN for static — plus write-triggered invalidation, not TTL-only hope.

2-minute answer

Clarify what's cached (API objects vs static assets), staleness tolerance, and multi-region. Default: consistent-hash cluster, read-through on miss, explicit invalidate/tag purge on write, jittered TTLs so popular keys don't expire together. Thundering herd: coalesce misses so one fetch fills for waiters. CDN for edge static; app cache for computed/DB data — don't conflate them. Cross-region: async invalidation bus with a bounded staleness window (H); sync invalidate only when write-path latency allows. Never share negative-cache across authz tenants. If a whole cluster dies, rate-limit origin and warm gradually — cold cutover is how you DDoS yourself.

Quantitative trade-offs

DecisionTrade-off and reversal evidenceEvidence class
TTL-only vs write invalidationTTL is simple; reverse when user read-after-write must see updates inside the TTL window.H
Sync vs async cross-region invalidateSync cuts staleness; reverse when write p99 can't afford the RTT.H
Aggressive CDN 301/long TTL vs freshnessLong TTL saves origin; reverse when content mutates and purge lag becomes product-visible.H

What I'd ask them

  1. What's the max acceptable staleness after a write?
  2. Personalized vs public content — any edge caching of user-specific responses?
  3. Single region or global PoPs day one?

Requirements

Functional

  • Serve frequently-accessed data with much lower latency than hitting the backing database/ origin every time.
  • Support cache invalidation when the underlying data changes, so clients don't see stale data indefinitely.
  • Scale reads far beyond what the backing store alone could sustain.

Non-functional

  • Cache hit rate matters enormously — a low hit rate means most requests still hit the expensive backing store, defeating the cache's purpose.
  • Must handle individual cache-server failure without a full cache-wide outage (a "thundering herd" of requests hitting the origin all at once when a cache node goes down is a real, documented failure mode).
  • For a global/CDN-style layer specifically: content should be served from a location physically close to the requesting user, not always from one central region.

Core entities

  • Cache entry: a key, its cached value, and a TTL (time-to-live) or explicit invalidation marker.
  • Cache cluster/pool: a set of cache servers, with a consistent-hashing scheme determining which server owns which keys.
  • Origin/backing store: the source of truth a cache miss falls back to.
  • Edge location (for the CDN variant): a geographically distributed point-of-presence serving cached content close to users.

API / interface

Auth: service identity for cache admin; CDN purge is dual-controlled in production.

GET /v1/cache/{key}
→ 200 {"value":"...","hit":true,"ttl_remaining_sec":42}
→ 404 {"hit":false}

PUT /v1/cache/{key}
{"value":"...","ttl_sec":300,"tags":["user:42"]} → 200 {"stored":true}

DELETE /v1/cache/{key} → 200 {"deleted":true}
POST /v1/cache/invalidate
{"tags":["user:42"]} → 202 {"invalidation_id":"inv_..."}

POST /v1/cdn/purge
{"urls":["https://cdn.example/a.js"],"approver_id":"u_..."} → 202 {"purge_id":"pg_..."}

GET /v1/cache/stats
→ {"hit_ratio":0.93,"evictions_per_min":120,"replication_lag_ms":5}

Staff+ callout: tag-based invalidation + CDN purge are separate APIs with different blast radii.

Data Flow

Read-through on miss; writes set TTL; tag invalidation and CDN purge are separate, higher-blast-radius paths.

Rendering architecture diagram…

High-level design

Rendering architecture diagram…

Deep dive 1: cache invalidation strategy

ApproachStaleness windowComplexityWhen it's the right call
TTL-only (no explicit invalidation)Bounded by TTL, can serve stale data until expiryLowestData where brief staleness is acceptable
Write-through invalidation (update/invalidate cache on every write)Minimal — cache reflects writes near-immediatelyMediumData where correctness after a write matters, and write volume is manageable
Write-behind / async invalidationSome staleness window while invalidation propagatesMedium-highHigh write-throughput systems where synchronous invalidation would add unacceptable write latency

Trap: proposing TTL-only invalidation as the complete answer without addressing that a write followed by an immediate read (a very common real pattern — a user updates something and expects to see the update reflected right away) can still return stale data for the remainder of the TTL window, unless writes trigger explicit invalidation.

Deep dive 2: the real cross-region/cross-cluster scaling problem — Meta's documented approach

Meta's own "Scaling Memcache at Facebook" paper describes the real problem at a scale most system-design answers never reach: a single cache cluster isn't enough — the real architecture scales across multiple axes: within a cluster (consistent hashing across many memcached servers), across clusters in the same region (a "regional pool" reducing redundant cache misses across clusters), and across regions entirely (replicating cached data closer to users while handling cross-region invalidation consistency, which is a hard problem — an invalidation in one region needs to propagate to others without either being so slow that stale data lingers cross-region, or so synchronous that it adds unacceptable latency to every write). Staff+ goes multi-axis; senior stops at "add a consistent-hashed cluster."

Deep dive 3: the thundering herd problem

When a popular cache entry expires or a cache node fails, many concurrent requests can suddenly all miss the cache simultaneously and hammer the origin at once — potentially overloading it just as it was supposed to be protected by the cache. Real mitigations: a request-coalescing mechanism (the first request that misses fetches from origin and populates the cache; concurrent requests for the same key wait on that result rather than each independently hitting origin), or staggered/jittered TTLs so popular entries don't all expire in the same instant.

What's expected at each level

  • Mid-level: proposes a single cache layer with TTL expiration, without addressing invalidation-after-write consistency or multi-cluster scaling.
  • Senior: identifies write-triggered invalidation as necessary alongside TTL, and proposes consistent hashing for distributing keys across cache servers.
  • Staff+: designs for cross-cluster/cross-region cache scaling explicitly (regional pools, cross-region invalidation propagation), matching the real scale Meta's own published architecture operates at.
  • Principal: additionally addresses the thundering-herd failure mode with a concrete mitigation (request coalescing, jittered TTLs) and can reason about the cost/consistency trade-off of synchronous vs. asynchronous cross-region invalidation explicitly.

Follow-up questions to expect

  • "Whole cache cluster dies?" — Thundering herd at cluster scale. Coalesce/rate-limit in front of origin and warm gradually — never dump 100% traffic onto a cold origin in one cut.
  • "CDN vs Memcache?" — CDN sits at the network edge for mostly-static bytes. App cache sits next to services for computed/DB objects. Related idea, different layer.
  • "Multi-tenant cache poisoning?" — Tenant-prefix keys; never share negative-cache across authz boundaries; purge is a blast-radius event — rate-limit it.
← Design a collaborative document editing systemDesign a notification system →