Run a mock round

A stampede (thundering herd) happens when a popular key expires or is cold, and many concurrent requests all miss and hit the database at once, potentially overwhelming it.

Preventions:

• Locking or single-flight: the first request loads the value while others wait for it or return a short-lived stale value. Libraries and singleflight helpers implement this.
• Probabilistic early expiration: refresh the key slightly before it expires, with jitter, so requests do not converge on the same instant.
• Stale-while-revalidate: serve the stale value immediately and refresh in the background.
• TTL jitter: add randomness to expiry so many keys do not expire together (avoiding an avalanche).
• Request coalescing at the application or cache layer.

const lock = await mutex.acquire(key);
if (lock) { value = await loadAndSet(key); lock.release(); }
else { value = await cache.getStale(key); }

Also warm critical keys on deploy and after restarts, and consider serving a degraded but useful response rather than blocking every request.

Invalidation is the hardest part of caching. Options, often combined:

• TTL expiry: every entry has a lifetime; simple and self-healing, but data can be stale until it expires. Use short TTLs for volatile data and longer for stable data.
• Explicit delete or update on write: after committing to the database, delete the cache key so the next read reloads it. Simpler and race-safer than updating in place.
• Versioned keys: include a version or content hash in the key so old entries become unreachable, then let them expire.
• Event-driven invalidation: publish a change event and have caches evict affected keys or tags.
• Cache tags or surrogate keys: group entries so a single purge clears a whole logical set.

await db.update(...);
await cache.del(`user:${id}`);

Prefer delete-on-write over update-on-write to avoid inconsistent overlapping writes, and always keep a TTL as a backstop in case an invalidation message is lost. Measure hit rate and staleness; invalidation bugs are subtle and usually appear under concurrency.

• Write-through: write to the cache and the database in the same operation, synchronously. The cache stays fresh and reads are fast, but writes are slower and every write also populates the cache, which can evict hot data.
• Write-behind (write-back): write to the cache and acknowledge, then flush to the database asynchronously. Writes are very fast and can be batched, but a crash before flush loses data, so it is used where durability can be relaxed.
• Write-around: write directly to the database and bypass the cache. This avoids polluting the cache with data that may not be read, but the next read pays a miss and the cache can serve stale values until TTL expiry.

write-through: app -> cache -> db (sync)
write-behind:  app -> cache ~~> db (async)
write-around:  app -> db (cache bypassed)

Most web systems use cache-aside reads plus write-through or write-around writes, with TTLs as a safety net. Choose based on durability requirements, write volume and how soon data must be visible.

A content delivery network caches content on edge servers close to users. It serves static assets such as images, CSS, JavaScript and fonts, and can cache dynamic responses and API responses when headers permit.

Benefits: lower latency from geographic proximity, offloaded origin traffic, absorption of traffic spikes and DDoS mitigation, plus TLS termination at the edge.

Cache-Control: public, max-age=31536000, immutable
Cache-Control: s-maxage=60, stale-while-revalidate=30

Control behavior with Cache-Control (max-age for browsers, s-maxage for shared caches, private, no-store), ETag for revalidation, and Vary to prevent serving one user's response to another. Invalidate by purging URLs or using surrogate keys. A crucial warning: never cache authenticated or personalized responses in a shared cache unless the cache key includes the identity, or you will leak data across users.

Perfect consistency is not achievable without giving up caching; aim for bounded staleness and correct behavior.

Practical rules:

• On write, commit to the database first, then delete the cache key. Deleting is safer than updating because concurrent writers can otherwise interleave stale values.
• Use a TTL as a backstop so any missed invalidation self-heals.
• For read-modify-write races, use a short delay before the second delete (double delete) or a versioned key so stale writes cannot overwrite newer data.
• Consider write-through when reads must immediately reflect writes.
• For cross-service data, publish change events and let interested caches evict.

1. UPDATE db
2. DEL cache:user:42
3. next read repopulates from db

A known race: reader misses, reads old DB value, writer updates DB and deletes cache, then the reader writes its stale value. Versioned keys or delayed double delete mitigate it. Document the accepted staleness window and test under concurrency.

Eviction decides what to remove when the cache is full or memory is constrained:

• LRU (least recently used): evicts the item not accessed for the longest time. A solid default for general workloads.
• LFU (least frequently used): evicts the item accessed least often. Better when popularity is stable, but slow to adapt when access patterns change and it can retain stale hot items.
• FIFO: evicts the oldest inserted item regardless of use. Simple but ignores access patterns.
• Random: cheap and surprisingly effective under some workloads, with no bookkeeping.
• TTL-based: expire by age, often combined with another policy.

redis: maxmemory-policy allkeys-lru | allkeys-lfu | volatile-ttl

Redis exposes these via maxmemory-policy and evicts when maxmemory is reached. Choose by access pattern: LRU for recency-heavy traffic, LFU for skewed stable popularity, and TTL for data with natural freshness. Whatever you pick, expect misses: the application must handle a cache miss correctly and cheaply.

Three classic failure modes, often confused:

• Penetration: requests for keys that do not exist always miss the cache and hit the database. Attackers can exploit this with random ids. Mitigate by caching null or empty results for a short TTL, using a Bloom filter to reject impossible keys, or validating input.
• Breakdown: a single very hot key expires and many requests hit the database simultaneously. Mitigate with a mutex, logical expiry, or never-expiring hot keys refreshed in the background.
• Avalanche: many keys expire at the same moment, or the cache restarts and is empty, causing a flood to the backend. Mitigate with randomized TTLs, warming the cache, and using a highly available clustered cache.

penetration: non-existent key -> always misses
breakdown:   hot key expires   -> stampede
avalanche:   mass expiry/restart -> flood

Add circuit breakers and load shedding on the database so a cache incident degrades gracefully rather than taking the whole system down. Monitor miss rate and backend load together to detect these patterns.

Cache-aside (lazy loading): the application checks the cache first. On a miss it reads the database, then writes the value into the cache and returns it. The application owns all cache interaction.

let v = await cache.get(key);
if (!v) {
  v = await db.get(key);
  await cache.set(key, v, ttl);
}
return v;

Read-through: the application always asks the cache, and the cache itself knows how to load from the origin on a miss. It is simpler for callers and often provided by a library or proxy, but hides the latency and error behavior of the loader.

Both leave the write path separate, so they are usually paired with a write policy. Cache-aside is the most common and most flexible, at the cost of duplicated loading logic and more code paths to maintain. Read-through centralizes loading but requires cache support for it and can mask backend problems.

A hot key is accessed so frequently that a single cache node or partition becomes a bottleneck. Keys are distributed by consistent hashing, but one key always maps to one shard, so it cannot be spread automatically.

Mitigations:

• Add a small local (in-process) cache for the hottest keys to absorb most reads before they reach the shared cache.
• Replicate the hot key across several shards by appending a random suffix and writing to N copies, then read from a random copy.
• Use read replicas and client-side load balancing to spread read traffic.
• Precompute and refresh the value in the background so it never expires under load.
• Shard the underlying data differently if a single logical key is genuinely too large.

hotkey:product:99:{0..9}  -> spread across 10 slots

Monitor per-key and per-shard metrics, not just cluster averages, because a hot key hides behind healthy aggregates. Also watch for large values and hot partitions in stream processing, which follow the same pattern.