Lesson 31 / 32

Design a Distributed Cache

Building a Redis-like sharded, replicated key-value cache.

Requirements

GET/SET/DELETE by key with sub-millisecond latency, capacity far beyond one machine's RAM, and survives individual node failure without losing everything.

Sharding by consistent hashing

Distribute keys across nodes with consistent hashing so the cluster can grow without a mass rehash. Clients (or a proxy layer) hash the key locally to know which node to talk to directly, in one hop.

Eviction as a bouncer

Memory is finite, so each node runs an eviction policy (usually LRU) — a bouncer letting new guests in by turning away whoever's been standing around longest without being touched. Set per-key TTLs on top so nothing lingers forever.

Quick check: Why replicate each cache shard rather than relying on the origin database on a node failure?

  • Replication is required by law
  • Without it, losing a shard sends a flood of misses straight to the database
  • Replication makes writes faster
Answer

Without it, losing a shard sends a flood of misses straight to the database — A cold shard means every request for its keys becomes a miss, hammering the database until it's warm again — replicas avoid that cliff.