When you scale a system, caching is usually the first lever you pull. It's cheap, it's fast to add, and it buys you time before you have to do the harder work of sharding databases or splitting services.
It's also the most common thing people get wrong in interviews. Not because caching is hard, but because most candidates stop at "I'll add a cache" without saying where it goes, what it stores, or what happens when it's wrong.
This post walks through the parts that actually matter.
Why caching works
Caching trades memory for time. You take an expensive result - a database query, an API call, a rendered page - and keep a copy somewhere faster to read.
The reason it pays off is that most systems are read-heavy. The same handful of items get requested over and over. A user profile, a product page, a config value. If 90 percent of your reads hit a cache that's 100x faster than the database, your average read time drops hard and your database does a fraction of the work.
That's the whole pitch. Skip the slow path when you can.
Where you can put a cache
A cache isn't one thing. It's a place. And you've got several places to choose from, each closer to the user or closer to the data.
- Client-side. The browser or app holds the result. Fastest possible, but you can't invalidate it easily once it's out there.
- CDN. A copy lives at edge servers near users. Great for static assets and cacheable pages.
- Application memory. An in-process cache inside your service. Sub-millisecond, but it doesn't survive a restart and isn't shared across instances.
- Distributed cache. Redis or Memcached sitting between your app and the database. Shared across all your servers, survives restarts, and this is what most people mean when they say "add a cache."
- Database cache. The database's own buffer pool, plus things like a materialized view.
In an interview, naming two or three of these and explaining the tradeoff shows you understand caching is about layers, not a single box you draw on the whiteboard.
Keeping the cache fresh
This is where it gets interesting. A cache that never updates serves stale data. A cache that updates constantly isn't saving you much. You need a strategy.
Cache-aside is the default. Your app checks the cache first. On a miss, it reads the database, writes the result into the cache, and returns it. Simple, and the cache only holds things that were actually requested. The downside is the first request for any item is always slow, and you have to handle invalidation yourself when data changes.
Write-through writes to the cache and the database at the same time on every write. The cache is always current, but every write is slower, and you cache things that might never get read.
Write-back writes to the cache first and flushes to the database later. Fast writes, but you can lose data if the cache dies before the flush. Use it only when you can tolerate that.
Most systems use cache-aside with a TTL - a time-to-live that expires entries after some window. The TTL is your safety net. Even if you forget to invalidate something, it ages out on its own.
The failure modes interviewers probe
Saying "add a cache" is easy. Knowing how a cache breaks is what separates a real answer from a hand-wave.
Stale data. The database changed but the cache didn't. You fix this with invalidation on write, shorter TTLs, or accepting some staleness where it's harmless.
Thundering herd. A popular key expires, and a thousand requests all miss at once and hit the database together. You can stagger TTLs, or let one request rebuild the value while the others wait or serve the old copy.
Cache stampede on cold start. Right after a deploy or a cache flush, everything is a miss. Warm the cache for known-hot keys before taking traffic if you can.
Memory limits and eviction. Caches are finite. When they fill up, they evict. LRU - least recently used - is the common policy. Know that your cache will drop things, and make sure a miss is correct, just slower.
A cache should never be the only place data lives. If it can be wrong or empty without breaking correctness, you've designed it right.
How to use this in an interview
When the interviewer asks you to scale a read-heavy system, lead with caching. But be specific. Say which layer, say cache-aside with a TTL, and name one failure mode and how you'd handle it.
That structure - where, how, what breaks - is exactly what graders look for. If you want to drill it, set up a mock interview on a system like a URL shortener or a news feed and force yourself to walk all three out loud.
Caching won't solve every scaling problem. But it's almost always the first thing worth reaching for, and explaining it well is one of the highest-leverage things you can practice.