Sharding comes up in almost every system design interview that touches scale. It's also where a lot of candidates fumble, because they reach for it too fast and can't defend the tradeoffs. This post covers what sharding actually is, when to bring it up, and how to talk about it so you sound like someone who has thought it through.
What sharding actually is
Sharding means splitting one logical database into multiple physical databases, each holding a subset of the data. Each subset is a shard. A user lookup that used to hit one big table now hits whichever shard owns that user.
That's the whole idea. You're spreading data across machines so no single machine has to hold all of it or serve all the traffic for it.
It helps with two problems at once: storage (the dataset is too big for one box) and throughput (one box can't handle the read/write load). If you only have a throughput problem and the data fits comfortably on one machine, you probably want read replicas first. Sharding is the heavier hammer.
Sharding vs replication
Interviewers love to see if you mix these up.
Replication copies the same data to multiple machines. It's for read scaling and availability. Every replica has the full dataset.
Sharding splits different data across machines. It's for write scaling and storage. Each shard has only its slice.
Most real systems do both - you shard, and then each shard has its own replicas. Saying that out loud shows you understand they're not competing choices.
How you pick a shard key
The shard key is the field you use to decide which shard a row lands on. This is the most important decision in the whole design, so spend time on it in the interview.
A good shard key spreads data and traffic evenly, and lets your common queries hit a single shard. A bad one creates hot spots or forces every query to fan out to all shards.
The three common strategies:
Range-based. Split by ranges of the key, like users A-M on shard 1 and N-Z on shard 2. Easy to reason about and good for range queries. The risk is uneven load - if half your active users have last names in one range, that shard burns while the other idles.
Hash-based. Run the key through a hash function and use the result to pick a shard. This spreads data evenly and avoids hot spots. The cost is that range queries get expensive, because related rows scatter across every shard.
Directory-based. Keep a lookup table that maps keys to shards. Most flexible, since you can move keys around. The downside is the lookup table becomes a dependency and a possible bottleneck of its own.
If you only remember one thing: pick a key with high cardinality and even access. User ID is usually safe. Something like country or status is usually a trap, because the distribution is lopsided.
The hard parts (this is what they're testing)
Anyone can split a table. The interview is really about whether you understand what breaks.
Cross-shard queries. A query that needs data from multiple shards has to fan out and merge results. Joins across shards are painful. The usual answer is to denormalize, or to choose a shard key that keeps related data together.
Rebalancing. When you add a shard, data has to move. Naive hashing (hash(key) % N) reshuffles almost everything when N changes. Mention consistent hashing here - it limits how much data moves when you add or remove a node. That single term signals you've seen this problem before.
Transactions. A transaction spanning two shards needs distributed coordination, which is slow and complex. Honest answer: avoid cross-shard transactions by design, or accept eventual consistency where the business allows it.
Hot shards. Even with a decent key, one shard can get hammered - think a celebrity user on a social app. You handle it by splitting that shard further or caching aggressively in front of it.
How to bring it up without overreaching
The strongest move is to not jump straight to sharding. Walk the ladder. Start with a single database. Add indexes and caching. Add read replicas. Only then, when writes or storage actually exceed one machine, introduce sharding.
When you do introduce it, say what it costs. Something like: "I'd shard users by user ID with consistent hashing. That scales writes and storage, but it makes cross-user queries harder and adds operational complexity, so I'd only do it once a single primary can't keep up."
That sentence does a lot. It names the key, names the strategy, and names the tradeoff. Interviewers want the tradeoff most of all.
Quick checklist before the interview
- Explain sharding vs replication in one breath
- Name range, hash, and directory strategies with a tradeoff for each
- Justify a shard key by cardinality and even access
- Know that consistent hashing limits data movement on rebalance
- Be able to say why cross-shard joins and transactions hurt
- Always reach for sharding last, after replicas and caching
If you can hold a five-minute conversation on those points, you're ahead of most candidates. The best way to get there is to say it out loud, ideally in a mock interview where someone pushes back on your shard key. Reading about it gets you the vocabulary. Practicing it gets you the answer.