Read consistency - linearizable, serializable, or eventual?

Linearizable. Every successful write is durable on a majority before ack. Every read returns the latest committed value (or fails). The system is CP (consistent + partition-tolerant); during partition, the minority side becomes unavailable.

Maximum allowed write latency - 10ms (single AZ), 100ms (cross-region)?

Write p99 < 20ms (single AZ), < 100ms (cross-region). Linearizable read p99 < 10ms (with leader lease) or < 50ms (round-trip quorum read).

gitGood.dev

Distributed Systems

Design a Consensus System (Raft / Paxos / etcd)

Hard Premium

Raft leader election, log replication, snapshots - and the CAP theorem in operational practice. The substrate every other distributed system stands on.

The problem

Design a strongly-consistent distributed coordination service - the kind that backs etcd, ZooKeeper, Consul. Multiple replicas accept writes. They agree on a totally-ordered log of operations. The system survives minority failures without data loss and without serving stale committed reads.

This is "implement Raft" framed as a system design question. Strong candidates explain leader election, log replication, the safety invariants, and the snapshot mechanism. Excellent candidates draw the operational reality - latency budgets, quorum sizing, cross-AZ vs cross-region trade-offs, and when consensus is the wrong tool.

Clarifying questions

Asking these before diving into a solution is the difference between a "hire" and a "no signal" rating. Pick the questions whose answers would change your design.

→What's the workload - small metadata (etcd-style) or full database (Spanner-style)?
→Throughput target - hundreds, thousands, or tens of thousands of writes/sec?
→Read consistency - linearizable, serializable, or eventual?
→Cluster size - 3, 5, 7 nodes? Cross-AZ or cross-region?
→Maximum allowed write latency - 10ms (single AZ), 100ms (cross-region)?
→Snapshot frequency requirements - how much disk are we willing to spend?
→Multi-tenancy - one cluster per use case, or shared?
→Disaster recovery - active-passive cross-region, or eventual restore from backup?

Requirements

Functional requirements

·Propose: any node accepts a value; the cluster agrees on a total order
·Linearizable reads (read sees all writes that completed before)
·Watch / subscribe: clients notified when a key changes
·Atomic compare-and-swap (CAS) primitives
·Membership change: add or remove cluster nodes safely
·Snapshot and restore for compaction and recovery
·Survive any minority failure without data loss

Non-functional requirements

Scale: 10K-50K writes/sec for etcd-class. 1M+ keys. Up to 1GB total state. 5-7 node clusters. Read-heavy: 100K-1M reads/sec via read replicas / leader leases.
Latency: Write p99 < 20ms (single AZ), < 100ms (cross-region). Linearizable read p99 < 10ms (with leader lease) or < 50ms (round-trip quorum read).
Availability: Cluster survives floor((N-1)/2) failures - 5-node cluster survives 2 failures. During leader election (~150-1500ms), brief write unavailability. Reads can continue from replicas during election if relaxed consistency is acceptable.
Consistency: Linearizable. Every successful write is durable on a majority before ack. Every read returns the latest committed value (or fails). The system is CP (consistent + partition-tolerant); during partition, the minority side becomes unavailable.

The full walkthrough covers

Architecture components

·Replica state machine
·Raft module
·Write-ahead log (WAL)
·Snapshot store
·Replication RPC layer
·Client-facing API
·Lease manager
·Watch service
·Membership manager

Deep dives

Leader election: terms, votes, and split-vote
Log replication and the commit rule
Snapshots and log compaction
Linearizable reads: leader leases and the read index
Membership changes and the joint consensus problem
When consensus is the wrong tool

What's in the full walkthrough

Capacity estimation · architecture with all 9 components explained · 6 deep dives · trade-off analysis · 8 common follow-up questions

Subscribe to Continue

Get full access to gitGood.dev with a subscription.

Start Your 10-Day Trial

Try everything free for 10 days. Cancel anytime.

1000+ practice questions
AI mock interviews (3/day)
Coding challenges with real execution
Verified skill badges
105+ certification study guides

View Plans

10-day trial, then $5/mo - or $40/yr ($3.33/mo, save 33%). Cancel anytime.

Companies that test this topic

Practice in interview format

Reading is the floor. The interview signal is in walking through this live with someone probing follow-ups. Use the AI mock interview to practice talking through requirements, architecture, and trade-offs out loud.

Start an AI mock interview →

Distributed Systems

Design a Consensus System (Raft / Paxos / etcd)

Hard Premium

Raft leader election, log replication, snapshots - and the CAP theorem in operational practice. The substrate every other distributed system stands on.

The problem

Clarifying questions

Asking these before diving into a solution is the difference between a "hire" and a "no signal" rating. Pick the questions whose answers would change your design.

→What's the workload - small metadata (etcd-style) or full database (Spanner-style)?
→Throughput target - hundreds, thousands, or tens of thousands of writes/sec?
→Read consistency - linearizable, serializable, or eventual?
→Cluster size - 3, 5, 7 nodes? Cross-AZ or cross-region?
→Maximum allowed write latency - 10ms (single AZ), 100ms (cross-region)?
→Snapshot frequency requirements - how much disk are we willing to spend?
→Multi-tenancy - one cluster per use case, or shared?
→Disaster recovery - active-passive cross-region, or eventual restore from backup?

Requirements

Functional requirements

·Propose: any node accepts a value; the cluster agrees on a total order
·Linearizable reads (read sees all writes that completed before)
·Watch / subscribe: clients notified when a key changes
·Atomic compare-and-swap (CAS) primitives
·Membership change: add or remove cluster nodes safely
·Snapshot and restore for compaction and recovery
·Survive any minority failure without data loss

Non-functional requirements

Scale: 10K-50K writes/sec for etcd-class. 1M+ keys. Up to 1GB total state. 5-7 node clusters. Read-heavy: 100K-1M reads/sec via read replicas / leader leases.
Latency: Write p99 < 20ms (single AZ), < 100ms (cross-region). Linearizable read p99 < 10ms (with leader lease) or < 50ms (round-trip quorum read).
Availability: Cluster survives floor((N-1)/2) failures - 5-node cluster survives 2 failures. During leader election (~150-1500ms), brief write unavailability. Reads can continue from replicas during election if relaxed consistency is acceptable.
Consistency: Linearizable. Every successful write is durable on a majority before ack. Every read returns the latest committed value (or fails). The system is CP (consistent + partition-tolerant); during partition, the minority side becomes unavailable.

The full walkthrough covers

Architecture components

·Replica state machine
·Raft module
·Write-ahead log (WAL)
·Snapshot store
·Replication RPC layer
·Client-facing API
·Lease manager
·Watch service
·Membership manager

Deep dives

Leader election: terms, votes, and split-vote
Log replication and the commit rule
Snapshots and log compaction
Linearizable reads: leader leases and the read index
Membership changes and the joint consensus problem
When consensus is the wrong tool

What's in the full walkthrough

Capacity estimation · architecture with all 9 components explained · 6 deep dives · trade-off analysis · 8 common follow-up questions

Subscribe to Continue

Get full access to gitGood.dev with a subscription.

Start Your 10-Day Trial

Try everything free for 10 days. Cancel anytime.

1000+ practice questions
AI mock interviews (3/day)
Coding challenges with real execution
Verified skill badges
105+ certification study guides

View Plans

10-day trial, then $5/mo - or $40/yr ($3.33/mo, save 33%). Cancel anytime.

Companies that test this topic

Practice in interview format

Start an AI mock interview →

Design a Consensus System (Raft / Paxos / etcd)

The problem

Clarifying questions

Requirements

Functional requirements

Non-functional requirements

The full walkthrough covers

Architecture components

Deep dives

What's in the full walkthrough

Subscribe to Continue

Start Your 10-Day Trial

Related system design topics

Distributed Cache

Message Queue

Companies that test this topic

Practice in interview format

Design a Consensus System (Raft / Paxos / etcd)

The problem

Clarifying questions

Requirements

Functional requirements

Non-functional requirements

The full walkthrough covers

Architecture components

Deep dives

What's in the full walkthrough

Subscribe to Continue

Start Your 10-Day Trial

Related system design topics

Distributed Cache

Message Queue

Companies that test this topic

Practice in interview format