Interactive Explainer

🎯Key Takeaways

Use the 5-step framework: clarify → estimate → high-level → deep-dive → cross-cutting concerns

Back-of-envelope estimation drives every architectural decision — practice until it is automatic

CAP in practice: choose between CP (consistency) and AP (availability) based on your use case

Core building blocks: load balancer, cache, database, message queue, object storage

Memorize latency numbers: memory (100ns) vs SSD (150µs) vs network (0.5ms-150ms)

Design for failure: thundering herd, hot partitions, cascading failures — add circuit breakers everywhere

System Design Fundamentals: From Idea to Architecture

The mental frameworks, vocabulary, and structured approach that separates senior engineers from the rest. Learn to scope, estimate, and design any system under interview pressure.

~8 min read

Be the first to complete!

What you'll learn

Use the 5-step framework: clarify → estimate → high-level → deep-dive → cross-cutting concerns
Back-of-envelope estimation drives every architectural decision — practice until it is automatic
CAP in practice: choose between CP (consistency) and AP (availability) based on your use case
Core building blocks: load balancer, cache, database, message queue, object storage
Memorize latency numbers: memory (100ns) vs SSD (150µs) vs network (0.5ms-150ms)
Design for failure: thundering herd, hot partitions, cascading failures — add circuit breakers everywhere

Lesson outline

Why "just start coding" fails at scale

At a startup, the fastest path to production wins. At companies with millions of users, a wrong architectural decision can cost months of migration work and cause outages affecting real people. The gap between "code that works" and "system that scales" is architectural thinking — deciding the right abstractions, data flows, and failure boundaries before writing code.

System design is not about memorizing architectures. It is about asking the right questions in the right order: Who are the users? What scale do we need? Where are the failure points? What are we trading off? An engineer who can answer these questions systematically is worth 10x one who cannot.

The Principal Engineer Mindset

Before you open your IDE, spend 20 minutes on a napkin design. Every hour saved in design saves 10 hours of debugging later. This is not optional at scale — it is survival.

The FAANG system design framework (USE it in every interview)

Step 1 — Clarify requirements (5 min). Never assume. Ask: functional requirements (what the system must do), non-functional requirements (latency, availability, consistency, scale). Example: "Is this read-heavy or write-heavy? What is the acceptable p99 latency? Do we need strong consistency or is eventual okay?"

Step 2 — Estimate scale (3 min). Back-of-envelope: DAU (daily active users), requests/second, storage needed per year. These numbers drive every architectural decision. 1M DAU × 10 requests/day = ~115 RPS. At 1KB per request that is 115 KB/s throughput.

Step 3 — High-level design (10 min). Draw the boxes: client → load balancer → app servers → cache → database. Identify the critical path. Do not go deep yet — establish the skeleton.

Step 4 — Deep dive on bottlenecks (15 min). Pick the hardest parts: How do you shard the database? What is the caching strategy? How do you handle hotspots? How does the system behave under failure?

Step 5 — Address cross-cutting concerns (5 min). Monitoring, authentication, rate limiting, data migration, cost.

Most Candidates Fail Here

They jump to Step 3 without clarifying requirements. The interviewer then asks "but what if we need 10x scale?" and the candidate has to redesign everything. Clarify FIRST.

back-of-envelope.md

1# Back-of-Envelope Estimation Template
2 
3## Useful Numbers to Memorize
Memorize these — you will need them in every system design interview
4- 1 million seconds ≈ 11.5 days
5- 1 billion requests/day ≈ 11,600 RPS
6- SSD sequential read: ~500 MB/s
7- Network within datacenter: ~10 Gbps
8- HDD seek time: ~10ms | SSD: ~0.1ms | Memory: ~100ns
9 
10## Scale Estimation for a Twitter-like Feed
11DAU: 100M users
12Avg tweets/user/day: 1 (writes), 50 reads
13Writes: 100M / 86,400 ≈ 1,160 writes/sec
14Reads:  100M × 50 / 86,400 ≈ 58,000 reads/sec → read-heavy → cache aggressively
15 
16Storage (tweets):
17- 1 tweet = 280 chars ≈ 560 bytes + metadata ≈ 1 KB
Read:write ratio drives architecture — cache for reads, queue for writes
18- 1,160 writes/sec × 86,400 × 365 ≈ 36.5 TB/year
19 
20## Capacity Planning Rule of Thumb
21- Add 3x headroom for spikes
22- Plan for 2x next year's projected growth
23- Cache can absorb 80%+ of reads if hit rate > 90%

Core building blocks: what every system is made of

Load balancers distribute traffic across servers. Layer 4 (TCP) is faster; Layer 7 (HTTP) is smarter (route by URL, host, headers). Use L7 for microservices.

Caches sit in front of slow resources. L1 = in-process (fastest, not shared). L2 = Redis/Memcached (shared, network hop). CDN = edge cache (geographic). The cache hit rate is the most important metric — target 95%+.

Databases store durable state. Relational (PostgreSQL, MySQL) for structured, ACID-needed data. NoSQL (Cassandra, DynamoDB) for massive write throughput and flexible schema. Choose based on your access patterns, not popularity.

Message queues (Kafka, SQS, RabbitMQ) decouple producers from consumers, buffer spikes, and enable async processing. Critical for anything that can be processed later: emails, analytics, notifications.

Object storage (S3) for blobs: images, videos, backups. Unlimited scale, 99.999999999% durability, cheap. Never store blobs in your relational database.

Building Block	Latency	Scale	When to Use
In-process cache	< 1µs	Single server	Computed values, hot config
Redis/Memcached	< 1ms	Cluster (TBs)	Shared session, hot objects
PostgreSQL	1-10ms	Vertical + read replicas	ACID, complex queries
Cassandra	1-5ms	Petabytes	High write throughput, time-series
Kafka	5-10ms	Millions/sec	Event streaming, audit log
S3	10-100ms	Unlimited	Blobs, backups, static assets

The CAP theorem — what it actually means in practice

CAP theorem states that a distributed system can guarantee at most two of three: Consistency (every read sees the latest write), Availability (every request gets a response), Partition tolerance (system works despite network splits).

In practice, you always need Partition tolerance (networks fail). So the real choice is CP vs AP: Do you want strong consistency (risk unavailability) or high availability (risk stale data)?

CP systems (ZooKeeper, HBase, etcd): When a partition occurs, they refuse requests rather than return stale data. Use for: distributed locks, leader election, financial transactions.

AP systems (Cassandra, DynamoDB, CouchDB): When a partition occurs, they keep serving requests but may return stale data. Use for: social feeds, DNS, shopping carts.

The interviewer trap: "Do you prefer SQL or NoSQL?" Wrong answer: "It depends." Right answer: "It depends on my consistency and availability requirements. Let me explain my access patterns..."

PACELC is more useful than CAP

PACELC extends CAP: even when there is no Partition (normal operation), you still trade off latency vs consistency. DynamoDB defaults to eventual consistency (low latency) but offers strongly consistent reads at 2x cost.

Latency numbers every engineer must know

These numbers from Jeff Dean (Google) are essential for system design. They tell you where time actually goes:

Operation	Latency	Human Scale
L1 cache reference	0.5 ns	1 second
L2 cache reference	7 ns	14 seconds
Main memory reference	100 ns	3.5 minutes
SSD random read	150 µs	6 days
HDD seek	10 ms	16.5 minutes
Network: same datacenter	0.5 ms	...
Network: US cross-country	40 ms	...
Network: US to Europe	150 ms	...

Key insight: memory is 1000x faster than SSD. SSD is 100x faster than disk. Network between datacenters adds 40-150ms. Design to minimize network hops and maximize cache hits.

latency-budget.ts

1// Latency Budget Analysis — a real engineering discipline
2// Every user-facing request has a latency budget.
3// Allocate it consciously.
4 
5interface LatencyBudget {
6  target_p99_ms: number;
7  allocations: Array<{
8    component: string;
9    budget_ms: number;
10    actual_p99_ms?: number;
11  }>;
12}
13 
14const checkoutLatencyBudget: LatencyBudget = {
15  target_p99_ms: 300,  // 300ms total budget for checkout API
16  allocations: [
17    { component: 'Auth middleware',      budget_ms: 5,   actual_p99_ms: 3  },
18    { component: 'Redis cart read',      budget_ms: 5,   actual_p99_ms: 4  },
19    { component: 'Postgres: user data',  budget_ms: 20,  actual_p99_ms: 18 },
20    { component: 'Inventory service',    budget_ms: 50,  actual_p99_ms: 45 },
21    { component: 'Payment gateway',      budget_ms: 180, actual_p99_ms: 200 }, // ← OVER BUDGET
22    { component: 'Write order to DB',    budget_ms: 20,  actual_p99_ms: 12 },
23    { component: 'Emit Kafka event',     budget_ms: 10,  actual_p99_ms: 8  },
24    { component: 'Serialization/misc',   budget_ms: 10,  actual_p99_ms: 6  },
25  ],
Always trace latency per component — gut feel is wrong
26};
27 
28// Payment gateway is 200ms actual vs 180ms budget → p99 SLO breach
29// Action: add timeout + async fallback, or negotiate SLA with payment vendor

Common failure modes and how to prevent them

Thundering herd: A cached key expires. 10,000 concurrent requests all miss cache simultaneously, hammering the database. Prevention: probabilistic early expiry (recalculate before expiry at random), cache stampede locks (only one request refills), background refresh.

Hot partition: 90% of traffic goes to one database shard (e.g. all users with last name A-B are in shard 1). Prevention: consistent hashing with virtual nodes, random suffix on hot keys.

Cascading failures: Service A is slow → B times out waiting → B's thread pool exhausts → B goes down → C goes down. Prevention: circuit breakers, bulkheads (separate thread pools per dependency), timeouts everywhere.

Head-of-line blocking: One slow request in a queue blocks all subsequent requests. Prevention: request hedging (send to 2 replicas, take first response), async processing, priority queues.

Cascading failures kill companies

The 2021 Facebook outage was a cascading failure: BGP route withdrawal → DNS servers offline → all services trying to reconnect simultaneously → overloaded internal systems. Duration: 6 hours. Cost: ~$60M in lost revenue.

How this might come up in interviews

System design interviews test structured thinking under uncertainty. Interviewers want to see your process, not a memorized architecture.

Common questions:

Design a URL shortener (classic warm-up)
Design Twitter's feed generation
Design a distributed rate limiter
How would you design the storage system for a video platform like YouTube?
Walk me through how you would scale an e-commerce checkout system to handle Black Friday traffic

Strong answers include:

Asks clarifying questions before drawing anything
Does back-of-envelope math to validate design choices
Proactively identifies failure modes and trade-offs
Adjusts design when requirements change mid-interview

Red flags:

Jumps straight to drawing without clarifying
Proposes a specific technology without justifying why
Cannot explain trade-offs when pressed
Designs only the happy path — ignores failure modes

Quick check · System Design Fundamentals: From Idea to Architecture

1 / 3

Your system design interview question asks you to design Twitter. What is the FIRST thing you should do?

Key takeaways

Use the 5-step framework: clarify → estimate → high-level → deep-dive → cross-cutting concerns
Back-of-envelope estimation drives every architectural decision — practice until it is automatic
CAP in practice: choose between CP (consistency) and AP (availability) based on your use case
Core building blocks: load balancer, cache, database, message queue, object storage
Memorize latency numbers: memory (100ns) vs SSD (150µs) vs network (0.5ms-150ms)
Design for failure: thundering herd, hot partitions, cascading failures — add circuit breakers everywhere

From the books

System Design Interview – An Insider's Guide — Alex Xu (2020)

Chapter 1: Scale From Zero To Millions Of Users

Start with a single server, then add each component only when you have a concrete reason. Premature optimization is the root of all evil — but so is premature simplicity at scale.

Designing Data-Intensive Applications — Martin Kleppmann (2017)

Chapter 1: Reliable, Scalable, and Maintainable Applications

The three concerns of every data system: reliability (correct under adversity), scalability (coping with load), maintainability (other engineers can work on it). Optimize in this order.

Ready to see how this works in the cloud?

Switch to Career Paths for structured paths (e.g. Developer, DevOps) and provider-specific lessons.

View role-based paths

Discussion

Questions? Discuss in the community or start a thread below.

Join Discord

In-app Q&A