Shard the keyspace with consistent hashing
Continues from the last build: The replicated database now has read copies for scale, but every write and every new row still lands on one primary, and that primary is running out of room and write headroom.
Rung 5 gave Linkly a primary with two read replicas, and reads stopped being the problem.
What you'll build
A sharded link store fronted by a consistent hash ring with virtual nodes, a documented shard key decision, a hot-shard mitigation plan, a cross-shard query strategy for 'list my links', and an ADR with capacity math a reviewer can check line by line.
See how we teach, before you sign up
You don't just get code dumped on you. Every starter file and every solution is explained line-by-line, in plain English. Here's one real file from this project:
# ADR-006: Shard the link keyspace ## Status Proposed ## Context <!-- What limit did rung 5 hit? Cite the two numbers: peak write QPS and projected storage. --> ## Options considered 1. Vertical scale the primary further 2. Modulo-N hashing over the short code 3. Consistent hashing with virtual nodes over the short code 4. Range-based sharding by creation date ## Decision <!-- State the shard key and the number of shards, and why. --> ## Consequences <!-- What gets harder: cross-shard queries, hot shards, operational complexity. What gets easier. -->
Reading this file
# ADR-006: Shard the link keyspaceTitle carries the same numbering scheme as the rest of the ladder's ADRs.## ContextThis must cite the specific numbers that forced the decision, not a vague 'it got slow'.2. Modulo-N hashing over the short codeListing this option, and rejecting it, is what proves you understand the tradeoff, not just the answer.## DecisionEvery ADR in this ladder ends in a single, defensible sentence here.
Fill in Context, Options, Decision, Consequences before you touch code. This is the skeleton for milestone 6.
That's 1 of 9 explained code blocks in this single project.
The build, milestone by milestone
- 1
Pick the shard key and prove you need to shard
4 guided stepsA wrong shard key is the most expensive mistake in this rung to undo later; it decides whether reads for a single redirect stay on one shard (good) or every 'list my links' query stays on one shard (also fine, actually the opposite problem, which milestone 5 handles). Sharding by short_code keeps the hot path, the redirect, a single-shard lookup.
- 2
Reproduce the modulo-N meltdown
4 guided stepsEveryone reaches for modulo hashing first because it is one line of code. You need to feel, in a number, why it is the wrong one line of code before consistent hashing's extra complexity feels justified.
- 3
Build the consistent hash ring with virtual nodes
4 guided stepsVirtual nodes are what make consistent hashing actually consistent in practice: without them, adding one real shard can still create wildly uneven load, because one lucky shard might own half the ring by chance.
- 4
Find and fix a hot shard
4 guided stepsConsistent hashing balances key count evenly across shards, but it says nothing about traffic per key. One celebrity link can still turn one shard into a bottleneck even on a perfectly balanced ring.
- 5
Design for cross-shard queries
4 guided stepsSharding by short_code was the right call for the hot path, but it makes the second most common query, a user's dashboard of their own links, a scatter-gather across every shard unless you build a secondary index.
- 6
Write the ADR and defend the shard count
4 guided stepsA staff engineer reviewing this rung should be able to check every claim against a number in the doc, not take 'sharding seemed like a good idea' on faith.
What's inside when you start
You'll walk away with
This is portfolio-grade. Build it free.
Sign up to unlock every milestone step-by-step, the code skeletons, full reference solutions, and checkable tasks, with your progress saved as you build.
Start building