Sharding
Sharding comes up when a single database won’t work (e.g., hit storage limits, write throughput limits, or read throughput limits) and you need to split your data across multiple independent servers.
For a user-centric social media app, sharding by user_id means all of a user’s
posts, likes, and comments live on one shard. User-scoped queries are fast, but
“trending posts across all users” become expensive because of hitting every
shard and aggregating results. Mutating transations need to account for the
distributed nature of the shards, making them complex and slow.
There are several sharding techniques to decide where the data goes, e.g.,
- Hash the shard key and use modulo to pick a shard. Assuming randomness, this avoids hot spots.
- Range-based sharding if access patterns naturally partition, e.g., multi-tenant SaaS where each company only queries their own data. Can create hot spots.
- Directory-based sharding consults a lookup table, but this adds a dependecy and latency to every request.
Consistent Hashing helps avoid data movements as DB shards (or cache nodes) come offline or go offline.
Load Balancer
Horizontal scaling involves distributing traffic across multiple machines capable of handling the same request in order to avoid overloading any single machine or creating a hotspot.
Layer 7 load balancers operate at the application level and can route based on the HTTP request content. Layer 4 load balancers work at the TCP level, and are faster (but dumber) because they don’t look at the content. Layer 4 balancing is useful for WebSockets because a persistent TCP connection is needed.
For problems with extremely high traffic, opt for specialized hardware load balancers. Otherwise, you’ll be pulled into the complicated world of network engineering. That said, contends that the differentiation is between building it yourself, and buying a proven tech stack as an appliance with committed support from the vendor. Advantages like offloading SSL computation to hardware is also available from the opensource software load balancers.
Queue
Queues serve as buffers for bursty traffic or as a means of distributing work across a system. During peak hours, a ride sharing application can buffer ride requests, allowing the system to manage them at a manageable rate. A queue can also be used to distribute work, e.g., different worker nodes can pull tasks off the queue.
Sample Flow for a Queue Buffer. Credits: Hello Interview.
Queues introduce latency, so beware introducing them into synchronous workloads with strong latency requirements, e.g., \(< 500\text{ms}\).
If the system supports 200QPS, then 300QPS leads to requests that’ll never get processed. Backpressure involves rejecting [some of the] new requests to slow down the rate at which new messages are accepted.
References
- Core Concepts for System Design Interviews. www.hellointerview.com . Accessed Jun 6, 2026.
- System Design Key Technologies. www.hellointerview.com . Accessed Jul 14, 2026.
- Hardware vs Software load balancers: Just a cost issue? - Server Fault. serverfault.com . Accessed Jul 16, 2026.