Distributed Systems Potpourri

Dated Jul 21, 2026; last modified on Thu, 03 Sep 2026

Contains content that is not extensive enough for its own page.

Streams and Event Sourcing

Unlike message queues , streams can retain data for a configurable period of time, allowing consumers to read and re-read messages from a specified time. You can use a stream to ingest high volumes of events in real-time, e.g., real-time analytics of user engagements in social media.

Event sourcing involves storing changes in application state as a sequence of events. These events can be replayed to reconstruct the system’s state at any point in time. The technique is useful for systems that require a detailed audit trail, e.g., a banking system, where every transaction (deposits, withdrawals, transfers) need to be recorded and may affect multiple accounts.

Doesn’t this then mean that the system needs to be backward compatible to the stored sequence of events? The events from 5 years ago should still work[?]

In a real-time group-chat application, when a participant sends a message, it’s published to a stream associated with the group chat. All chat participants are subscribers to the stream, allowing for real-time communication.

There can be different consumer groups on the same stream. In a real-time analytics system, one consumer group might process events to update a dashboard, while another group processes the events to store them in a database.

Distributed Lock

Traditional databases with ACID properties use transaction locks, e.g., when one user is updating a record, no one else can update it. However, there’s need for longer term locking, e.g., when a user is booking a ticket, other users shouldn’t be able to grab it from underneath them.

Distributed locks are based on distributed key-value pairs. Suppose the agreed upon key is lock:concert:343. To acquire the lock, do SET lock:concert:343 my-token NX EX 30, wait and retry if someone else owns the lock. NX makes SET succeed iff the key doesn’t already exist. EX 30 puts a 30s expiry on the key so that a crashed process doesn’t hold the lock forever. my-token is a random value.

To release the lock, a blind DEL wouldn’t work because someone else might be holding the lock after it expired on us. Instead, do:

if redis.call("GET", KEYS[1]) == ARGV[1] then return redis.call("DEL", KEYS[1]) end

… as Redis executes the whole script atomically as one command.

Suppose process \(p_1\) acquires lock \(l_A\) and then tries to acquire lock \(l_B\), while another process \(p_2\) acquires \(l_B\) and then tries to acquire \(l_A\). This leads to deadlock where both processes are waiting on each other to release a lock. A common cause of this is pulling in locks from far-flung pieces of system, making it hard to recognize deadlocks.

Because Redis’s replication is asynchronous, it’s possible that the primary dies right after granting your lock but the promoted replica may have never heard of your lock. If a stale lock corrupts your system, either enforce the invariant where the data lives (e.g., UPDATE ... WHERE version = X in your DB), or use a consensus system like ZooKeeper or etco.

References

  1. System Design Key Technologies. www.hellointerview.com . Accessed Jul 14, 2026.
  2. Redis Deep Dive for System Design Interviews | Hello Interview System Design in a Hurry. www.hellointerview.com . Jan 16, 2026. Accessed Sep 3, 2026.