Queues & Streaming · lesson 1 of 5 · about 30 min
How to answer the user now and do the rest in the background: workers, queue depth, why every message can arrive twice, idempotency, poison messages, ordering, and what a growing backlog really means. Tap any for a plain-language card.
Why a queue
That buys three things: speed (the user waits only for the essentials), isolation (a slow or broken email service no longer breaks checkout) and smoothing (a traffic spike becomes a longer queue, not a crash). The price: the work is now eventually done, and you must handle jobs that run twice or fail. The rest of this lesson is about those costs.
Switch to the queue, then mark the email service as down. Compare what the customer sees.
Everything runs one after another inside the request, so the customer waits 610 ms for work they do not need to see.
Producers, queue, workers
Capacity is simple arithmetic: workers × jobs-per-worker-per-second. Here a job takes half a second, so one worker does 2 jobs a second. While jobs arrive slower than capacity, the queue stays near empty. The moment they arrive faster, the queue grows without limit, and so does the wait.
That is why the queue depth (and the age of the oldest message) is the number to watch, and a good signal to autoscale workers on.
Press “Traffic spike”, then add workers and watch the backlog drain. Then remove workers until it grows again.
6 jobs arrive each second but 2 workers can do only 4. The queue grows by 2 every second, forever, until you add workers or the producers slow down.
Delivery guarantees
The flip side: if the job takes longer than the timeout, the message reappears while the first worker is still working, and a second worker starts the same job. That is at-least-once delivery: never lost, sometimes duplicated. (Exactly-once is only possible with help from the handler: the next scene.)
Set how long the job really takes and the timeout, then crash the worker or let the job run long.
For long jobs, either set the timeout above the worst case or have the worker keep extending it (a heartbeat).
The job (20 s) finishes inside the timeout (30 s), so the message is deleted and runs exactly once.
Handling duplicates
The standard help is an idempotency key: a unique id for the logical action (here order-42:charge), sent with the message. Before doing the work the handler records the key in a table with a unique constraint (or SET key NX in Redis). A repeat finds the key already there and skips. Stripe's API works the same way: you send an Idempotency-Key header and a retried request returns the original result instead of charging again.
Detail that matters: record the key and do the work in the same transaction, or the crash window between the two reopens the duplicate.
Deliver the message several times with and without the key and watch the customer's card.
The same message arrived 2 times and the handler obeyed every time. The customer paid $100 for a $50 order.
When a message can never succeed
The cure is a (DLQ): after a message has been received N times, move it to a separate queue where it sits safely for a human or a repair script to inspect. In Amazon SQS this is the redrive policy's maxReceiveCount. Always alarm on the DLQ depth: a growing DLQ means something is broken.
Twelve messages, two of them poison, one worker, 60 steps.
Compare the three modes. Then change how many tries a message gets before it is parked.
Good messages still get through, but the two poison ones are retried again and again: 38 wasted attempts in 60 steps, and they will never stop. Capacity quietly leaks away.
Order versus speed
You usually do not need global order, only order per key (per account, per order, per user). So send all messages for one key to the same place, one at a time: a in Kafka, a message group in an SQS FIFO queue, a consistent worker for each key. Different keys still run in parallel.
Twelve accounts each send three messages in quick succession that must be applied in order.
The trade-off: strict order per key means one hot key cannot be spread across workers.
Messages for one account ran on different workers at the same time, and 11 of 24 follow-up messages finished before the one they depended on. Finish time 28 steps.
Queue, log, or no queue?
A queue is not always the answer, and a log is not a queue. Each answer is worth 20 XP.
Ask: is someone waiting for the answer? and does one reader or many need each message?
Send a welcome email after signup. Any worker may do it, once, with retries on failure.
Tip: press 1 to 3 to answer, Enter for the next one.
Who lives by this
Four real systems and the design choices they publish.
producer → SQS queue → workers (visibility timeout) → dead-letter queue
SQS standard queues deliver at least once, with best-effort ordering and very high throughput. A received message is hidden for the visibility timeout (30 seconds by default) until the worker deletes it, and a redrive policy moves messages that keep failing to a dead-letter queue. FIFO queues add ordering per message group.
producer → exchange → queue → consumer ack
RabbitMQ delivers messages to consumers who acknowledge them when done. If a consumer dies without acknowledging, the broker puts the message back in the queue for another consumer, which is the same at-least-once idea as the visibility timeout, expressed as an ack.
app enqueues job → Redis list → Sidekiq workers → retry with backoff
Sidekiq, the popular Ruby job processor, keeps jobs in Redis and retries failures with exponential backoff, by default up to 25 times over about three weeks, before moving a job to a dead set to inspect. It is a good picture of a queue built from the Redis lists you saw earlier.
producers → partitioned log (retained) → many consumer groups
A queue deletes a message once handled. Kafka keeps it for a retention period, so several consumer groups read the same events independently and a new service can replay history. See the Kafka lesson for partitions, consumer groups and lag.
Interview angle
Six questions, each worth 20 XP. Queue questions are about failure: duplicates, crashes, poison and backlog.
Whenever you add a queue to a design, say three things: how a duplicate is handled, where failed messages go, and what you alarm on.
A worker takes a message from an SQS queue and crashes before finishing. What happens?
Tip: press 1 to 3 to answer, Enter for the next one.
Break it
Adding workers helps only while the workers are the bottleneck. Once the shared database is the limit, more workers just wait in the same line (and in real life they often make the database slower still). The levers that work: fix or scale the downstream, drop stale jobs with a message TTL when an old result is worthless, and alarm on queue age, not only depth.
100 jobs per second arrive. One worker does 10 per second. The database normally handles 160 per second, but at second 20 it slows to 50.
Switch the database to slow, then try adding workers. Then turn on the 30-second TTL.
Workers (and the database) can keep up with 100 jobs a second, so the queue stays empty.