Building Software

Engineering Fundamentals for the Agent Era

Contents Section 5, State

Concurrency and Race Conditions

Mistakes to catch in review

  1. Check-then-act races, such as checking stock and then decrementing it in two separate steps.

  2. An unbounded fan-out of thousands of parallel requests that overwhelms a downstream service or trips its rate limits.

  3. Mutable module-level state shared across requests, leaking one user's data into another user's response.

  4. A deadlock from two code paths taking the same two locks in opposite order.

What goes wrong when many things happen at once, within one process or across many workers: races, deadlocks and overload.

Topics

Race Conditions
How interleaved operations produce results no single ordering would, and how to spot check-then-act patterns.
Locks and Deadlocks
Mutual exclusion, lock ordering, and why holding a lock across a slow operation hurts.
Async and Parallelism
Concurrency without parallelism in event loops, true parallelism across cores, and the bugs each allows.
Message Passing and Shared-Nothing Design
Avoiding shared mutable state by communicating through queues and messages.
Backpressure and Bounded Queues
Limiting work in flight so a fast producer cannot bury a slow consumer.

You understand it when you can

  • Find the race in a short code sample and describe an interleaving that breaks it.
  • Explain how a concurrency limit on a fan-out protects the service downstream, and how you would choose the limit.
  • Explain the difference between concurrency and parallelism with an example of each.

Drill

An agent wrote a script that starts sending a welcome email to all 80,000 new users at once with no concurrency limit. The loop sets a module-level greeting variable just before each call to the send function, and the send function reads that variable only after it has awaited the mail connection. Find both defects and the symptom each one produces in production.

Start here

Watch

Concurrency is not Parallelism by Rob Pike

Rob Pike, 2012. 31-minute talk.

Separates concurrency, which is structuring independent tasks, from parallelism, which is running them at the same time, and builds the idea around communicating through channels instead of shared state.

Zach Tellman - Everything Will Flow

Zach Tellman, 2015. 40-minute talk.

Explains queueing behavior under load and why unbounded queues hide overload until latency collapses, making the case for bounded queues and backpressure.

Read

Operating Systems: Three Easy Pieces

Remzi H. Arpaci-Dusseau and Andrea C. Arpaci-Dusseau, 2023, Version 1.10. Free to read online.

Its concurrency part covers race conditions, locks, condition variables, event-based concurrency and deadlock prevention through lock ordering, in short free chapters.

The Art of Multiprocessor Programming

Maurice Herlihy, Nir Shavit and 2 others, 2020, 2nd edition.

The rigorous treatment of mutual exclusion, lock implementations and concurrent data structures, for readers who want to know why a lock is correct.

Primary sources