Building Software

Engineering Fundamentals for the Agent Era

Contents Section 9, Directing Agents

Accountability: Owning Code You Did Not Type

Mistakes to catch in review

  1. A 600-line agent change approved within a minute, with approval treated as a formality.

  2. A postmortem that names 'AI-generated code' as the root cause, with no owner and no change to tests, review or gates.

  3. A function reproduced nearly verbatim from a copyleft-licensed project and merged into proprietary code with no license review.

  4. An agent stuck in a fix-and-retry loop on a problem it does not understand, with nobody stepping in to take over.

What stays yours when agents do the work: the approval, the incident, the license and the dependencies, plus the skills to act on them yourself.

Topics

Owning the Result
Responsibility for a change stays with the person or team that approved it, whoever or whatever typed it.
On Call for Code You Did Not Write
Being the one paged means understanding agent-written code well enough to diagnose it under pressure.
Provenance, Licensing and Generated Code as Supply Chain
Treating agent output like any third-party contribution, reviewed, attributed and traceable to its source, including the license obligations of code a model reproduces and a record of which agent and model produced each change.
Meaningful Approval
Treating approval as a decision backed by evidence, including the willingness to say no.
The Ironies of Automation: Deliberate Practice and Taking the Controls
Bainbridge's 1983 finding that automation leaves people the hardest tasks with the least practice, the regular hand-flying that counters it, and the signals (novel problems, security-critical code, repeated agent failure) that tell you to do the work yourself.

You understand it when you can

  • Diagnose and fix a bug in your own codebase with no AI assistance.
  • Explain to a colleague why you approved a change and what evidence you relied on.
  • For a recent agent-written change, name who is on call for it, which dependencies and licenses it brought in, and how you would roll it back.

Drill

An agent-written change to a refund service caused a weekend outage, the draft postmortem says 'root cause: AI-generated code' with no action items, and the same change added a 200-line function copied from a GPL-licensed project. Find what the postmortem gets wrong about ownership, the licensing problem you now own, and the review step that would have caught both.

Start here

Watch

Children Of The Magenta Line

Capt. Warren Vanderburgh, 1997. 26-minute talk.

An American Airlines training captain explains how pilots who lean on automation lose the ability to take over, and teaches choosing the level of automation that fits the situation, including flying by hand.

Velocity 2012: Richard Cook, "How Complex Systems Fail"

Richard Cook, 2012. 28-minute talk.

The author of 'How Complex Systems Fail' explains to an operations audience why a single root cause like 'AI-generated code' is a misleading story, and why the people at the sharp end are what keep the system safe.

SREcon26 Americas - The Ironies of AI²

J. Paul Reed, 2026. 41-minute talk.

A resilience-engineering practitioner applies Bainbridge's ironies of automation to AI in operations: how AI changes who can diagnose failures and what the on-call engineer is left doing.

Read

The Field Guide to Understanding 'Human Error'

Sidney Dekker, 2014, 3rd edition.

Shows why labelling a cause as 'human error', or by extension 'AI error', ends the investigation too early, and how to write a postmortem that changes tests, reviews and gates instead of blaming someone.

The Glass Cage: Automation and Us

Nicholas Carr, 2014.

Uses aviation and medicine to show how automation wears down the skills of the people supervising it, which is why an engineer needs regular practice doing the work without assistance.

Software Supply Chain Security: Securing the End-to-End Supply Chain for Software, Firmware, and Hardware

Cassie Crossley, 2024.

Covers provenance, SBOMs and third-party code intake. The same controls apply when agent output is treated as an outside contribution that must be attributed and license-checked.

Primary sources