AI in the Engineering Team

011. Two ways to adopt AI

The two modes

The ceiling on effort-saving

The first mode has a hard limit, and the arithmetic is simple. Coding is only part of the time it takes to deliver a change. Suppose it is a quarter, with the rest spent waiting for review, testing, clarifying requirements and releasing.

No model improvement can lift a team past that ceiling. The only way through is to speed up the other three quarters, and that means changing the process. Redesign is not the ambitious option. It is the only option that leads anywhere.

Why effort-saving is the default

It is worth understanding why sensible people settle there, because exhortation will not move them.

• It feels responsible. Reading every line seems like diligence, and nobody was ever criticised for it.
• Trust has not been earned. Without reliable automated checks, reading everything really is the only way to know the code is right.
• The process assumes human pace. Review queues, test cycles, sprint rituals and release trains were all sized for people typing.
• The incentives point that way. If output expectations stay the same, using AI to reduce personal effort is entirely rational. If going faster only brings more work, slack is the better deal.

Three of those four are properties of the system, not of the individuals. An engineer cannot go faster than the review queue, the test suite and the release process allow, whatever tools they hold. Changing those is management work.

A fair word for the first mode

Effort-saving is not worthless. It is the on-ramp. People learn what the tools can do, where they fail and how to brief them. The mistake is stopping there and calling it adoption. The rest of this part is about what comes next.

022. Where the time goes

The value stream

Follow one change from idea to production: refined, in development, in review, in test, released. At each stage there is time when someone is working on it and time when it sits in a queue. The share of elapsed time spent in active work is called flow efficiency, and in most teams it is well under half.

The theory of constraints gives the rule. A system’s throughput is set by its narrowest point. Improving any other stage only builds a queue in front of that point. Faster coding upstream of a slow review process produces a growing pile of open pull requests, and nothing more.

What the studies found

Reading them together

• Gains are real on isolated, greenfield or unfamiliar tasks. That is where the first study sits.
• Gains are smallest for experts in code they know well, using interactive tools in effort-saving mode. The METR participants spent their time prompting, waiting and reviewing output. That is a precise picture of the first mode from section 1.
• Perception is unreliable. People felt faster while being slower. A 39-point gap between belief and measurement is reason enough to measure.
• Local speed did not reach delivery. DORA’s explanation for 2024 was that AI led to larger batches of change, which are riskier, while the fundamentals downstream had not changed.

The tools have moved on a great deal since early 2025, so treat the specific numbers as dated. The two durable lessons are the perception gap, and that the result depends on the system the tool is placed in.

Where the bottleneck goes

When coding speeds up, the constraint moves to one of these:

• Review. More pull requests, and larger ones, waiting on the same few reviewers.
• Testing. Slow continuous integration, flaky tests and limited QA capacity.
• Decisions. Engineering now waits for product answers. “What should we build?” becomes the constraint.
• Integration. Parallel work produces merge conflicts and duplicated effort.
• Release. Approval steps, release trains and change boards sized for a slower flow.

Finding yours

Take the last 30 completed tickets. For each, record how long it spent waiting at each stage, using timestamps your tracker and source host already hold. The stage with the largest total wait is your constraint. That is where AI, and your own effort, should go first. Section 10 builds this into a plan.

033. Verification at machine speed

Assurance has to come from somewhere

Confidence that a change is safe can come from many sources. Line-by-line human reading is the most expensive of them, and one of the least reliable. Studies of code review have long found that reviewers’ defect detection falls away sharply once a change passes a few hundred lines. A person skimming a 1,500-line AI-generated diff is performing a ritual, not a check.

The question for a team is therefore not “should we review less?” It is “where should our assurance come from, and how much of it can run without us?”

The assurance stack

Every layer above the last row runs at machine speed. The more you trust them, the less a person has to read.

A different review question

The reviewer’s question changes from “is each line correct?” to three others. Is this the right change? Is it adequately tested? What is the damage if it is wrong, and how fast can we undo it?

In practice that means reading the specification and the tests more carefully than the implementation. If the tests truly pin down the required behaviour, the implementation matters much less, and it can be regenerated at will.

Review by risk

Agree the tiers with security and compliance before you use them. Encode them as path-based ownership rules in the repository, so that tooling enforces the policy and nobody has to remember it. In a regulated business, this written policy is also what you show an auditor.

What must be true first

Trusting code that nobody read line by line is reasonable only when these hold:

• Tests you believe. They would fail if the behaviour broke. Check by breaking something on purpose, or with mutation testing.
• Fast continuous integration, ideally under ten minutes, so that agents and people get feedback while it is useful.
• Small changes, so that any failure is easy to locate and revert.
• Rollback in minutes, and feature flags for anything user-facing.
• Monitoring that would notice a problem before a customer reports it.
• Ownership rules that route high-risk paths to people automatically.

Where these do not hold, you have found your investment backlog. It is work that agents are well suited to: backfilling tests on legacy code is tedious for people and cheap for a model. The loop reinforces itself. Better verification allows more delegation, and delegation builds better verification.

Two warnings

The capability trap. Under delivery pressure, it is tempting to skip this groundwork and simply review less. Output rises for a quarter. Then defects, rework and incidents consume the gain and more. The investment has to be protected like any other preventive work.

Accept and watch the residual risk. No stack catches everything. Track the defect escape rate and the change failure rate. If they rise, tighten a tier or strengthen a layer. The goal is a deliberate level of risk, not zero and not unknown.

044. Working patterns at AI pace

The specification is the new source

When implementation is cheap, the scarce skill is saying exactly what is wanted. A good task specification states the problem, the acceptance criteria as concrete examples or tests, the constraints, and what is out of scope. Fifteen to thirty minutes spent here regularly saves hours.

This also moves review upstream. A colleague can review a one-page specification in five minutes, and an error caught there costs almost nothing. The same error caught in a finished diff costs the whole run. Peer review of specifications is the cheapest quality practice a team can adopt.

The directing cycle

The engineer’s loop becomes: specify, dispatch, verify, integrate. Time shifts to the first and third steps. The second belongs to the agent, and the fourth is mostly automated.

• Parallel work. One engineer runs several agent tasks at once, each in its own branch or environment. Three to five is a realistic ceiling, because human attention is the limit. Choose tasks that touch different areas, to avoid merge conflicts.
• Background work. Tickets that are “agent-ready” go to background agents and come back as tested pull requests. A ticket is agent-ready when it has clear acceptance criteria, pointers to the relevant code and a way to verify the result. Writing tickets to that standard helps human engineers too.
• Overnight work. Long-running tasks such as migrations, test backfill and dependency upgrades run outside working hours, and are triaged in the morning.

Limits on work in progress still apply

Agents make starting work free. Finishing it still costs human attention. An engineer can easily end a day with ten half-verified pull requests and nothing shipped. Limit the number of items awaiting a person’s judgement, and finish before starting more. The old rule holds: value comes from work completed, not work begun.

Keep batches small

The 2024 DORA finding in section 2 was traced to larger batches of change. AI makes a 2,000-line change as easy to produce as a 200-line one, and it is ten times harder to verify and to revert. Ask agents to split work into a series of small, independently shippable changes. Set a size guideline for pull requests and have tooling flag breaches.

What changes in planning

• Estimates move. For many tasks the implementation cost collapses, and the uncertainty shifts to specifying the work and verifying it. Estimate those.
• Capacity is no longer typing time. It is the team’s capacity to specify, decide and verify.
• Show, do not describe. A working prototype settles a design argument faster than a document. Bring one to the meeting.
• Regenerate, do not patch. When a generated solution is wrong in approach, fix the specification and run it again. Editing it by hand is often slower.

Token cost is not the constraint

Part 3 of the language models module shows that heavy agent use costs a few dollars per engineer per day. Running three attempts in parallel to pick the best, or letting an agent work overnight, is cheap next to an hour of a person’s time. Teams that ration tokens while engineers wait have the economics backwards.

055. Doing what you would not have done

The economics

Every team carries a long list of things that would be valuable and never rise above the line: too tedious, too large, too uncertain. Those judgements were made at human cost. Many of them are now wrong, and few teams have revisited them.

Where to look

Choose what compounds

Cheap to build is not cheap to own. Every line shipped is a line to maintain, a prototype is not a product, and saying no still matters. More output is not the same as more value.

Give priority to work that compounds: anything that widens your constraint from section 2, strengthens the verification stack from section 3, or lowers the future cost of change. Test backfill, faster CI and completed migrations all make the next piece of AI-assisted work faster and safer.

Make it a habit

Keep a visible list titled “not worth it before”. Add to it whenever someone says “we would never have time for that”. Review it each quarter, and reserve explicit capacity for it. These items also make the most persuasive stories when you report upward, because they are things the organisation can see that it would not otherwise have had.

066. Quality and accountability at speed

Failure patterns and what catches them

Part 2 of the language models module explains why several of these occur. A model rewarded for passing checks will sometimes satisfy the check and miss the intent.

The ownership model

The person who dispatches a task and merges the result owns it. “The agent wrote it” is never an acceptable line in a postmortem.

Accountability does not mean having read every line. It means having made sure that the assurance was adequate for the risk tier. A surgeon is accountable for an operation without having personally sterilised the instruments, because there is a system they can rely on and are responsible for using.

When something escapes, the blameless question is which layer of the stack should have caught it. The action is to strengthen that layer, and not to reintroduce blanket manual review.

Comprehension debt

This is the new risk. Code that nobody on the team has read or understands works well until an incident at 2am, when someone must reason about it under pressure.

• Keep architecture human-owned. People decide module boundaries, data models and interfaces. Agents fill them in.
• Require a summary with every change. The agent writes what changed and why, and drafts a decision record for anything structural.
• Keep modules small. Understanding can then be rebuilt quickly. Comprehension has become cheaper too: a model can explain an unfamiliar module on demand, which softens this risk without removing it.
• Test for on-call readiness. Ask whether the engineer on call could diagnose this service. If not, that is a gap to close before the incident.
• Keep high-risk code human-understood, always.

Entropy

Many small generated changes let a codebase drift towards inconsistency and duplication. Schedule regular clean-up runs by agents, with a person setting the direction, and keep the instruction file current so that new work follows the patterns you want.

077. Measuring impact

What not to measure

• lines of code, commits and pull request counts
• the percentage of code written by AI
• suggestion acceptance rates
• tokens consumed or licences activated
• hours saved, as estimated by the people concerned

The first four measure activity and are easily inflated, by accident or on purpose. The last is unreliable for the reason section 2 gave: people who were 19% slower believed they were 20% faster.

What to measure

Add a product outcome measure, so that faster delivery is tied to something the business values. Add a short, regular survey on what helps and what gets in the way.

Two signatures

Effort-saving adoption: usage up, satisfaction up, pull requests up, cycle time flat, review queue longer, pull requests larger, stability down.

Redesign: cycle time falling stage by stage, pull request size steady or down, change failure rate steady, more items finished, and long-deferred debt items closing.

If your dashboard shows the first pattern, you have a tooling rollout and not yet a change in how work is done.

Ground rules

• Baseline first. Four to six weeks of history can usually be pulled retrospectively from your tracker and source host.
• Compare a team with its own past. Do not compare individuals or teams with each other, because their work differs.
• Never use these figures for individual performance. The moment you do, people manage the number, and you lose both honesty and the measure.
• Report upward with outcomes first: delivery measures, one or two stories of things that would never otherwise have been done, and cost per engineer from part 3 of the language models module beside them.

088. People and skills

The skill shift

The skills on the left still matter, because you cannot judge what you could not have written. They are no longer where most of the hours go.

Uneven adoption

Expect a few enthusiasts, a majority who dabble, and some who decline. Mandates and usage quotas produce the activity signature from section 7 and little else. What spreads practice is seeing it work.

• Run a short weekly session in which someone shows a real piece of work, including what went wrong.
• Pair enthusiasts with sceptics on a real task.
• Keep prompts, instruction files and agent-ready ticket templates in a shared repository that everyone improves.
• Name two or three champions, and give them time for it.

Take the resistance seriously

Much of it is reasonable. Some engineers had poor experiences with earlier, weaker tools. Some have well-founded quality concerns, which sections 3 and 6 exist to answer. Some find meaning in the craft of writing code. Many fear that gains will be turned into headcount cuts or simply higher targets.

Answer honestly. Say what the gains are for: more scope, paying down debt, less crunch. Make no promises you cannot keep. Involve the sceptics in designing the verification stack, because their instincts about what could go wrong are exactly what it needs.

Fix the incentive

Section 1 noted that effort-saving is rational when going faster only earns more work. Change what is recognised. Credit the people who build the verification that lets everyone delegate more, who write the tickets that agents can complete, and who produce shared tooling. This is preventive, enabling work, and it stays invisible unless you make it visible.

Junior engineers

The risk is real. The small, well-defined tasks that juniors once learned on are the ones agents do best. A junior who only prompts can ship code they do not understand and cannot judge.

The opportunity is also real. A patient tutor is available at any hour, and exposure to varied problems comes faster than it used to.

• Juniors explain the code they submit, in their own words.
• Schedule deliberate practice without AI for the fundamentals, as a musician practises scales.
• Pair juniors with seniors on reviewing agent output, which teaches judgement directly.
• Give juniors ownership of verification work such as tests, checks and evals. It builds the central skill.
• Keep hiring them. A team that stops has no seniors in five years.

Seniors, hiring and roles

Senior engineers can lose sharpness in design and debugging if they only ever supervise. Rotate them through hands-on work on the hardest problems.

In interviews, allow AI and watch how the candidate directs and checks it. Assess specification, judgement of output, debugging and system design. How fast someone types a sorting algorithm tells you little now.

QA moves from executing tests towards verification strategy and tooling. Platform and developer-experience work gains in value, because it is what lets everyone else delegate.

Watch the load

Directing several agents means constant switching of attention and a stream of judgement calls. It is more tiring than it looks. Watch for fatigue, and do not assume that an engineer with five agents running is having an easy day.

099. Beyond developers

The constraint crosses boundaries

Section 2 listed “decisions” as a place where the bottleneck lands. When engineering can build in a day what took a fortnight, the scarce resource becomes knowing what to build. Bring product and design into the change early. A product manager who can test three ideas with working prototypes in a week changes the pace of the whole team.

Non-developers building software

Product managers and analysts can now build working prototypes and small internal tools themselves. This is very good for learning and for speed. It needs one guardrail: a clear line between a personal or sandboxed tool and anything that touches customer data or production, with engineering review at that line.

1010. A 90-day plan

Days 1 to 30: baseline and find the constraint

• Pull baseline measures from existing tools: cycle time by stage, pull request size, review wait, change failure rate, rework rate.
• Map the value stream for the last 30 tickets and name the constraint, as in section 2.
• Settle the basics: approved tools, licences, and an instruction file in each main repository, as part 3 of the practical AI module describes.
• Agree the risk tiers from section 3 with security and compliance. Write them down.
• Choose two or three champions and give them time.
• Start the “not worth it before” list from section 5.

Days 31 to 60: build verification and pilot the new patterns

• Attack the constraint directly. If it is review, add an AI first pass and a pull request size limit. If it is testing, put agents on test backfill and CI speed.
• Close the biggest gap in “what must be true first” from section 3.
• Pilot spec-first work and background agents with the champions, on the low-risk tier only.
• Pick one long-deferred migration or debt item as a showcase, and finish it.
• Build a small eval set for your coding agent on your own repository, as part 9 of the practical AI module describes.
• Hold the weekly show-and-tell.

Days 61 to 90: spread and embed

• Extend the working patterns to the whole team.
• Codify what worked: an agent-ready ticket template, review-by-risk encoded as repository ownership rules, and a short team working agreement.
• Compare the measures with the baseline, using the two signatures from section 7.
• Find the new constraint. It will have moved, quite possibly out of engineering.
• Report upward: delivery outcomes, one or two stories of work that would never otherwise have happened, and cost per engineer.
• Plan the next quarter around the new constraint.

What to expect

Output usually dips in the first few weeks, while people learn and the verification work draws effort. Gains are uneven across people and across kinds of work. When the constraint leaves engineering, the conversation you need is with product, release management or compliance, and part 3 prepares you for it.

What not to do

• Do not set usage quotas or track individuals.
• Do not promise headcount savings.
• Do not remove review before the assurance stack is ready to replace it.
• Do not run the pilot on effort-saving mode alone, because it will only confirm the ceiling.