AI Across the Software Lifecycle

011. The whole process, not the coding step

How small the coding step is

Surveys of how developers spend their time keep finding the same thing. An IDC survey in 2024 put application development itself at 16% of a developer’s time. The rest went to writing requirements and test cases, security, building and running the delivery pipeline, deploying, and monitoring. A Microsoft study of 484 of its own developers, published in 2025, found coding to be about 11% of the working week, a little behind meetings and messages, and found that the wider the gap between the week developers had and the week they wanted, the lower their productivity and satisfaction.

Those figures are for developers alone. Add the product managers, designers, testers and operations staff whose work is also part of delivery, and coding’s share of the total effort is smaller still.

The arithmetic

The rule is an old one from computing, Amdahl’s law: speeding up one part of a process helps only in proportion to that part’s share.

gain = 1 / ( (1 - share) + share / speed-up )

coding, 16% of the work, 2 times faster   ->    9% overall
coding, 16% of the work, made instant     ->   19% overall
every stage 25% faster                    ->   25% overall
every stage 2 times faster                ->  100% overall

The second line is the ceiling. If code wrote itself, in no time, for nothing, a team whose other work was unchanged would deliver about a fifth faster. Any larger figure has to come from the other 84%.

Try it on your own work

The figures below are a made-up feature that takes thirty working days, with building at a sixth of it. Put in how long each stage takes where you work, then move a slider to say how much of that stage’s time AI saves, and watch what happens to the whole. Start with the examples.

Two things are worth trying. Make building instant, and see that the feature still takes twenty-five days. Then put building back, and take a third off everything else: the feature arrives sooner than it did when the code wrote itself, and nobody wrote any code faster. The days are days of effort. Add the days a piece of work spends waiting between stages, which the next paragraphs come to, and coding’s share gets smaller still.

Three reasons the real gain is larger than the arithmetic

The sum above counts hours of effort. Delivery is worse than that in ways that favour the whole-process view further.

• Most elapsed time is waiting. Part 2 describes flow efficiency: in most teams a change spends well over half its life in a queue, waiting for a review, an answer, a test run or a release slot. Faster coding lengthens those queues. Work on the stage that owns the queue shortens them.
• Mistakes cost more the later they are found. A misunderstanding caught in discovery costs a conversation. Caught in review it costs a rewrite. Caught by a customer it costs an incident, a fix, a release and some trust. Help that improves the early stages pays back at every later one.
• The stages feed each other. A clear specification makes the build faster, the review easier and the tests obvious. A vague one slows all three, however good the tools at each.

What it looks like when it works

A study published in July 2026 followed a mid-sized company that set out in 2025 to double the merged pull requests of each engineer. Over 802 developers and nearly 200,000 pull requests it got there: 2.09 times the baseline by April 2026, among the largest gains reported from a real deployment. The title of the paper is the finding that matters here: AI writes faster than humans can review. The load on each reviewer roughly doubled. The gain was held only because review was rebuilt around automation, with automated review overtaking human review, while the rates of merging and of reverting stayed steady.

DORA’s 2025 research says the same thing across thousands of teams: AI amplifies what is already there. A team with a sound process gets faster. A team with a broken one gets a faster-filling queue in front of the break.

The stages, and where this part goes

The line is drawn straight and is really a loop. What is learned in operation, from customers and from incidents, is the raw material of the next round of discovery. The tags say what kind of work each stage is: deciding, making, checking and running. Most of the stages are not making.

022. Discovery

Reading everything

Discovery is limited by attention. A product manager with four hundred support tickets, twelve interview transcripts and a survey’s free-text answers will read some and skim the rest. A model will read all of it, the same way, in minutes. Useful jobs:

• Clustering and counting. Group the tickets by underlying problem, not by the label the customer chose. Say how many fall in each group, and quote examples.
• Finding what was not said outright. People describe their workarounds and frustrations far more readily than the feature that would remove them. A 2026 industrial study of interview transcripts from a security operations team had a model extract the stated requirements and also infer the latent ones, each linked back to the passage that suggested it, so that a person could check the inference.
• Joining sources. What people say in interviews, what they do in the usage data and what they complain about in tickets are usually held by three different teams. The retrieval of part 8 of the practical AI module and the connections of part 7 of the practical AI module put them side by side.
• Reading the old system. Much discovery is archaeology: what does the current system do, and why? A coding agent that can search and read the codebase will answer “what happens to an order when payment fails?” with the files and lines, in minutes, where the alternative was finding the one person who remembers.

Prototypes as questions

The cheapest way to find out whether people want something is to show it to them. When a working prototype took a fortnight, teams argued about one idea. When it takes an afternoon, as part 5 of the practical AI module says, a team can put three in front of users and learn from the reaction. This moves building into discovery, which is where it is most valuable, since the code is thrown away and the knowledge is kept.

What the evidence says about method

A controlled experiment in 2026 compared four ways of producing requirements: people collaborating without AI, people collaborating with AI support, a model generating requirements directly, and a model working from the transcript of people’s discussion. The best-rated results came from the combinations, where people did the talking and AI did the synthesis. The model alone did worse. A separate evaluation of seven models, across a hundred simulated interviews, found them limited at the interviewing itself: fluent in conversation, and poor at drawing out what a person has not yet put into words.

That matches the shape of the technology. A model is strong at condensing what it is given and has no access to what it is not given. The needs that matter most are often the ones nobody has said yet.

Cautions

• Quotes must be checked. A model summarising interviews can produce a quotation that nobody said. Require a pointer to the source for every quote, as part 8 of the practical AI module described for citations, and check them.
• Summaries average. The one customer in forty with a strange, important problem disappears in a summary of themes. Ask for outliers as well as clusters.
• Personal data. Interview transcripts and tickets are full of it. Settle where they may be sent before anyone pastes one into a tool.
• It agrees with you. Ask a model whether your idea fits the evidence and it will tend to say yes. Ask it for the evidence against.

033. Planning

Why this stage pays most

Part 3 of the practical AI module described working from a specification: intent, then specification, then plan, each reviewed by a person before the next. It was presented there as a way to get better work from an agent. It is also the highest-leverage use of AI in the whole lifecycle, for the reason in section 1: an error caught here costs a sentence. The engineer’s effort moves to the two ends of the work, saying precisely what is wanted and judging whether it was delivered, and this is the first of those ends.

What to use it for

• Finding the gaps. Give the model a draft specification and ask what is ambiguous, what is missing, and what question a developer would have to stop and ask on day three. This is the contractor test from part 3 of the practical AI module run in reverse, and it is cheap and consistently useful. Do it before the refinement meeting, not during it.
• Acceptance criteria. Turn a description into statements that can be checked, including the unhappy paths: the empty list, the expired session, the second click. These become the tests in section 6.
• Impact analysis. A coding agent can read the codebase and report what a change will touch: which modules, which callers, which tests, which other teams. People are poor at this in large systems, since nobody holds the whole thing in their head.
• Options. Ask for three designs with the trade-offs of each, not one. Record the choice and the reasons in a short decision record, which the next agent, and the next engineer, will read.
• Breaking work down. Small, independent pieces with a clear definition of done are what both agents and reviewers handle best, and part 2 explains why small batches matter more when AI is writing.
• The test plan, first. Decide how the change will be verified before it is built. If nobody can say, the specification is not finished.

Cautions

• Plausible is not right. A model will produce a confident, well-formatted plan for a system it has misunderstood. The plan is for a person to review, and the review is the point, as part 3 of the practical AI module said.
• Estimates. Models have no basis for estimating how long your team takes. Your own tracker does. Use the model to find comparable past work, and the history to estimate.
• Volume. It is now effortless to produce a twelve-page specification. Nobody will read it. Ask for the shortest document that answers the questions.

044. Building

Parts 3 to 5 of the practical AI module covered it: instruction files, briefs, skills, agent definitions and the working patterns that hold up. The only point to add is about proportion. Building is the stage where AI is already most used and where a further gain is worth least to the whole, by the arithmetic of section 1.

What building well does do is help the stages either side. Code written against a reviewed specification, in small changes, with tests, is quick to review and quick to verify. The practices of part 5 of the practical AI module matter as much for what they do to review and testing as for the time they save in writing.

055. Review

Why it is the constraint

Section 1’s study is the general case. Agents produce more changes, and each human reviewer has the same number of hours. Part 2 lists review first among the places the bottleneck goes. Whatever is done here decides whether the speed of building reaches the customer.

The division of labour

Part 5 of the practical AI module recommends review by a fresh session with no knowledge of the author’s reasoning, and review by risk: automated checks and AI review on everything, human reading concentrated on design and on the paths where a mistake is expensive. That is the working pattern.

Signal, or it is worthless

A reviewer that raises forty comments of which three matter is worse than none, because people stop reading. The evidence says this is the common failure. A 2026 study of code review agents on open-source pull requests rated the agents’ comments for usefulness and found that twelve of the thirteen agents averaged under 60% signal, and that among pull requests reviewed only by an agent and then abandoned, most had received comments that were under 30% signal. Pull requests reviewed only by an agent were merged 45% of the time, against 68% for those reviewed only by people.

Those are agents running on strangers’ projects with default settings. The remedy is the craft of this module applied to the reviewer:

• Tell it what to look for and what to leave alone. Style belongs to the formatter and the linter, as it always did.
• Give it the specification and the instruction file, so that it reviews against intent and not only against taste.
• Ask for findings ranked by severity, each with the failing case, and cap the number.
• Have a second pass try to disprove each finding before it is posted. A finding that survives is worth a person’s time.
• Measure it: what share of its comments lead to a change? That figure is the reviewer’s eval, in the sense of part 9 of the practical AI module, and it should be watched.

Review is not only for code

The same first pass works on everything else a team writes and another person must approve: specifications, designs, test plans, database migrations, infrastructure changes, documentation, the instruction files and skills of parts 3 and 4 of the practical AI module. A model that has read the specification is also well placed to answer the reviewer’s first question, which is whether the change does what was asked.

Authors should use it first. A change that has been through an AI review before a colleague sees it arrives without the small problems, and the human review is shorter and about the right things.

066. QA and testing

Why this stage multiplies the others

This course’s second idea is that verification converts compute into reliability. Part 5 of the practical AI module says the largest single factor in an agent’s success is feedback from the environment, which mostly means tests. Every test added makes every later agent run more reliable and every later review lighter. Work on tests is the investment that compounds.

What to use it for

• Backfilling tests on legacy code. The job nobody had time for. Done carefully, it turns code that nobody dares touch into code an agent can safely change.
• Tests from the specification. The acceptance criteria from section 3 become tests, written before the code, as the test-first pattern in part 5 of the practical AI module describes.
• Hunting for gaps. Meta has published its approach: generate small deliberate faults in the code that the existing tests fail to notice, then generate tests that catch them. Applied to over ten thousand classes in its Android apps, it produced several hundred new tests aimed at privacy faults, and engineers accepted 73% of the tests it proposed.
• Testing by using the product. A model that can see and operate an interface, as part 6 of the practical AI module described, can do exploratory testing: follow a user journey, try the odd inputs, report what broke with screenshots. It is slow and imperfect, and it finds the kind of problem that scripted tests never look for, including visual and accessibility faults.
• Test data. Realistic, varied, and containing no real customer’s details.
• Triage. Reading a failed pipeline run, deciding whether it is a real failure or a flaky test, and pointing at the likely cause. Flaky tests are a tax on every stage, and an agent can find and fix them in the background.

The oracle problem

A test needs to know the right answer. Testers call the source of that knowledge the oracle. When a model writes tests by reading the code, the code is its oracle, and the tests assert whatever the code currently does. They pass, coverage rises, and nothing has been verified. If the code has a bug, the bug now has a test defending it.

Three defences:

• Generate from intent. The specification, the acceptance criteria, the bug report, the documentation. Not the implementation.
• Check that tests can fail. Put a fault in the code and see that a test catches it. This is mutation testing, it is what Meta’s system automates, and it is the only real evidence that a test is worth having.
• Watch for tampering. Part 2 of the language models module describes reward hacking. An agent asked to make the tests pass may weaken the tests. The grader for coding agents in part 9 of the practical AI module checks that test files were not changed, for this reason.

The AI features themselves

A product that contains a model needs the evals of part 9 of the practical AI module as part of its QA, run in the pipeline like any other test. A change to a prompt, a skill or a model version is a release, and is tested like one.

077. Release and operations

Release

• Saying what changed. Release notes, change summaries for the change board, upgrade guides, all drafted from the merged work and the specifications behind it.
• Flagging risk. Which changes in this release touch payments, authentication or data migration? Which have thin tests? This is a classification job, and part 4 of the other AI models module describes the kind of model suited to doing it on every change.
• The deferred chores. Dependency upgrades, framework migrations and the removal of dead feature flags are well-specified, repetitive and verifiable by the test suite, which makes them ideal work for background agents. They are also what makes every later release safer.

Operations

The published industrial results are instructive for their modesty. Meta described in 2024 a system that, when an investigation opens, narrows thousands of recent code changes to a few hundred with ordinary rules, then has a model rank them. For 42% of investigations in its main web codebase the true cause was among the top five suggestions. That is far from solving the problem, and it is a large saving in the first minutes of an incident, when the search space is the enemy. Meta’s write-up is explicit about the risk: a wrong suggestion can mislead the responders, so results must be explainable and low-confidence answers are withheld. Microsoft has reported similar work, using a model to propose root causes across many thousands of cloud incidents.

Useful jobs, in rising order of risk:

• summarising an alert storm into what is actually wrong
• gathering context: recent deployments, related past incidents, the relevant runbook, who owns the service
• proposing likely causes, with the evidence for each
• drafting the customer update and, afterwards, the first version of the incident review from the timeline
• carrying out a remediation, which belongs behind the approval gates of part 9 of the practical AI module until a great deal of trust has been earned

The access this needs, to logs, metrics, deployments and tickets, is what the connections of part 7 of the practical AI module provide, and the read-only default there applies with full force to production.

Documentation, continuously

Documentation rots because updating it is nobody’s job. An agent in the pipeline can compare each change with the documents that describe the code, and propose the update in the same pull request. Part 8 of the practical AI module showed why this matters beyond tidiness: every retrieval system, and every agent’s instruction file, is only as good as the documents behind it.

088. Putting it together

Start at the constraint

Part 2 gives the method: take the last thirty completed pieces of work, find where they spent their time waiting, and go there first. For most teams in 2026 that is review, then testing, then decisions, which is to say planning and discovery. It is rarely coding.

The practical AI module, applied

Measure each stage

The course’s fourth idea applies at every row. Each stage has a figure that existed before AI: how long discovery takes and how often the wrong thing is built, how many questions surface after work has started, how long a change waits for review, how many defects escape to production, how long an incident takes to resolve. Record it first. A stage where AI has been added and its figure has not moved is a stage where AI has been added to the activity and not to the outcome.

The whiteboard version

Coding is about a sixth of the work. Make it instant and delivery improves by a fifth. Make every stage a quarter better and delivery improves by a quarter, and the gains compound, because good discovery makes planning easier, good planning makes building and review easier, and good tests make everything after them safer. Point AI at the whole line, start where the queue is longest, and measure each stage against how it was before.