The tool that wrote the code shouldn’t be the only one checking it

A developer asks an AI assistant to add a discount field to checkout. It writes the change, then writes tests for the change, and the tests pass. Everything is green. The question nobody asks is what those tests actually prove.

This is not an argument against AI coding tools. They are genuinely useful, and most good teams use them. It is an argument for an idea engineering settled long before AI: the thing that produces the work should not be the only thing that verifies it.

Independence is an old idea

Finance separates the person who approves a payment from the person who makes it. Companies hire external auditors rather than relying on their own finance team to sign off its numbers. Aviation and space programmes go further. NASA runs a dedicated Independent Verification and Validation programme for its critical software, and defines independence along three axes — technical, managerial and financial — with IV&V funded and managed separately from the project it reviews.¹

None of this assumes the original team is careless. It assumes something more basic: whoever produced a piece of work shares its assumptions, and assumptions are exactly where mistakes hide.

Why self-checking misses things

When the same process writes the code and the tests, the tests tend to confirm what the code does rather than what it should do. Research on AI-generated tests shows this directly. Konstantinou, Degiovanni and Papadakis studied whether LLM-generated test oracles capture the expected behaviour of a program or its actual behaviour, and found that they mainly capture the actual behaviour.² If the code has a bug, a test derived from the code is likely to encode the bug as correct.

Asking the model to review its own work helps less than you might expect. Huang and colleagues found that large language models struggle to self-correct their reasoning without external feedback, and that performance sometimes degrades after self-correction.³ Re-reading your own answer is not the same as having someone else check it.

People are affected too. In a user study published at ACM CCS 2023, participants with access to an AI assistant wrote significantly less secure code than those without — and were more likely to believe their code was secure.⁴ The risk is not only the error. It is the confidence that comes with it.

The signal at team level

These effects show up in delivery data. Google’s 2024 DORA report found that increased AI adoption was associated with real benefits — better documentation, code quality and review speed — but also with an estimated 1.5% decrease in delivery throughput and a 7.2% reduction in delivery stability.⁵ One explanation raised in commentary on the report is that AI makes larger changesets easy to produce, and larger changes have long been riskier to ship.⁶

The takeaway is not to use less AI. It is that faster code production moves the bottleneck to verification. If verification stays inside the same loop that writes the code, it scales with the same blind spots.

What independent verification looks like in practice

Independence is not about who is smarter. It is about checking from a different position. For a web application, that means four things:

A different source of truth. An independent check starts from what the user sees — the pages, buttons, forms and flows of the running product — not from the code that generated them. It asks “does signup work?”, not “does this function return what this function returns?”.

A different moment. Tests in the editor and in CI check a change before it ships. An independent check runs against the deployed environment, after every deploy and on a schedule, because that is where configuration, third-party scripts and real data meet.

A different owner. The check should not be something the author of a change can quietly edit to make pass. Changes to what is tested should be visible and approved, not a side effect of fixing the build.

Evidence, not verdicts. “Tests passed” is a claim. A screenshot, a trace, the console output and the failed request are evidence. An independent check is only as useful as what it can show.

Keep one check outside the loop

Use AI to write code. Use it to write tests, too — it is good at it, and those tests catch real problems. But keep at least one check that sits outside the loop: one that looks at the product the way a user does, runs whether or not anyone remembers to run it, and reports only what it can show.

The tools that write software are getting faster every month. The case for checking that software independently gets stronger at the same rate.

Next articleWhy test suites decay — and what it actually costs