I deleted my AI's self-check — and the output got better

A paradox I built myself

I keep a kind of personal operating manual for my AI — a document that tells the model, at the start of any job, how to think. Not what to do; that changes task to task. How to go about it: break the problem into checkable pieces, verify where being wrong is most expensive, and keep what you know separate from what you merely suspect.

I was rather proud of that document. It grew over months, and every section was a reaction to some concrete slip I'd watched the model make. Among other things, it contained a passage where, after drafting an answer, the model was told to attack its own conclusion — to look for how it might be wrong before showing it to me at all. And right at the end, a short self-check ritual: before you send the answer, run through this list of questions and make sure you missed nothing.

It sounded reasonable. Surely more care can't hurt. And that is exactly where I was wrong. It turned out those instructions made the model's output worse, not better. I had given the AI more rules about how to be thorough — and paid for it in lower quality and wasted computation. That's the whole paradox, and it took me a while to accept it.

Why the instructions made sense

I don't want to make it sound like I wrote those passages out of foolishness. When they were written, they made good sense — and for weaker or older models they still do.

Earlier generations of language models tended to produce fluent, confident text and hand it over without a single backward glance. An error in an intermediate step would ride all the way into the conclusion, because nothing stopped it on the way. When I explicitly ordered such a model to "stop now, go back, and try to break your conclusion," it often did catch a mistake. I was forcing a second look it would not have taken on its own.

The closing ritual worked much like a checklist for a pilot or a surgeon: not because the expert doesn't know their job, but because routine oversights happen to everyone, and a structured pass catches them. For a model that doesn't tidy up after itself, a forced final sweep is cheap insurance.

So I had written a manual for the tool I knew. The trouble is that the tool got swapped out underneath me.

What the new generation changed

A few weeks ago Anthropic released a new generation of its models, and with it a guide on how best to prompt them. I opened it expecting to learn how to polish my manual further. Instead I found a recommendation that ran straight against what I had written.

In its guide for the Opus 5 model, Anthropic advises removing explicit verification instructions. It literally names the kind of thing — "double-check this," "add a verification step," "have the result checked by another agent" — and says that on the new generation these lead to over-verification: the model over-checks, burns extra compute, and quality doesn't rise for it. That passage stopped me, because it described precisely what was in my manual.

The reason is that the new generation verifies itself, unprompted. It re-reads what it wrote. It checks its own arithmetic. It hunts for the edge cases where a solution breaks. It does all this as part of its ordinary work. When I add my own "verify" instruction on top, it doesn't add up to a better result — it adds up to a double check where mine only gets in the way of the model's.

The most convincing evidence didn't come from the article but from the vendor's own action. For the new generation, Anthropic removed over 80 percent of the system prompt of its own coding tool — and its coding evaluations showed no measurable loss. The people who build the model threw out four fifths of the instructions they themselves used to give it, and nothing broke. That's not an opinion from a forum; it's a decision by whoever built the machine.

The decision: delete

I took it seriously and deleted both passages from the manual — the section on attacking one's own conclusion and the closing self-check ritual.

Let me be precise about why. I didn't delete them because I suddenly believed self-checking is pointless. I deleted them because the people who build the model say so, and because it makes sense once you think it through. The gap between "I believe it" and "it matches how the thing actually works" matters to me — the first is a feeling, the second is a reason. Here I had a reason.

There's a counterintuitive lesson in it, the kind you have to live through to believe: more prompting makes a better model worse. On a weaker tool, an extra instruction fills a gap. On a stronger tool, the gap isn't there, and the instruction only takes up room, splits attention, and forces the model to do work it would have done better on its own. Piling on rules poses as care, but it's really distrust — and that distrust costs quality.

What I did not delete, and why

This is where the whole thing turns, because I didn't delete everything. I kept one single part on purpose, and to me it's the most important point in this piece.

I kept the section on epistemic labelling — the rule that, for every claim, the model must signal whether it is verified (I checked this here and now), believed (I have this from memory; it's probably right, but it may have changed), or guessed (I made this up to fill a gap). And that it must say the difference out loud, not keep it in its head.

Why this one? Because it catches an entirely different class of error than the model's self-check does. When a model re-reads itself and looks for mistakes, it looks for mistakes in reasoning: a mis-added number, a missed edge case, a step that doesn't follow from the last. It's good at that, and there I truly have nothing to teach it.

What it does not catch that way is a different kind of slip: a claim about an external fact, pulled from memory and passed off as certain. A library's version number. An interface parameter. A price. The date of an event. The wording of some rule. These things sound perfectly confident and are internally entirely consistent — the model has no way to disprove them by checking itself, because from its point of view there is no reasoning error. The error isn't in how it thinks, but in that it confuses "I remember this" with "I verified this." No amount of re-reading its own text will find that.

The only thing that catches this error is exactly that labelling. When the model has to say, for each fact, "this is only something I recall from memory, verify it," I get a warning precisely where I would otherwise have accepted a confident sentence as truth. The model's self-check doesn't cover this layer, because it isn't about reasoning — it's about honestly admitting what is knowledge and what is a guess.

The difference between what I deleted and what I kept fits in one sentence: the ceremony goes, the epistemic honesty stays. A ritual at the end of every answer merely stacked pointlessly on the model's own check. Labelling the verified against the merely believed solves an error the model can't see in itself — and that's why no improvement to the model replaces it.

What I take from this

The most useful lesson isn't about any particular model, because the specific numbers won't hold a year from now. It's about how to think about steering AI at all.

Prompt engineering is often understood as adding: more rules, more examples, more safeguards, longer and longer instructions. On a weaker tool that even works. But as the tool improves, the equation flips, and more instructions start to hurt more than they help. The craft isn't writing the most complete manual. The craft is knowing what to remove — which instructions now merely obstruct an ability the model already has.

And at the same time — this is the harder half — knowing what is irreplaceable. Not everything that looks like "one more instruction about being careful" is the same thing. Most such instructions really do just slow a good model down. But some solve an error the model fundamentally can't see in itself, because it lies outside its own check. Those must not be deleted, even though on the surface they look just as redundant as the rest.

Telling the two apart isn't an afternoon's work with a guide. It's work that requires understanding where the tool is strong and where it has a blind spot — and that can't be copied, it has to be understood. This exact kind of judgement is what turns "we have access to AI" into "we get value from AI." And it's exactly what the work I do for companies revolves around: not switching the model on, but knowing what to tell it and what to stop telling it.

Facing a similar decision in your own company? → AI consulting