Why six
When I write code, I have a luxury I can't afford with judgment: there's an objective truth. A test passes or it doesn't. A query returns the right value or the wrong one. For that kind of work one good model is enough, because I can check its output against reality rather than against taste.
The positioning of a site, its copy, the way text lands on a decision-maker — none of that has such an anchor. "Does this read as credible?" isn't a question with one right answer; it's an average across readers' heads I'll never see. Asking a single model here means getting a single taste, delivered with confidence. And I know how persuasively a model can defend a view that happens to be wrong. I wanted a distribution of opinion, not one voice. So I put six models on the same task at once.
The task was specific and uncomfortable: review the commercial and content positioning of my own website. Not proofreading, not a technical audit — the hard question of whether the text would convince someone weighing a serious engagement that I'm the right person for it. Precisely the kind of judgment where one opinion means little and where a person tends to hear what they want to hear.
How it was set up
The rules were the same for all six. Each model received identical inputs — the same site text, the same context, the same questions. None of them saw the others' answers; they judged blind and independently. The six came from different labs: Claude, two variants from the GPT family (one via the Codex tool), Gemini, Kimi and Fable. The two from the GPT family are close relatives that differed mainly in configuration — I left them in on purpose, to see whether even they would diverge.
Above them all sat one more model, in the role of coordinator. It didn't review the site itself — its job was to gather the six independent verdicts and synthesise them: where there was agreement, where conflict, what only one had said. That division matters. Let one model both review and synthesise, and you get its taste twice. Separating the judging from the assembling is a cheap insurance against one voice passing itself off as a chorus.
Technically none of this was rocket science — a few parallel runs from the command line and a single synthesis step. The value wasn't in the tooling; it was in the discipline: identical inputs and genuine isolation. Without isolation a panel of models collapses into an echo, the second nodding along to the first, and agreement then means nothing.
Where they all agreed: that's the signal
The strongest output of the whole exercise wasn't any single finding. It was the agreement. Independently, in different words and with different emphasis, all six models arrived at the same core: the site spoke in the language of the person who builds it, not the person meant to buy it. It described what I can make, rather than what the buyer gets out of it — and it kept its strongest evidence exactly where no one would look.
Six models that couldn't see each other landed on that sentence independently. Epistemically that is something different from the same sentence out of the mouth of one of them. A single model can have a blind spot, a style that tints everything it says, or simply bad luck with a phrasing. Six models from different families have their blind spots elsewhere. When they all converge on the same conclusion with no way to coordinate, the odds drop that it's an artefact of one particular way of thinking. Agreement across independent observers isn't a sum of opinions — it's a different class of evidence.
In practice this meant I stopped treating the list of unanimous points as opinion. I treated it as a brief. Those points weren't worth debating further; they were worth fixing.
Where they split: calibration, not chaos
More interesting — and, at first, more confusing — were the conflicts. The clearest was severity. On the very same thing the models differed like a strict examiner and a lenient one: one model was by far the harshest critic of the six, another by far the mildest. Had I read only one of them, I'd have walked away either sure it was a disaster or sure a coat of paint would do.
The key was what the conflict didn't mean. It didn't mean they disagreed on the problem — they agreed. They differed only in their estimate of how much the problem hurt. The harshest and the mildest model were pointing at the same spot; one called it a blocker, the other a missed opportunity. That isn't chaos, it's calibration. The conflict didn't tell me "we don't know"; it told me "here's a problem, and here's the range of how seriously to take it". And that range is itself information — a map of uncertainty you never get from a single model, because a single model gives you just its one point on the scale and hides from you that a scale exists at all.
I came to read model "personality" as a quantity to account for. One model leans toward strictness and safety warnings, another toward encouragement and finding the good. Neither is a fault — as long as you know which is which and read its verdict with the right correction. A panel gives you that correction for free: the strict and the lenient anchor each other. A single model has nothing to anchor its calibration against, so it quietly passes it off as absolute truth.
What one caught that the rest missed
There was a third kind of finding, and it's the best argument for the whole exercise. While five models focused on the big picture — positioning, audience, the order of the evidence — one noticed a small thing the others walked past. A tiny, concrete, easily fixed spot where the strongest content failed to reach the right destination. Nothing strategic. Just something that would have worked had it been a few lines elsewhere, and didn't, because it wasn't.
This is exactly what one model finds and another doesn't — not because it's smarter, but because it looks from a different angle and at a different resolution. Had I bet on one voice, even the best one, I'd never have gotten this find. It wouldn't have vanished for being wrong. It would have vanished because a big model preoccupied with the big picture tends to underline the small stuff and move on. The value of a panel isn't only that it tallies what everyone agrees on. It's also that it catches what only one saw — the thing that would drown in an average.
There's a boundary here worth admitting, though: a find from a single model is weaker evidence than agreement among six. So I treated it differently — not as a verdict but as a lead to check. A lone observation is a hypothesis, not a conclusion. But even a good hypothesis you'd otherwise never have gotten is worth those extra few minutes.
What I take from it
For subjective judgment, a panel of independent models is a better tool than one model you loop into an answer. Not because six heads are smarter than one — with a good enough model they need not be. It's better because it hands you a structure of uncertainty that a single model, by its nature, cannot. Agreement is signal. Conflict is a map. And a lone finding is a trail you'd otherwise have missed.
I have to add two things so this doesn't sound like a cure-all. First: the decision stayed mine. The panel sorted the evidence and showed me where it was firm and where it was soft — but what I do with the site isn't a vote of models. Follow only the harshest and I'd rewrite even what worked. Follow only the mildest and I'd fix nothing. The work was the synthesis, not obedience.
And second: this approach is pointless for a large share of work. When a task has an objective right answer — code, a calculation, a fact — six models are expensive theatre; one will do, plus a check against reality. The panel pays off precisely where the anchor is missing and where one confident voice most easily leads you astray: positioning, copy, strategy, anything decided by taste and judgment. Where no test can say "yes, this is right", the nearest substitute is the independent agreement of those who couldn't coordinate.
Wondering how to bring AI into your own decisions — and where it's just overhead? → AI consulting