Suppose your coding assistant wrote something and you have budget for two review passes. The fashionable advice says to make one of them a different vendor's model, on the theory that a model trained elsewhere has different blind spots. A paper posted on 1 October puts a number on that advice, and then spends most of its length explaining what the number does and does not show.

When Does a Second Model Help? Cross-Model Review in LLM Verification is a single-author preprint by Tae-Eun Song. The headline: one same-model review plus one cross-model review matched 85 of 150 planted errors, which is 56.7 per cent. Two same-model reviews matched 64, or 42.7 per cent. Fourteen points for swapping one reviewer, at the same number of calls.

I wrote on Wednesday about a cross-check that agreed on 932 when the answer was 220, and I ended up wanting exactly this experiment: what happens when the second opinion comes from a different vendor. This is the nearest thing to an answer I have found, and the answer is more interesting than "yes".

The setup

Thirty artifacts, all generated by Claude Opus 4.6 according to the author's run logs: ten Python modules, ten technical tutorials and ten presentation scripts. Each had five errors planted in it, one each of five types (a factual error, an internal inconsistency, missing context handling, a recoverability gap and missing functionality), for 150 in total.

Ten review conditions followed, 90 sessions apiece, 900 in all. Four used the generating model: review in the same session, a second pass in the same session, a sub-agent, and a fresh session with no access to the conversation that produced the artifact. That last one the author calls cross-context review, from an earlier preprint. The other six used outside reviewers at three tiers: GPT-5.4 as the top tier, Gemini 2.5 Pro as the capable middle and Gemini 2.5 Flash as the lightweight option, each with and without the original requirements.

The control the headline needs

Here is the comparison I would read first. The mixed pair beat two same-model reviews by 14.0 points, with a confidence interval of 6.7 to 22.0. Fine. But the paper also ran the mixed pair against two reviews by the top-tier outside model. On the 29 artifacts where no session failed, the mixed pair matched 82 of 145 errors and the two outside reviews matched 76. That is a 4.1 point gap with an interval running from minus 2.1 to plus 10.3, which is to say no detectable difference.

So the experiment cannot tell you whether the second model helped because it was different or because it was stronger. The author says this plainly in the abstract: model difference and reviewer capability are not separated.

The analogy that made this click for me: you swap your second proofreader for someone from another department, and the catch rate goes up. Then you learn the newcomer is also the most experienced editor in the building. You have learned that the swap helped. You have not learned whether to recruit from other departments or simply to hire better editors, and those lead to different budgets.

The lightweight tier settles which way to lean. The cheap outside model scored an F1 of 24.0 per cent, below the 27.1 the generating model managed reviewing its own work in the same session. A different vendor's model, on its own, bought nothing. Whatever is doing the work, a vendor logo on the invoice is not sufficient to get it.

What was actually significant

The paper tests nine conditions against fresh-session review, with a correction for running nine tests. Exactly one differs: the second pass in the same session, at 21.7 per cent against 28.6. Asking the same model to look again in the same conversation scored significantly below a single fresh-session review, and lowest of all ten conditions.

Everything else in the single-review table is a point estimate. The top-tier outside reviewer led at 32.3 per cent, and the paper is careful to say that a non-significant gap is no evidence the two are equal either.

Among single reviews, that leaves one result a team can act on today with real confidence, and it is a negative one. "Review it again" in the same chat is the option to drop.

Different findings, correlated misses

The two reviewers in the headline pair overlapped on 35 of the 85 errors either one found, a Jaccard overlap of 41.2 per cent. Same-model sessions overlapped with each other at 52 to 62 per cent, and cross-model pairs at 37 to 46. So a different model does find somewhat different things.

Two cautions sit beside that. First, the paper notes its unions fall below what you would expect if the two reviewers' misses were independent. Their blind spots overlap more than chance would give. That fits Kim, Garg, Peng and Garg's study of over 350 models, which found that on one leaderboard, two models that both err agree on the wrong answer about 60 per cent of the time, and that larger, more accurate models are more correlated.

Second, the best pair still missed 65 of 150 planted errors. A third model raised coverage from 56.7 to 58.0 per cent. Stacking reviewers runs out of road quickly.

Code one way, documents the other

Split by artifact type, the ranking flips. On code, the generating model in a fresh session scored 40.7 per cent against 37.2 for the top outside reviewer. On documents, all six cross-model conditions (30.1 to 42.1 per cent) beat all four same-model conditions (17.4 to 24.5).

These category figures carry no uncertainty intervals and the paper labels them descriptive. I would treat them as a hypothesis worth testing on your own material, and a cheap one to test.

The new kind of false alarm

Bringing in an outside reviewer adds a failure the same-model reviewer does not have. Of 1,995 false positives from the cross-model conditions, 187 were classed as model bias, and 136 of those were the reviewer insisting that a feature or tool the generator had used does not exist. The classification is keyword-based and unvalidated, so hold the counts loosely. The shape is plausible on its face: a reviewer with different training data flags what it has never seen.

For anyone wiring a second model into a review gate, that is a triage rule. A "this API does not exist" finding from the outside reviewer gets checked against the documentation before anyone acts on it.

Read the provenance section

The part of this paper I admire most is section 4.1. Before the analysis, the author audited all 900 session records against the raw files. One baseline run was excluded because 92 per cent of its finding descriptions appeared verbatim inside the script that wrote the files, so nobody could confirm they were unedited model output. Fourteen cross-model calls that had timed out or returned unparseable output had been scored as reviews that found nothing, and those came out too. The paper reports results both ways.

The limitations go on in the same spirit. The planted errors were injected by the generator model and never independently annotated. The automatic matcher will credit a finding for landing on the right line even if it describes the wrong problem, and was not checked by hand. All the artifacts are in Korean while 20 of 30 fresh-session reviews in the headline run came back in English, which the keyword matcher may under-credit. The three fresh-session runs varied widely, matching 56, 37 and 29 errors, and the paper says the headline pairing is among the higher of the available combinations. Records and scripts are available on request and are not public.

That is a long list, and I would rather have it than not. It tells you how hard to lean on each number.

Who this is for

It is for teams deciding how to spend a fixed review budget on model-written documents, tutorials and specifications, where the outside reviewer looked strongest, and for anyone who has a "review again" step in the same session and can now delete it.

It is not for anyone hoping a cheap second model gives them independence at a discount. In this data it did not. And it does not address the failure in Wednesday's post: a second model reading the same wrong definitions will check your work against the same wrong definitions.

What to do with this

  • Drop the same-session second pass. It is the one single-review condition that tested significantly worse.
  • Give the reviewer a fresh context. Send the artifact, not the conversation that produced it.
  • If you add a second model, add a strong one. Budget for capability first and treat vendor diversity as a possible bonus.
  • Route by artifact type and measure. Plant five known errors in a document you already understand and see which reviewer finds them. Thirty artifacts was enough for this paper to see a pattern.
  • Stop at two. The third reviewer added 1.3 points.
  • Keep a human on anything that matters. Sixty-five of 150 got past the best pair.

The experiment I would still like to see is the factorial one the author names as future work: several generators, several reviewers at matched capability, context varied independently. Until someone runs it, has any vendor shipping cross-model review in a coding assistant published which of the two effects they are selling?