The Same-Model Ceiling

Abstract

This study tests whether three reviewers from other model families surface decision-relevant considerations that three matched reviews by the originating model do not, and whether incorporating those considerations produces analyses preferred under blind model assessment. Claude Opus 5 answered twelve strategic-analysis tasks on one constructed acquisition case; each answer was critiqued under two matched conditions, by three fresh Claude Opus 5 sessions and by GPT-5.6 Sol, Muse Spark 1.2 and DeepSeek V4 Pro. Eligible candidates, pooled under shuffled neutral identifiers, were grouped by two blinded model coders with a model adjudicator; a group containing only heterogeneous candidates counted as set-only. Claude Opus 5 rated each set-only group's decision effect and revised where a group qualified; two model scorers assessed each pair blind, with adjudication on disagreement. The critiques produced 216 candidates; 152 passed the precommitted screen, 74 same-model and 78 heterogeneous. Blinded coding identified 37 set-only groups, at least one in every task. Claude rated 35 at Level 2, a change to terms, evidence requirements or commitment conditions; two reached Level 1 and no group reached Level 3, a route change. The final blinded preference favoured the revision in eight of twelve tasks and the original in one, with three ties. The two primary scorers agreed in five of twelve pairs, and six of the eight supported outcomes were decided by the adjudicator, which also served as reviewer H1. A post-collection audit reclassified 20 eligible candidates and retained 31 of the 37 set-only groups; four newly eligible candidates were never coded, so the audit provides no complete preference-based sensitivity result. Every judgement came from a language model, scores compressed against the top of the 0-to-2 scale, and no matched same-model revision arm was run. The result is a descriptive pilot bounded to this case, this panel and this procedure.