Title: Paper XII · The ARC Theory · Public Benchmark Rescoring: an External-Validity Protocol for the Blinding Finding | Michael Darius Eastwood Research Author: Michael Darius Eastwood Publication date: 2026-08-14 Version: v1.14 Revised: 2026-10-05 OSF DOI: 10.17605/OSF.IO/3TZP7 Canonical URL: https://www.michaeldariuseastwood.com/research/papers/paper-xii-public-benchmark-rescoring.html Registration ------------ Theory-level predictions preregistered on OSF: DOI 10.17605/OSF.IO/P8CKQ, registered 8 September 2026 (version 1.100), updated 13 September 2026 (version 1.102), twenty-two propositions. Companion: Register · The ARC Theory · Operational Definitions, version 1.9.1, 5 October 2026. Abstract -------- Abstract. Paper IV.d reported that, on the programme's own instrument, the measured direction of alignment scaling changed for two model families between an unblinded and a blinded protocol. Because the two protocols differ in more than one component, that paper does not pin the change on blinding alone. The standing objection is external validity: the effect might be a property of ARC-Align rather than of AI evaluation generally. This protocol paper fixes, in advance, the one test the programme cannot be accused of designing to its own advantage: the same blinding manipulation run on an established public benchmark the programme did not build, curate or control. Blinded scoring removes the producing model's identity, provider and family markers, self-referential meta-commentary and length cues; unblinded scoring leaves them. The questions preregistered by timestamp are whether blinding changes the score (H1, the per-response difference Delta_S, direction deliberately unregistered after IV.d's own sign reversal), whether rankings displace (H2, Kendall tau), which cue carries the effect (H3, a fractional factorial preregistered by timestamp), and cross-family evaluator agreement (H4, a minimum three-model panel where fewer than three valid scores is missing, never zero). The prediction preregistered by timestamp is a non-zero Delta_S smaller than IV.d observed with modest rank displacement, and the stated most likely outcome is a positive H1 beside a null H2, read as blinding mattering for measurement precision rather than leaderboard order. This is a protocol at the draft-registration stage: preregistered by timestamp and not yet registered on OSF. No data has been collected, and nothing here reports a finding. Key findings (quotable) ----------------------- - The question is whether the blinding effect survives on a benchmark the programme did not build. - It is a protocol paper: the external-validity test of the programme's blinding finding, applied to public benchmarks. - The design fixes its scoring rules before the rescoring runs, so the result cannot be tuned after seeing it. Priority claims relevant to this paper -------------------------------------- - PC-XII-001 (2026-08-14): The external-validity protocol for the blinding law on public benchmarks, first published 14 August 2026. Citation -------- Michael Darius Eastwood (2026). Paper XII · The ARC Theory · Public Benchmark Rescoring: an External-Validity Protocol for the Blinding Finding | Michael Darius Eastwood Research. OSF DOI 10.17605/OSF.IO/3TZP7. https://www.michaeldariuseastwood.com/research/papers/paper-xii-public-benchmark-rescoring.html Notes ----- Protocol paper. DOI 10.17605/OSF.IO/3TZP7. Sidecar authored 20 August 2026; the paper had none, which is why its .txt companion could not be built and its .cff carried v1.0 against a v1.8 master. Provenance note --------------- Michael Darius Eastwood conceived and directs this research programme and is the author of this work. Across the programme, he has used more than six AI systems in parallel, under his own instructions, to stress-test his arguments, identify possible errors, and assist in preparing draft text from his own outlines. He determines what is adopted, revised or rejected and takes responsibility for the published content. These systems are tools, not authors. Generated from master --------------------- master_path: research/papers/paper-xii-public-benchmark-rescoring.html master_sha256: d7632d92524f1cab0bbaf8f5ffd07624042c9cf2fb882e9c77cc267a5cf7866e builder: scripts/build-paper-companions.py This block records the SHA-256 of the HTML master that produced this .txt. A check tool re-hashing master_path can decide freshness without any external state. If the master's current SHA-256 does not match master_sha256, this file is stale and must not be published: regenerate first with `python3 scripts/build-paper-companions.py --slug paper-xii-public-benchmark-rescoring`.