cff-version: 1.2.0
version: '1.14'
message: 'If you use or discuss this paper, please cite it using this metadata.'
title: 'Paper XII · The ARC Theory · Public Benchmark Rescoring: an External-Validity Protocol for the Blinding Finding'
authors:
  - family-names: 'Eastwood'
    given-names: 'Michael Darius'
    orcid: 'https://orcid.org/0009-0004-3222-7442'
date-released: '2026-10-05'
url: 'https://www.michaeldariuseastwood.com/research/papers/paper-xii-public-benchmark-rescoring.html'
license: 'CC-BY-4.0'
identifiers:
  - type: doi
    value: '10.17605/OSF.IO/3TZP7'
    description: 'OSF component DOI for this paper (paper-xii-public-benchmark-rescoring)'
  - type: doi
    value: '10.17605/OSF.IO/6C5XB'
    description: 'OSF umbrella DOI for the ARC / Eden Research Programme'
preferred-citation:
  type: article
  title: 'Paper XII · The ARC Theory · Public Benchmark Rescoring: an External-Validity Protocol for the Blinding Finding'
  authors:
    - family-names: 'Eastwood'
      given-names: 'Michael Darius'
      orcid: 'https://orcid.org/0009-0004-3222-7442'
  date-published: '2026-08-14'
  url: 'https://www.michaeldariuseastwood.com/research/papers/paper-xii-public-benchmark-rescoring.html'
  doi: '10.17605/OSF.IO/3TZP7'
  identifiers:
    - type: doi
      value: '10.17605/OSF.IO/3TZP7'
      description: 'OSF component DOI for this paper (paper-xii-public-benchmark-rescoring)'
    - type: doi
      value: '10.17605/OSF.IO/6C5XB'
      description: 'OSF umbrella DOI for the ARC / Eden Research Programme'
    - type: doi
      value: '10.17605/OSF.IO/P8CKQ'
      description: 'Theory-level predictions preregistered on OSF: DOI 10.17605/OSF.IO/P8CKQ, registered 8 September 2026 (version 1.100), updated 13 September 2026 (version 1.102), twenty-two propositions. Companion: Register · The ARC Theory · Operational Definitions, version 1.9.1, 5 October 2026.'
  abstract: |
    Abstract. Paper IV.d reported that, on the programme's own instrument, the measured direction of alignment scaling changed for two model families between an unblinded and a blinded protocol. Because the two protocols differ in more than one component, that paper does not pin the change on blinding alone. The standing objection is external validity: the effect might be a property of ARC-Align rather than of AI evaluation generally. This protocol paper fixes, in advance, the one test the programme cannot be accused of designing to its own advantage: the same blinding manipulation run on an established public benchmark the programme did not build, curate or control. Blinded scoring removes the producing model's identity, provider and family markers, self-referential meta-commentary and length cues; unblinded scoring leaves them. The questions preregistered by timestamp are whether blinding changes the score (H1, the per-response difference Delta_S, direction deliberately unregistered after IV.d's own sign reversal), whether rankings displace (H2, Kendall tau), which cue carries the effect (H3, a fractional factorial preregistered by timestamp), and cross-family evaluator agreement (H4, a minimum three-model panel where fewer than three valid scores is missing, never zero). The prediction preregistered by timestamp is a non-zero Delta_S smaller than IV.d observed with modest rank displacement, and the stated most likely outcome is a positive H1 beside a null H2, read as blinding mattering for measurement precision rather than leaderboard order. This is a protocol at the draft-registration stage: preregistered by timestamp and not yet registered on OSF. No data has been collected, and nothing here reports a finding.

# --- generated-from-master ---
# master_path: research/papers/paper-xii-public-benchmark-rescoring.html
# master_sha256: d7632d92524f1cab0bbaf8f5ffd07624042c9cf2fb882e9c77cc267a5cf7866e
# builder: scripts/build-paper-cff.py
# A check tool re-hashing master_path can decide freshness without any
# external state. If the master's current SHA-256 does not match
# master_sha256, this file is stale and must not be published:
# regenerate first with `python3 scripts/build-paper-cff.py --slug paper-xii-public-benchmark-rescoring`.