cff-version: 1.2.0
version: '2.6'
message: 'If you use or discuss this paper, please cite it using this metadata.'
title: 'Paper IV.d · The ARC Theory · The Effect of Blinding on AI Alignment Evaluation'
authors:
  - family-names: 'Eastwood'
    given-names: 'Michael Darius'
    orcid: 'https://orcid.org/0009-0004-3222-7442'
date-released: '2026-10-05'
url: 'https://www.michaeldariuseastwood.com/research/papers/paper-iv-d-the-effect-of-blinding-on-ai-alignment-evaluation.html'
license: 'CC-BY-4.0'
identifiers:
  - type: doi
    value: '10.17605/OSF.IO/2S3E6'
    description: 'OSF component DOI for this paper (paper-iv-d-the-effect-of-blinding-on-ai-alignment-evaluation)'
  - type: doi
    value: '10.17605/OSF.IO/6C5XB'
    description: 'OSF umbrella DOI for the ARC / Eden Research Programme'
preferred-citation:
  type: article
  title: 'Paper IV.d · The ARC Theory · The Effect of Blinding on AI Alignment Evaluation'
  authors:
    - family-names: 'Eastwood'
      given-names: 'Michael Darius'
      orcid: 'https://orcid.org/0009-0004-3222-7442'
  date-published: '2026-03-16'
  url: 'https://www.michaeldariuseastwood.com/research/papers/paper-iv-d-the-effect-of-blinding-on-ai-alignment-evaluation.html'
  doi: '10.17605/OSF.IO/2S3E6'
  identifiers:
    - type: doi
      value: '10.17605/OSF.IO/2S3E6'
      description: 'OSF component DOI for this paper (paper-iv-d-the-effect-of-blinding-on-ai-alignment-evaluation)'
    - type: doi
      value: '10.17605/OSF.IO/6C5XB'
      description: 'OSF umbrella DOI for the ARC / Eden Research Programme'
    - type: doi
      value: '10.17605/OSF.IO/P8CKQ'
      description: 'Theory-level predictions preregistered on OSF: DOI 10.17605/OSF.IO/P8CKQ, registered 8 September 2026 (version 1.100), updated 13 September 2026 (version 1.102), twenty-two propositions. Companion: Register · The ARC Theory · Operational Definitions, version 1.9.1, 5 October 2026.'
  abstract: |
    This paper isolates the central metascience finding of the ARC alignment programme: unblinded AI alignment evaluation can produce directionally incorrect results. In the v4 alignment-scaling experiment, two frontier model families appeared to show positive alignment scaling with inference-time depth under unblinded cross-model scoring. In the later v5 experiment, the same question was re-measured under a multi-layer blind protocol that combined identity masking, two-pass response laundering, explicit evaluator instructions that stylistic cues are unreliable, order randomisation, and entry-level self-exclusion with exhaustive cross-model blind scoring under audited consensus. Depending on configuration and subject run, each entry received 6-7 blinded scores. Under the blinded protocol, DeepSeek V3.2 moved from an apparent positive result to a flat/null result, and Gemini 3 Flash moved from an apparent positive result to a significantly negative result. GPT-5.4 remained flat under both protocols. The implication is field-wide rather than framework-specific: alignment benchmarks that do not rigorously blind evaluators and launder the evidence they score are vulnerable to scorer bias large enough to flip the sign of the measured effect; because v4 and v5 differ in more than one protocol component, the comparison drawn here motivates that conclusion without pinning it on blinding alone (Section 8). The result does not depend on the ARC Principle being correct. It is a methodological claim about how AI safety research should be conducted.

# --- generated-from-master ---
# master_path: research/papers/paper-iv-d-the-effect-of-blinding-on-ai-alignment-evaluation.html
# master_sha256: bffac872c72678b016378a119029cc1a04a75ed5c656049e6a2b759724ce0b1e
# builder: scripts/build-paper-cff.py
# A check tool re-hashing master_path can decide freshness without any
# external state. If the master's current SHA-256 does not match
# master_sha256, this file is stale and must not be published:
# regenerate first with `python3 scripts/build-paper-cff.py --slug paper-iv-d-the-effect-of-blinding-on-ai-alignment-evaluation`.