Reusable Gated Research Method · 可复用门控研究方法
Blind Evaluation Pipeline v1.0
盲评标准流程 v1.0
Method release 1.0 · 2026-08-27 · Status: Frozen / Released
Purpose
Turn a registered Unilanguage observation into a reproducible experiment without allowing outcome knowledge to leak backward into frozen rules, annotations or quality review.
目的
把已登记的 Unilanguage 观察转化为可复现实验，同时禁止结果信息反向渗入已冻结的规则、标注或质量复核。
01
Pipeline Contract · 流程契约
流程可以由一个自动化程序统一启动，但程序必须在每道 Gate 停止。只有在前置材料存在、哈希与状态检查通过、并且该 Gate 的指定授权者留下批准记录之后才能继续。技术检查通过不等于获得授权；自动跨越、推定批准或事后补批均无效。
REGISTERED → G1 🔒 → G2 🔒 → G3 🔒 → G4 🔓 → G5
Machine responsibility
Validate, hash, stop and report
The runner may create permitted artifacts, validate schemas, calculate checksums and report readiness. At every boundary it writes AWAITING_AUTHORIZATION and exits or pauses.
程序可生成获准材料、验证结构、计算校验和并报告就绪状态；在每个边界写入 AWAITING_AUTHORIZATION 后退出或暂停。
Human authority
Approve a specific frozen state
The authorized person approves an identified artifact set and checksum manifest. Approval does not cover later edits; any changed prerequisite returns the workflow to the relevant earlier gate.
授权者只批准由明确文件与校验清单标识的冻结状态。批准不涵盖后续修改；任何前置材料变化都必须退回相应的早期 Gate。
02
Five Non-bypassable Gates · 五道不可绕过 Gate
Gate 1 · Preregistration Freeze 🔒
Freeze the claim before outcomes exist · 在结果产生前冻结主张
Required artifacts: stable experiment ID and metadata; preregistration; operational specification (OPS); dataset source/sampling plan; exclusion, control, effect-size and classification rules; randomization method and seed-record policy.
Exit condition: all documents are complete, versioned, timestamped and hashed; the preregistration authority approves the exact manifest. After exit, amendments are append-only and cannot silently replace the frozen record.
必需材料：稳定实验编号与元数据、预注册、操作规范（OPS）、数据来源／抽样计划、排除／对照／效应量／分类规则、随机化方法与种子记录规则。
放行条件：文件完整、带版本和时间戳并已哈希；预注册授权者批准精确清单。放行后只能追加修订，不得暗中替换冻结记录。
Gate 2 · Annotation Freeze 🔒
Freeze A/B judgments while identities remain hidden · 在身份仍隐藏时冻结 A/B 判断
Required artifacts: blinded packages A and B; codebook and annotation protocol; annotator provenance permitted by the preregistration; completed annotations; annotation freeze/checksum manifest.
Exit condition: packages contain no analysis key or group identity; completeness and schema checks pass; all permitted annotation streams are separately labeled and frozen.
必需材料：A/B 盲标包、码本与标注协议、预注册允许的标注者来源、已完成标注、标注冻结／校验清单。
放行条件：标注包不含分析密钥或组身份；完整性与结构检查通过；所有获准标注流分别标明并冻结。
Gate 3 · Blinded Quality Review 🔒
Review reliability before learning the groups · 在获知组别前复核可靠性
Required artifacts: blinded agreement report; missingness, disagreement and protocol-deviation review; reviewer attestation that identities were unavailable; resolution policy fixed without outcome knowledge.
Exit condition: preregistered quality thresholds are evaluated and a blinded reviewer authorizes either continuation, a documented return to Gate 2, or termination. Group identity remains sealed.
必需材料：盲态一致性报告、缺失／分歧／协议偏离复核、评审者未接触组身份的声明、在不知道结果时固定的处理规则。
放行条件：完成预注册质量阈值评估；盲态评审者批准继续、书面退回 Gate 2 或终止。组身份仍须密封。
Gate 4 · Authorized Unblinding 🔓
Reveal only the approved mapping · 只揭示获准的映射
Required artifacts: Gate 1–3 approval records and immutable hashes; explicit unblinding decision; named authority, timestamp, purpose and scope; separate restricted identity mapping and analysis key.
Exit condition: an authorized unblinding log binds the revealed mapping to the already frozen inputs. The analysis key and group-identity mapping remain in the Private Research Package and are never publicly deployed.
必需材料：Gate 1–3 批准记录与不可变哈希、明确揭盲决定、授权者／时间／目的／范围、单独保存的受限身份映射与分析密钥。
放行条件：授权揭盲日志把所揭示映射绑定到已冻结输入。分析密钥与组身份映射继续保存在 Private Research Package，绝不公开部署。
Gate 5 · Final Classification
Apply the frozen decision rule · 执行冻结的裁决规则
Required artifacts: reproducible analysis output; effect sizes and uncertainty; deviations and limitations; analysis report; classification worksheet linked to the preregistered rule.
Exit condition: the authorized classifier records exactly one outcome—Supported, Not Supported or Inconclusive—and publishes the Public Research Record. A result may not be upgraded by substituting an exploratory analysis for the preregistered analysis.
必需材料：可复现分析输出、效应量与不确定性、偏离与限制、分析报告、关联预注册规则的分类表。
放行条件：授权裁决者只记录一个结果——Supported（支持）、Not Supported（不支持）或 Inconclusive（结论不确定）——并发布 Public Research Record。不得用探索性分析替换预注册分析来升级结论。
03
Artifact and Status Model · 材料与状态模型
Every experiment receives a stable identifier, an append-only version history and two publication zones. Gate state is explicit rather than inferred from which files happen to exist.
Stable identity · 稳定编号
UNI-EXP-NNN[-variant]
Reserve a number once. Never reuse, renumber or fill a withdrawn experiment's number. Rounds and variants are recorded in metadata, while every artifact carries its own immutable version.
编号一经保留便不再复用、改号或用已撤回实验的号码补位。轮次与变体写入元数据，每份材料有自己的不可变版本。
Explicit gate status · 明确 Gate 状态
DRAFT / FROZEN / APPROVED / BLOCKED
Each transition records actor, authority, UTC time, prerequisite manifest hash and decision. File presence alone never opens the next gate.
每次转换记录执行者、授权依据、UTC 时间、前置清单哈希与决定。仅有文件存在绝不能打开下一道 Gate。
Public Research Record
Auditable, safe-to-publish evidence
May contain preregistration, public-safe manifests and checksums, blinded protocols, public reconstructions, aggregate results, analysis code, reports, deviations and final classification—only after release review.
可包含预注册、公开安全的清单与校验和、盲标协议、公开重建记录、汇总结果、分析代码、报告、偏离与最终分类，但必须先通过发布审查。
Private Research Package
Restricted operational material
Contains raw restricted data, completed item-level annotations when restricted, analysis keys, A/B group-identity mappings, credentials and any re-identification risk. It stays outside the public repository and deployment.
保存受限原始数据、受限的逐项完整标注、分析密钥、A/B 组身份映射、凭据及任何再识别风险材料；始终位于公开仓库与部署之外。
04
Human and Exploratory AI Streams · 人类与探索性 AI 标注流
预注册的人类标注流与后来进行的探索性 AI 标注流属于不同分析。探索性流必须标明协议、模型／来源、时间、权限与限制；不得把它写成预注册人类标注，不得静默合并，也不得用于满足仅限人类标注的裁决规则。
| Field · 字段 | Preregistered human stream · 预注册人类流 | Exploratory AI stream · 探索性 AI 流 |
|---|---|---|
| Authority | Defined and approved before Gate 1 closes. | Requires a separately dated exploratory authorization. |
| Role in classification | As specified by the frozen decision rule. | Exploratory unless the preregistration explicitly authorized it. |
| Reporting | Primary/confirmatory label if preregistered as such. | Always labeled exploratory; report divergence from the human plan. |
| Traceability | Annotator protocol, dates, blind condition and freeze hash. | Model/version, prompt/protocol, dates, blind condition and freeze hash. |
05
First Reference Implementation · 首个参考实例
Experiment 001A
UNI-EXP-001A-R2 · M → HUMAN
Experiment 001A is the first reference implementation from which Pipeline v1.0 abstracts preregistration freeze, blinded A/B work, agreement review, authorized unblinding, public/private separation and explicit classification discipline.
边界
抽象方法，不改写历史
Pipeline v1.0 从 001A 抽象预注册冻结、A/B 盲态工作、一致性复核、授权揭盲、公开／私有分界与明确分类纪律。它不修改、覆盖或重新解释任何 001A 冻结记录。
06
Future Use and Versioning · 未来使用与版本管理
Register this Language Book observation as UNI-EXP-002 and execute Blind Evaluation Pipeline v1.0.
将这个 Language Book observation 注册为 UNI-EXP-002，并按 Blind Evaluation Pipeline v1.0 执行。
This command means “initialize the approved template set and begin Gate 1.” It does not authorize any later gate. The runner must still halt for every artifact check and recorded authorization.
这条命令只表示“用批准的模板初始化并开始 Gate 1”，不授权任何后续 Gate；程序仍须在每次材料检查和书面授权处停止。
Pipeline v1.1 · Compatible refinement
Additive clarification
Use for backward-compatible clarifications, optional fields or stronger validation. Publish a new method page, changelog entry and template directory; existing experiments keep the version they registered under.
Pipeline v2.0 · Breaking change
New contract
Use when gate semantics, required artifacts, authority roles or classification logic change incompatibly. Never migrate a frozen experiment silently; a new experiment or explicitly preregistered amendment is required.