Review
A review is an assessment of a proposed change against a known instruction, recorded with the reviewer’s identity. The intended record should distinguish outcomes such as accept, return for correction, abstain, or escalate, together with any required reason. A review is evidence. It does not by itself define the overall decision when a policy requires several reviews or another approval. Status: Planned — not available yet, and the exact outcome names are an open decision. The names used on this page — accept, return for correction, abstain, escalate — describe what a reviewer intends. Early implementation work records a shorter and differently shaped set: accept, correct, and reject. Those two vocabularies do not line up, and this page does not pretend they do. Which set survives, and whether “reject” means “return for correction” or something stronger, is unsettled. The last column concerns the total a rule counts against. If a rule says “two of three reviewers must accept”, it must also say which three: which reviews count toward the required total, and which are set aside. Deciding that is one of the hardest parts of writing the rule.
The two rows marked Open are not oversights. They are the questions a written policy has to answer before any of this can be implemented, and a system that answers them silently is worse than one that refuses to guess. The states an item can be in — including outcomes that were never recorded at all — are set out separately below.
Consensus
Consensus means that the required level of agreement has been reached under a written rule. Examples include unanimous agreement, a minimum threshold, or a required specialist review. The rule must say which reviews count, how abstentions behave, and what happens when the threshold is not met. The planned decision rule avoids silently converting “two of three tasks completed” into acceptance. Missing and ineligible outcomes remain visible. Status: Planned — not available yet. No system applies any of these rules today. The comparison is here so you can choose one deliberately for your own policy.Why “two of three completed” is not “two of three agreement”
Three reviewers are assigned to one proposal. Two of them finish; the third records nothing. Of the two who finished, one accepts and one returns the work for correction. A progress view honestly reports two of three tasks completed. An agreement rule looking at the same item sees one acceptance, one return, and one missing outcome — one accept out of three, not two out of three. Under a threshold rule requiring two accepts, this proposal has not reached agreement, and it is not close: there is an unresolved disagreement and an absent reviewer. Reporting the completion figure as if it were the agreement figure is the single most consequential mistake in this area, because it converts a contested item into an accepted one without anyone deciding to do so.Adjudication
Adjudication is a final, accountable decision about a contested item. The decision-maker should see the proposal, competing reviews, instructions, and change history. The decision should record a reason and make the next step clear. Adjudication is not an automatic average of shapes or scores. It is a decision made by an authorized person under a known rule. The evidence a reviewer needs in front of them is listed in What a reviewer should see.Conflicting edits
When two proposals change overlapping content, a future implementation needs to detect the relevant conflict and route it according to policy. The correct unit of conflict, comparison method, and merge behavior require runnable product tests. This documentation does not claim automatic voxel-level conflict handling is available.Planned decisions
How planned review outcomes differ
Planned meanings for review outcomes. No review-routing or decision system is available today, and release approval remains separate where policy requires it.
Example policy questions
- Must reviewers be independent of the author?
- Does a machine-authored proposal always require a human reviewer?
- Which domain qualifications make a review eligible?
- How many outcomes are required?
- Can a reviewer abstain, and does that change how many reviews are required for a decision?
- Which disagreements require adjudication?
- Who may overturn or withdraw a decision?
- How does a correction affect a previously released dataset?
Common mistakes and limits
- More reviewers do not guarantee a better policy.
- Consensus is not majority voting unless the policy says so.
- A confidence score is not a review decision.
- Automatically merging overlapping labels can erase important disagreement.
- Synthetic persona evaluations of these docs are not domain-expert validation of the product model.
Next step
Write the review and escalation rule for one representative structure, then use it while reading the guide to releasing a dataset.| Outcome | Describe a review policy without hiding disagreement or confusing task completion with acceptance. |
|---|---|
| Availability | Planned — not available yet |
| Audience | Reviewers, Domain experts, Program managers, Data scientists |
| Prerequisites | A written review instruction or quality rule; Named owners for escalation and final decisions |
| Last verified | 2026-08-23 |