Skip to main content
All plans Review runs in the dashboard editor. A reviewer accepts, rejects or corrects each row, then closes the Task with Accept. A standing rejection sends the work back to the original annotator.

Review

A review is a Task at a review stage over a revision someone else produced. The version graph decides authorship. Every principal whose events are in the revision’s commits is an author, and the claim path refuses them the review Task. A check that cannot finish refuses for everyone rather than guessing. There is one audited exception. A coordinator holding operate may direct a review Task to its own author; that claim records the commit the author is reviewing as selfReviewCommitId, the Task names who directed it, and a completed self-review keeps the field so an audit can find it. Routing applies the same rule before anyone claims: a pool route never seats an author at a review stage. A reviewer records verdicts per row and closes the Task once. The row verbs exist in the dashboard editor; there is no public route for them. A confidence score is not a verdict; uncertainty gates only decide routing. A verdict carries the hash it judged. Each row verdict names judgedAnnotationHash; the closing Accept names judgedAnnotationsRoot. The control plane compares the carried hash with the current one at submit time and refuses a verdict on content that moved. There is no path that records a verdict without that comparison. There is no send-back verb. Row verbs are accept, reject and correct; task-level Accept is the one closing act. A review that closes over a standing rejection carries the reviewer’s reason and dispatches a rework Task to the original annotator. Rework is a consequence of the verdict, not a second operator act. Review state is derived, never written. An annotation is accepted when an accept verdict binds its current hash, corrected when a correct verdict does, or rejected when a reject verdict does. With no matching verdict, it is unreviewed. accepted and corrected both count as approved; they differ in who authored the approved content. Editing the annotation moves its hash, so every earlier verdict stops applying and the annotation becomes unreviewed. Keeping review state out of the content hash is safe for the same reason. The judgment can never change the identity of what it judged.

Consensus and adjudication

Nitsor has no consensus or adjudication stage. The pipeline designer offers Annotate, Review and, where a model Agent can run it, Auto-label stages. It does not offer consensus-read or adjudicate, and it refuses to apply a pipeline that still carries either one, asking you to change the stage to Review or remove it. Work that reaches either stage cannot finish: a disagreement creates no Task for anyone to settle, and the editor cannot close a consensus or adjudication Task. Neither the editor nor the public API records a consensus vote or an adjudication decision. A review stage has one reviewer per Task.

Conflicting edits

Two branches can change overlapping content. POST /api/v1/version-graph/merges performs a three-way merge against the merge base. Non-overlapping changes are auto-merged and listed by kind (project, workflow, taxonomy, annotation, instructions, answer), and overlapping changes are reported as typed conflicts (attribute, class, existence, geometry, provenance, review-state, instructions, taxonomy, workflow). Voxel-level merging inside one mask is not attempted; a mask conflict is a geometry conflict on that annotation.

Review

How a review stage moves work

Review flow as builtAn Annotate stage submits to a Review stage. Review records accept, reject or correct per row; task-level Accept forwards the work, or, with a reason over a standing rejection, dispatches Rework back to the annotator.AnnotateSubmit: annotate commitReviewrow verbs; Accept all closesForwardreview commit, next stageReworkto the original annotatorstanding rejectionAccept closes the review and records its outcome

Row verdicts bind annotation hashes. Task-level Accept writes a review commit and routes the next Task.

Example policy questions

The pipeline definition answers some of these for you and leaves others to your written instruction. The second column says which.

Next step

Write the review stage for one representative structure in the vocabulary above, then read the guide to releasing a dataset.
PlansAll plans
Last verified2026-09-28