Skip to main content
Planned — not available yet A model proposal is only useful if you know how often it is wrong. This page describes the guarantee Nitsor intends to attach to model-assisted prelabeling, and the check that refuses a job which cannot carry that guarantee. Most of what follows is a target contract: no public interface accepts a prelabel job, and the model execution behind the check is not available. The refusal rule itself is implemented and covered by automated tests, which is why it is described here in exact numbers rather than in intent.

​
A job that refuses to run

Model-assisted prelabeling means a model proposes labels before a person works on them. Attaching a statistical guarantee to those proposals requires calibration cases: held-out examples with trusted labels, used to measure how often the model is wrong before its proposals are trusted anywhere else. The guarantee is stated at a chosen risk level, written α. A smaller α is a stricter promise, and a stricter promise needs more calibration cases. The minimum is ceil(1/α − 1) — the smallest whole number that is at least 1/α − 1. At α = 0.05 that minimum is 19. Availability. The preflight check described here is implemented and covered by automated tests. It is not something you can run: no public interface accepts such a job, and the model execution behind the check is planned — not available yet. The numbers below describe the refusal rule itself, not a service you can call. Why refusal is the point:
  • A refused job produces no output at all. That is deliberate: a model with too little calibration evidence would otherwise return a degenerate prediction — “everything is foreground” — that looks like a result and passes casual inspection.
  • The refusal is recorded with its four values, so a later reader can see exactly why the job did not run instead of finding an empty gap.
  • The check happens before the model runs, so an under-calibrated job costs nothing and leaves nothing to clean up.
  • The guarantee vocabulary is false-negative and false-positive risk on the calibrated quantity. Those are the only quantities the calibration procedure controls.
  • Overlap scores such as Dice are not statistically guaranteed by this procedure. Treat any claim that a target Dice score is guaranteed as unsupported, on this site or anywhere else.

​
Why this sits with the review concepts

A refusal is a recorded outcome, in the same way a returned review is. Both say something happened and why, rather than leaving a gap that a later reader has to guess at. Read Review, consensus, and adjudication for the human half of the same idea, and Roles and provenance for why machine-authored work stays identifiable.

​
Common mistakes and limits

  • A model proposal is not an approval. It should remain identifiable as machine-authored work.
  • A statistical guarantee covers the calibrated quantity and nothing else. It says nothing about a metric it was not fitted to.
  • Calibration cases are not training data. Reusing training examples as calibration cases destroys the guarantee without producing an error.
  • No prelabel job can be submitted today, so none of the numbers above can be reproduced from outside a design-partner environment.

​
Next step

Read Review, consensus, and adjudication for how a proposal becomes a decision, then Plan an integration for the evidence a future prelabel interface would have to produce.
OutcomeState the calibration evidence a model proposal must carry before anyone reviews it.
AvailabilityPlanned — not available yet
AudienceML engineers, Data scientists, Reviewers, Technical leaders
PrerequisitesRead Review, consensus, and adjudication; Know the risk level your team would accept
Last verified2026-08-23