A confidence score is a system-generated measure of how strongly a model or rule supports a predicted label, extracted value or recommended result. It is not automatically the probability that the result is correct, and scores from different models or tasks may not be comparable.
How is a confidence score used?
- Accept results above an approved threshold
- Route uncertain results for review
- Prioritize exceptions by expected risk
- Compare performance by document or case type
- Monitor changes after model or data updates
Illustrative threshold outcome
Suppose 10,000 extracted invoice fields are processed, 8,500 score at or above a 0.95 threshold and 1,500 are routed to review. The illustrative automatic-processing rate is 8,500 ÷ 10,000 × 100 = 85%. This rate says nothing about accuracy until the accepted results are tested against known outcomes.
Confidence vs. accuracy
Confidence describes the system’s strength of belief for a particular output. Accuracy measures how often outputs are correct against a labeled reference. A model can be highly confident and wrong if its scores are poorly calibrated.
How should thresholds be set?
Use observed error costs, validation data and the consequence of a false acceptance or rejection. Set thresholds by task and field where needed, monitor overrides and recalibrate after material changes.
