Use case assessment
Is AI document classification worth building?
Depends on one number: how many categories, and how stable they are.
Twelve stable categories is a rules problem. Two hundred shifting ones is not. The assessment is mostly counting.
What it usually means
Sorting incoming documents — correspondence, forms, claims, records — into categories for routing, storage or processing.
Is it genuinely AI?
Sometimes, and the answer turns on two numbers you can establish this afternoon: how many categories, and how often does the list change.
A dozen categories that have been stable for years, distinguished by document type and a handful of terms, is a rules problem. Rules are auditable, they run for nothing, and when they are wrong you can see why.
Two hundred categories, or categories distinguished by meaning rather than form, or a list that changes as regulation does, is genuinely a model problem.
What the simpler version looks like
Take a thousand recent documents. Have someone classify a sample by hand and record how long it takes and how often they hesitate. The hesitation rate is the useful metric: if a human is confident, rules can usually encode the reason. If a human hesitates, a model will too, and you have found the cases that need review regardless of approach.
What it costs to run
A labelled training set, which is human time and is the largest single cost. Re-labelling as categories change. Monitoring for drift as document formats evolve. A confidence threshold and a review queue below it.
The review queue is not a failure state, it is the design. A system with no low-confidence path is one that is silently wrong.
Where it sits under the EU AI Act
Depends entirely on what the documents are. Internal records management is not Annex III. Classification that determines access to essential public or private services, or that touches migration and asylum processing, is squarely inside it.
This is a use case where the same technology sits on both sides of the line depending on the content, which makes the classification question one to answer early rather than late.
When it is worth building
When category count is high, the boundaries are semantic rather than structural, and volume justifies the labelling effort. When you have someone who will maintain the labelled set.
Not when the current pain is that nobody agreed the categories. A model trained on inconsistent labels reproduces the inconsistency at scale, and does it with confidence.
Common questions
When does document classification need AI?
When the category count is high, when boundaries are semantic rather than structural, or when the category list changes often. A dozen stable categories distinguished by document type is a rules problem, and rules explain themselves.
How do you test whether rules are enough?
Have someone classify a sample by hand and record how often they hesitate. Where a human is confident, rules can usually encode the reason. Where a human hesitates, a model will too, and those cases need review under either approach.
Is document classification high-risk under the EU AI Act?
It depends on the documents. Internal records management is not an Annex III use. Classification determining access to essential services, or touching migration and asylum processing, is inside it. The same technology sits on both sides depending on content.
Scoring 40 AI use cases in a regulated enterprise — how the same technology changed risk classification depending on what it processed.