What an AI agent actually does with a document
An agent for document processing is not a chatbot. It is a small pipeline with a clear job:
- Receive a document from where it already arrives — a mailbox, a portal upload, a scanner, an ERP attachment.
- Read and classify it: what type of document is this, which customer or supplier, which process does it belong to.
- Extract the fields the process needs — parties, dates, amounts, references, line items — and check them against what the company already knows.
- Prepare the case: fill the record, attach the source, flag what is missing or inconsistent.
- Act or hand over: start the task in the ERP when confidence is high, or route it to a person when it is not.
The language model is one component in step 2 and 3. Everything else is ordinary, careful software engineering: queues, validation rules, integration with the system of record, audit logs.
Why pilots stall
Pilots tend to fail for the same handful of reasons:
- Sample documents instead of real ones. A pilot built on twenty clean PDFs says little about the scanned, rotated, handwritten-annotated documents that arrive on a Monday morning.
- No connection to the ERP. If the output is a spreadsheet that someone still has to copy into the system, the process has not changed.
- Accuracy judged by impression. "It looks good" is not a decision criterion. Without a baseline, nobody can say whether the agent is better than today's process.
- No owner after the demo. When the pilot ends, nobody is responsible for monitoring, exceptions and change requests.
Design principles that hold in production
Start from one case type
Pick a single, frequent, well-understood document flow — for example incoming purchase orders from one channel, or first notifications of loss for one line of business. Narrow scope makes the measurement honest and the integration manageable.
Keep a person in control of critical decisions
Define, in writing, which steps the agent may complete on its own and which require approval. A useful rule: the agent prepares, a person approves anything that commits money, changes a contract position or affects a customer. The boundary can move later, on evidence.
Make every answer traceable
Every extracted field should point back to the page and passage it came from. Reviewers check faster when they can see the source, and auditors ask for exactly this. The same principle underpins retrieval-augmented generation when the agent has to consult internal knowledge, not just the document in front of it.
Validate against the system of record
Extraction is only half the job. A supplier number that does not exist in the ERP, an amount that does not match the order, a date outside the contract term — these checks catch most errors before a person ever sees them, and they rely on a clean integration with your ERP and CRM.
Decide where the data goes before you choose the model
Contracts and claims contain personal and confidential data. Agree up front where documents and prompts are processed, who holds the keys and which suppliers are involved. For some organisations that rules out public AI services altogether and points to models running on your own infrastructure.
How to measure a pilot so the result means something
Before the first line of code, record three numbers for the chosen case type on today's process:
- Accuracy: the share of fields and cases that are correct without correction.
- Processing time per case: from document received to task started in the ERP.
- Manual interventions: how many cases need a person to key in or fix data.
Run the agent on real documents for the same case type and measure the same three numbers. If the result does not justify the next step, the pilot has done its job: it has saved you a larger investment.
From pilot to production
Moving to production is mostly about the parts a demo leaves out:
- Exception handling: a queue for documents the agent cannot process, with a clear owner.
- Monitoring: accuracy and volumes tracked over time, because document formats and suppliers change.
- Change management: a simple way for the business to adjust rules ("from next month, orders above this threshold need a second approval") without a new project.
- Increments: add case types one at a time, each with its own baseline.
Where to start
Choose one document flow that is frequent, costly to handle by hand and well understood by the people who do it today. Write down the baseline, the boundary between what the agent may do and what a person approves, and where the data may be processed. That one page is worth more than any model comparison.
If you are weighing this for your back office, our applied AI service starts exactly there: one real case type, a fixed-price proof of concept, and numbers you can decide on.