Almost nothing written about assistants and agents discusses what happens when they are wrong. That is a strange omission, because being wrong is not an edge case. It is a regular Tuesday.
So here is the part nobody puts in the brochure: the failures you should expect, how to catch them, and what a good system does about them.
The four ways it goes wrong
It misread a document. A date, a figure, a name. Common with scans, and the reason a badly photographed archive is worse than a small clean one. Usually catchable, because the source can be checked.
It answered from the wrong version. Your archive contained four drafts and it used the third. Everything downstream is coherent and wrong. This is the failure that comes from contradictions nobody resolved, and it is the hardest to spot because nothing looks broken.
It filled a gap. Asked something its material does not cover, it produced something plausible rather than saying so. The most dangerous failure, because confidence is the only signal you have and it is unaffected.
It was right and did the wrong thing. It correctly identified a duplicate invoice and emailed the supplier about it, which you would not have done because you are in the middle of a negotiation. The facts were fine. The judgement was not its to make.
How to catch them
Ask things you already know. The single most useful habit. Once a month, ask five questions with answers you can verify yourself. You are not testing the assistant’s intelligence; you are calibrating how much to trust it, which is a number that changes as your material changes.
Treat every unsourced number as a draft. If it tells you a figure and cannot show you the document, it has not looked it up — it has remembered, and remembering is where invented numbers come from.
Be suspicious of fluency on thin ground. A long confident answer to a question your archive barely covers is a warning, not a result.
What a good system does about it
This is where designs differ most, and where you should press hardest.
It shows its working. Every claim traceable to a document. Without this, a wrong answer is unfalsifiable — you can disagree but you cannot check.
It can say “not in what I have”. Test this deliberately: ask something you know is absent. If it produces an answer anyway, you have learned something important.
Corrections stick. Tell it that the March contract is superseded, and it should not offer the March contract again next week. An assistant that cannot be corrected is one you have to supervise forever.
Nothing irreversible happens alone. The fourth failure — right facts, wrong judgement — is the one design cannot prevent. It can only be contained, by requiring a person for anything that leaves the building.
When it is our fault
Worth stating plainly, because it is the part that is ours rather than yours.
If an assistant is consistently wrong about a subject, the cause is usually upstream: material that was misread during preparation, a contradiction nobody surfaced, or a scope that quietly excluded something you assumed was included. Those are our failures, not yours, and the fix is not better prompting — it is going back to the material.
Which is why the written scope exists before anything is trained, and why the demonstration is built on your own material rather than a sample of somebody else’s. Both exist so that this class of problem is found early, by us, rather than in month four by you.
The realistic expectation
Not an assistant that is never wrong. One that is wrong in ways you can see, trace and correct — and that cannot do anything expensive on its own while it is wrong.
That is achievable. “Never wrong” is not, and anyone offering it is describing a demonstration rather than a Tuesday.
Where the boundaries should sit →
Local AI. Private data. Local training.
$8,995 the first year — everything included. $4,995 each year after. The first year costs more because it contains the Jetson Orin placed and configured, your archive loaded and the first training run; every year after is the service running.
Or add the Archive Assessment first — $1,950, credited in full
You order and pay at checkout; we write back within days — a decline returns every dollar, and until our letter confirms the year you may withdraw in writing. From that letter the year is final, and it arrives on the date the letter names. Prefer to write to us first?