An assistant that only talks is safe in a boring way. The worst it can do is be wrong, and you find out immediately because you are reading the answer.
The moment it can act — send the message, make the booking, move the file, pay the invoice — the question changes entirely. It is no longer what does it know. It is what is it allowed to do, and how would I find out if it did something I did not want.
Most of this category is currently answering the first question loudly and the second one not at all.
Three kinds of action, and they are not alike
Lumping them together is the root of most bad design.
Reversible and private. Reading a document, drafting a reply, organising a folder, preparing a summary. If it gets these wrong you delete the output. Nobody outside knows it happened. These can happen freely.
Irreversible but private. Deleting something, overwriting a file, cancelling a standing arrangement. Nobody else sees it, but you cannot undo it. These need a confirmation, every time, however tedious.
Anything that leaves the building. Sending an email, submitting a form, making a payment, posting anything anywhere. Once it has gone it has gone, and the cost is borne in front of other people. These need you, individually, each time — and a standing permission for this category is a mistake dressed up as convenience.
The useful rule is not about how confident the system is. It is about whether the action can be taken back and who sees it.
The valve
There is a second permission question underneath the first, and it is the one people miss: not what the assistant may do, but what may leave.
Even an assistant that never sends an email on your behalf may want to consult a larger model elsewhere for a hard question. Doing so means some of your material travels. That is sometimes exactly what you want — a big hosted model is genuinely better at general knowledge — and it should never happen quietly.
The arrangement we use is a valve with three settings, and the reason it has three is that the honest answer varies by situation:
- Once. For this question only. Then the valve closes again.
- For a while. For this piece of work, then it closes.
- Always. A standing decision you made deliberately and can revoke.
The default is closed. That matters more than the settings: a valve that defaults to open is not a valve, it is a label.
Why “it asked me first” is not enough
Consent stops being meaningful when it is requested too often. An assistant that asks permission forty times a day trains you to approve without reading, which is worse than not asking, because now there is a record of you agreeing.
Good design therefore asks rarely and specifically. Not “allow access to your documents?” but “send these two paragraphs to an outside model to answer this question?”. The first is unanswerable. The second you can judge in a second.
What to insist on
If you are evaluating anything that can act on your behalf:
A written record of every action taken. Not a feed of activity — a record you can read afterwards, showing what was done and on what basis.
Nothing leaves by default. If the honest answer to “what goes out without asking” is anything other than nothing, you are being asked to trust a policy rather than a design.
A hard boundary you cannot configure away. Some actions should be impossible rather than merely discouraged. Payment is the obvious one.
The ability to say no and keep the benefit. If declining the outside-model question breaks the assistant, the valve was decorative.
The honest tension
Every constraint above costs convenience. An assistant that asked nothing and did everything would be more useful on a good day and considerably worse on a bad one, and you cannot know in advance which kind of day it is.
Our position is that the boring arrangement is correct: read and draft freely, confirm anything irreversible, and require a person for anything that leaves the building. It makes for a less impressive demonstration and a system you can still trust in month eight.
What to do when it gets something wrong →
Every promise, in plain words →
Local AI. Private data. Local training.
$8,995 the first year — everything included. $4,995 each year after. The first year costs more because it contains the Jetson Orin placed and configured, your archive loaded and the first training run; every year after is the service running.
Or add the Archive Assessment first — $1,950, credited in full
You order and pay at checkout; we write back within days — a decline returns every dollar, and until our letter confirms the year you may withdraw in writing. From that letter the year is final, and it arrives on the date the letter names. Prefer to write to us first?