An agent finishes a task, writes "Done" and moves on. Later, someone checks the result and finds that the final path was never tested. The agent did something. It did not finish the job. That distinction is easy to miss when a response is polished and the activity around it looks busy. The user still carries the work of finding the gap and deciding what needs to happen next.
This is a problem I keep meeting in my own multi-agent setup. I want agents to move work without waiting for me at every step. I also need them to recognise when an answer is not enough. A task should close when the evidence meets its acceptance criteria, not when the author sounds certain. The product question is how to make that boundary useful without turning every small decision into an approval form.
My calibrated-decision diagram asks whether the agent should act, seek more information or defer to a person. It is a conceptual map. My operating rules use evidence gates, review and escalation. I am not claiming that the agents have learned a reinforcement-learning policy, that a confidence score is a calibrated probability or that this design has a measured reliability result. Those stronger claims would need their own evidence.
Match the proof to the claim
A useful completion statement has an appropriate test behind it. If the claim is that a file exists, inspect the file. If the claim is that a pipeline generates a usable output, test the relevant path and inspect what it produces. If the claim is that a service survives a restart, the evidence needs to address the restart. Each claim asks a different question, so a single generic success check cannot carry them all.
Consider an illustrative content pipeline. A summary and preview image are ready, but the output has not been exercised through the actual queue. That may be useful preparation. It does not establish that the pipeline publishes correctly. A safe test through the relevant route, followed by inspection of the resulting artifact and links, would address the stronger claim. This is a design example, not a report of a production incident I resolved.
The gate should not become a ritual of collecting screenshots. A screenshot can show a visible state while leaving delivery, permissions or behaviour unverified. The evidence needs to match the acceptance criterion. A task may require visual inspection, a current-state readback or a controlled test, and sometimes more than one. The point is to reduce what the owner must discover later, not to make the completion message look well documented.
Define the boundary before the final step
A task is easier to review when its acceptance criteria are clear before execution. What outcome is being prepared? What counts as complete? Which effects require a person's decision? If those questions remain vague, the agent may choose a convenient stopping point and call it done. The owner then has to reconcile the work with the original intention after the fact.
For the illustrative content task, drafting, approval, publication and verification are separate stages. The agent should know which stage it owns. A request for a draft does not authorise publishing it. A publication task is not finished just because a draft exists. Clear boundaries let the agent move the work confidently within scope and stop precisely where a new decision or permission is required.
I would also define the unresolved state. If a source cannot be verified, the agent should say what was checked and which claim remains unsupported. If a system cannot be tested safely, it should preserve reversible progress and explain the limitation. A blocked task is not a failed conversation. It can be a useful result when it prevents an uncertain interpretation from becoming a consequential action.
Act when the work can carry the decision
The agent should act when the task is within scope, the relevant inputs are clear and the evidence is adequate for the consequence. This is an operating rule, not a universal numerical threshold. A small internal formatting choice does not need the same review as a public factual claim. The decision should reflect what a mistake would make the user notice, repair or explain.
In an illustrative research note, the agent can organise verified findings into a draft without asking about every heading. That work remains reviewable before it represents the user publicly. The agent should still preserve source limitations and avoid inventing a conclusion. Autonomy within a bounded task is useful because it removes unnecessary coordination while leaving consequential choices visible.
The action boundary can also depend on current state. A task that was safe to perform earlier may no longer be appropriate after somebody else completes it or changes the destination. The agent should verify the state that drives the action at the relevant moment. Confidence about the original plan cannot substitute for checking whether that plan still fits the work now.
Ask when one missing fact changes the result
A useful question is about the missing fact the person can answer. It should not transfer the agent's entire reasoning burden to the user. The agent should first do reasonable research and distinguish a gap from something it can resolve independently. When several plausible interpretations would lead to different recipients, costs or commitments, a short clarification is cheaper than cleaning up a confident mistake.
For an illustrative scheduling request, two contacts with the same name may both fit. The agent should not choose the more familiar-looking record. It should recover context and verify the route where possible, then ask only if identity remains unresolved. The question should name the choices clearly. It should not bury the ambiguity in a long status report that the user has to audit before noticing the decision.
Asking should pause the affected action, not everything. The agent may still prepare options, gather sources or save a reversible draft. That is how useful autonomy and restraint fit together. A permission gap for an outgoing message does not necessarily prevent private research. The design should keep independent work moving while preserving the boundary around the effect the user has not approved.
Defer when the judgement belongs elsewhere
Some decisions cannot be settled by another search. The consequence may be high, the evidence may be inaccessible or a qualified person may own the judgement. Deferral should give that person a prepared decision: what was checked, what remains uncertain and which options are plausible. A bare "I can't" leaves the user carrying the research again and does little to reduce the task's burden.
For an illustrative live-system change, the agent might identify two possible remedies but be unable to establish which one is safe in the current environment. It can document the findings and preserve a backup or draft where authorised. It should not test a potentially disruptive option simply to avoid reporting a blocker. The boundary should reflect the consequence, not the agent's desire to finish the turn.
Deferral is also appropriate when the proposed action exceeds the authority received. A technically available action is not automatically within the task. The agent should not treat access as consent or another person's request as the owner's permission. The public design's act, ask and defer choices are meant to preserve this distinction rather than collapse every open issue into a confidence estimate.
A confidence number needs measured meaning
A model can print "76% confidence" without that number meaning that 76 out of 100 comparable decisions would be correct. I do not treat a self-reported percentage as a reliable permission gate. Calibration requires measurement against outcomes in a relevant setting. A confident explanation may be useful to read, but it does not establish the statistical meaning of its accompanying number.
Research on selective classification examines when a classifier can abstain rather than predict. Franc, Prusa and Voracek's work on reject-option classifiers makes the relationship between selection, coverage and risk explicit. [1] That research provides a useful lens for thinking about abstention. It does not prove that my agents have implemented those methods or that a rule-based escalation workflow is a trained selective classifier.
My practical starting point is simpler: which mistakes would I have to find and repair myself? That question helps identify the decisions that need stronger evidence or review. It is not a substitute for a formal evaluation if a product makes calibration claims. The design should keep its language honest about what is implemented, what is measured and what is only a conceptual direction.
Review should be proportional, not constant
Human review is useful at the boundary where evidence or authority cannot carry the next action. It is less useful when it asks the owner to approve every reversible detail. An agent that escalates all choices may avoid making mistakes while creating a new coordination job. The review design should therefore examine both missed escalations and unnecessary ones.
For the illustrative content pipeline, the owner may need to review the final public words and audience, while the agent can handle structure and source checks. For a private draft, the review boundary may be later. The task's consequence and permission determine the distinction. A universal approval step can be simpler to implement while being poorly matched to the work people actually want to hand off.
The broader analogy is a quality gate in manufacturing. A gate earns its place when it catches a defect before the next stage makes repair harder. Checking every item in the same way regardless of risk can slow the process without improving the relevant outcome. This is an explanatory analogy, not a claim about a manufacturing deployment. Agent oversight needs similarly clear criteria and a route for exceptions.
Verification should survive the handoff
Work can move between agents, models or people. A handoff should carry the original task, relevant restrictions, verified results and unresolved questions. It should not replace the evidence with a confident summary. Otherwise the receiving person may treat an inference as a fact or assume that an approval covers more than the user actually reviewed.
For the illustrative live-system task, the handoff should preserve what changed, what test was run and what remains unverified. A statement that the issue is "fixed" would be insufficient if only a setting was saved. The receiving reviewer needs the distinction between the action taken and the outcome established. That lets the next stage continue without repeating work or accidentally widening the claim.
The agent should also report corrections plainly. If a readback shows that an earlier completion statement was wrong, the next step is to correct the record and verify the state. It should not rely on the owner noticing the contradiction. An honest correction is part of the work. A system that hides uncertainty to maintain a smooth story gives the user more reason to audit every future result.
Stopping should preserve a usable next step
A stop should not discard progress or leave an unclear queue. The agent should save the reversible work, identify the blocker and say what would allow continuation. It should distinguish waiting for a person from waiting for an external event. The responsibility remains open until the outcome is complete or explicitly handed back, rather than disappearing when the agent reports a limitation.
For the illustrative publication task, the prepared draft and exact unresolved question can remain ready for review. The agent should not create repeated reminders without a reason, nor promise monitoring it cannot actually perform. A useful stop leaves the owner knowing what is safe, what is pending and which decision is theirs. That is different from a failure message that merely announces the task did not finish.
A controlled stop also needs to avoid duplicate effects. If an action timed out, the agent should verify whether it landed before repeating it. The missing confirmation does not establish that nothing happened. The design should carry uncertain-effect status into the next run or handoff. Otherwise a recovery attempt can create a new problem while trying to solve the original one.
Measure whether the gate catches the right failures
NIST's AI risk guidance recommends methods and metrics suited to the use case, tests of whether a system functions as claimed, documentation of errors and reassessment of controls as conditions change. [2] That supports treating the review boundary as something to evaluate. It does not certify my workflow or establish that any particular set of operating rules is sufficient.
A bounded agent trial could record appropriate actions, necessary questions, useful deferrals and cases where a human overturns the decision. It should also examine the burden of review. A high escalation count might reflect caution, poor evidence access or an overly broad task. A low count might reflect clear scope or missed risks. The number needs context before it becomes a success claim.
The trial should include failure cases rather than only well-chosen demonstrations. Unclear recipients, conflicting sources, missing authority and unavailable verification should all have an expected path. The public design reports no measured improvement from such a trial. My next interest is whether the gate catches the right problems without making the agent so cautious that it cannot carry ordinary work.
Widen scope only when evidence earns it
A review policy that works for a private summary cannot be copied unchanged to a public post, a customer commitment or a production change. The effect has changed, so the evidence and authority need to be reconsidered. An agent being useful in one area does not establish permission or competence in the next. Scope expansion deserves a decision, not an automatic promotion based on activity.
For founders building agent workflows, I would begin with one decision the system can make safely, one it should clarify and one it should defer. The trial should then establish how those boundaries behave. The purpose is not to make a grand claim that the agent handles everything. It is to develop a specific responsibility with a credible account of success, uncertainty and correction.
My view is that useful autonomy includes a useful stop. An agent should carry the research and preparation as far as the evidence allows, then leave a clear decision where judgement or authority is missing. If you are designing that boundary, I can help define acceptance criteria, failure cases and review rules. The first question is where a wrong action would create work somebody else must discover and repair.
Source notes
[1] Franc, Prusa and Voracek, "Optimal Strategies for Reject Option Classifiers," JMLR 24 (2023): https://jmlr.org/papers/v24/21-0048.html. Used for the narrow discussion of abstention, coverage and risk, not as evidence of an implemented agent policy.
[2] NIST AI RMF Playbook, "Measure": https://airc.nist.gov/airmf-resources/playbook/measure/. Used for use-case-specific measurement, testing claims, documenting errors and reassessing controls, not as certification of this experiment.