The Agent Autonomy Budget

Imagine a career assistant that has read a user's work history and saved job search. It finds a promising role, rewrites a resume, fills an application, and reaches the final submit button. The model reports high confidence. The form is complete. Should the product click submit?

This is a hypothetical scenario. The portfolio describes Bragora as a user-side career workspace for achievements and job applications. It does not establish that Bragora submits applications, and this exercise does not claim that it does.

The product rule I would test is this: as an agent receives authority over actions that are harder to reverse, affect more people, or are harder to inspect, teams should reduce its autonomy unless evidence and containment improve. Broader autonomy earns its way back only when comparable tests show equal or better task outcomes without more unauthorized actions, correction work, or severe incidents.

Agent quality does not determine acceptable authority

An accurate recommendation and an authorized action are different product outcomes. A model can choose the right job and still apply before the user is ready. It can draft a useful message and still send it from the wrong identity. It can correctly identify a stale record and still delete evidence the user needed.

The NIST AI Risk Management Framework Core asks teams to document human oversight, intended use, knowledge limits, and how outputs will be used. That is a useful starting point because it separates capability from context. A system that is safe to summarize a document may be unsafe to act on instructions inside that document.

Industry guidance reaches a similar product conclusion. OpenAI's guide to building agents recommends human intervention for high-risk, sensitive, or irreversible actions and when failure thresholds are exceeded. OWASP's Excessive Agency guidance identifies excessive functionality, permissions, and autonomy as separate roots of harm. A better model does not remove those three design choices.

I encountered this boundary in an internal financial workflow. The original ambition could have turned the product into a replacement for an established accounting platform. That would also have expanded the data we retained, the permissions we needed, the controls we had to prove, and the behavior users had to change. We narrowed the product instead: prepare and validate the work, then connect and send it into the system that already owned the downstream record.

The thinner boundary was not a lack of ambition. It reduced the security, compliance, and adoption surface while preserving the part of the workflow where automation created value. In my product analysis, it also made recovery easier to reason about. Staged testing, role-based access, stronger authentication, auditability, source linkage, and downstream validation became product requirements rather than details to solve after launch.

The autonomy ladder turns a vague ambition into five product states

I would make autonomy a named requirement with five levels. The ladder is a proposed product artifact, not a scientific standard.

Some teams will want a Level 5 for unattended consequential action. I would treat that as an exception requiring its own safety case, not the natural end of the ladder. The point is to earn authority by task, not award it to a product because the word "agent" appears on the roadmap.

A budget score is useful only when hard stops can override it

For each action, rate seven factors from one to five. Four describe protective readiness: reversibility, observability, evidence quality, and scope containment. Three describe risk pressure: consequence, permission breadth, and exposure outside the product.

Use the score as a discussion aid:

Protective readiness = the average of the four protective ratings.

Risk pressure = the average of the three risk ratings.

The difference between those averages is a neutral discussion signal from minus four to plus four. It is not a launch threshold. A positive result means the team has more protective evidence to inspect, not permission to automate.

Do not pretend the arithmetic is calibrated. Its value is that design, engineering, security, and legal partners must expose their assumptions. A team that calls an action reversible should be able to demonstrate the undo path. A team that claims observability should be able to replay who authorized what, which tool ran, and what changed.

Hard stops override a favorable score. I would require explicit human confirmation when an action creates a legal or financial commitment, sends content under a person's identity, discloses sensitive data, changes permissions, deletes irreplaceable information, or affects another person's opportunity. I would also stop when identity is uncertain or the agent encounters untrusted content that could redirect its goal.

Anthropic's discussion of trustworthy agents frames agents as systems that plan, act, observe, and repeat with less human oversight. A blanket approval at the beginning is weak protection. A scoped plan approval can be useful when the user can inspect and edit the plan, intervene during execution, and face a renewed gate if the agent crosses into a new tool, destination, identity, or consequence.

A hypothetical career workflow shows where the level should change

Consider four synthetic tasks for the same user:

The final step changes the user's representation in the world. A mistaken summary costs attention. A mistaken submission may cost an opportunity, expose personal data, or create a record the user cannot retract. Model confidence does not make those consequences equivalent.

The product requirement should include metrics for both task success and control failure. I would track completion rate, user correction rate, unauthorized-action rate, approval reversal rate, recovery time, and incidents by severity. The ladder moves only when the evidence shows that users gain value without losing meaningful control.

Fewer prompts can be safer than careless prompts

The strongest counterargument is approval fatigue. If every harmless action produces a modal, users will click through without reading. That creates the appearance of control while weakening it.

Approval fatigue is a boundary-design problem. Teams can group low-risk actions into a time-limited grant, allow a user to pre-approve a narrow tool and scope, keep sensitive tools unavailable by default, and request confirmation only when the action crosses the agreed boundary. Sandboxing and least-privilege credentials can let an agent work freely inside a small area while blocking movement outside it.

Before an agentic feature enters a roadmap, write one authority statement. It should name the exact state the agent may change, the people affected, the reconstruction and reversal path, the boundary that requires renewed approval, and the evidence needed to move the task one level higher. If the statement cannot fit in a short paragraph, the proposed authority is probably too broad.

"Make it agentic" is not a product requirement. Name the action, assign its autonomy level, and define the evidence needed to change that level. Then test the whole path against adversarial inputs, because a sensible budget still fails if untrusted content can spend it.