AI Agents in Production: The Real Challenge Is Identity, Not Capability

Many teams integrating AI agents into business systems initially focus on capabilities: can the agent retrieve data, generate reports, trigger workflows, or call APIs? Once deployed, however, the trouble usually stems from a more basic question: who actually performed this step?

A person delegates a task to an agent, which then calls a knowledge base, a process engine, and third-party tools. The UI only shows "completed," but behind that result at least four identities are intertwined: the requester, the judging agent, the executing system account, and the affected data or business objects.

Traditional permission management answers "does this account have this permission?" With agents, the question becomes "why is this action allowed right now?" The two differ by only a phrase, yet they represent entirely different governance objects.

From "Login Identity" to "Action Identity"

In traditional systems, accounts and people are tightly bound: who logs in, queries, or submits can generally be correlated. Even service accounts usually have clear system ownership.

Agents stretch this chain. They can orchestrate multiple tools without the user continuously watching, and they may decide the next step based on retrieval results, process state, and preset rules. At that point, looking only at the final robot account that called the interface no longer explains the action's origin.

This is why agent security discussions have expanded from "is the output appropriate?" to tool misuse, identity and permission abuse. OWASP's Agentic Applications Top 10 specifically highlights tool misuse and identity/permission abuse as key concerns. The warning is not that agents cannot act, but that when models can plan, invoke, and delegate, the old permission narrative is insufficient. (OWASP Agentic Applications Top 10)

A Commonly Confused Fact: An Agent Is Not an Account

Treating an agent as just a new account is the most common and convenient approach. It gets the system running but easily compresses "who requested," "who decided," "who executed," and "who was affected" into a single log line.

A more realistic view: an agent is an action orchestrator, not a natural bearer of responsibility. Each of its actions should be traceable back to four identity layers:

Initiator Identity : Who proposed the task, and is there an authorization source? If missing, a system auto-trigger may be mistaken for an explicit personal instruction.

Decision Identity : Which agent, version, or rule made the action choice? If missing, model judgment cannot be distinguished from a fixed workflow.

Execution Identity : Which tool, interface, or service account actually performed the operation? If missing, only a machine account is visible, and the action chain cannot be found.

Impact Identity : Which data, objects, or external systems were reached by the action? If missing, the true scope of a single invocation is underestimated.

This framework is not a new compliance clause but an observation framework for discussing "action attribution." Its value lies in restoring a vague "the agent did something" into explainable relationships: who handed over intent, what basis was used for judgment, whose hands executed, and who bears the result.

Permissions Haven't Disappeared, They've Just Become Contextual

Without agents, permissions are mostly static: a role can access certain data, an account can initiate certain operations. With agents, permissions increasingly resemble conditional passes.

The same "query" action may only be allowed for handling the current ticket; the same "send" action may only be allowed after human confirmation at a specific stage; the same "download" action may vary with data sensitivity, access location, and task validity period. The question is no longer just "have or have not," but "under what conditions, for what purpose, affecting what scope."

This does not mean every agent must become a complex approval machine. On the contrary, good boundaries should let low-risk, repetitive actions flow smoothly, reserving human confirmation for high-impact nodes. The key is not lengthening the process but letting the system know when to hand the action back to a person.

A lightweight "four questions" check can evaluate whether an action should proceed:

Whose task is this: is there a clear initiator or valid system trigger?

Why execute now: do current state, time, and business conditions match?

What will it touch: are data scope, tool capabilities, and impact objects bounded?

How to explain afterward: does it leave enough evidence of intent, version, tools, and results for replay?

These four questions do not replace permission systems, audit systems, or business rules. They act as a ruler: judging whether existing controls manage an account or a real, occurring action.

The Most Dangerous Isn't "Overreach," But "Looking Like No Overreach"

Many risks are not triggered by a single obviously abnormal instruction. Instead, they happen when every step looks normal: the initiator has access rights, the service account has interface permissions, the agent's tools report no errors.

But if the task purpose has shifted, the context has expired, or the service account's scope far exceeds what the current task needs, these "normal steps" stacked together can form an action that should never have happened.

This is where agent governance differs most from traditional account governance: it must watch the combination of permissions, not just the existence of individual permissions. Past control points were at the entrance; now we must also see the middle orchestration, context switches, and cross-system landing points.

China's opinion on deepening the "AI+" action calls for broad, deep integration of AI with various fields while emphasizing safety and controllability. For organizations putting agents into real processes, "safe and controllable" ultimately means being able to explain, constrain, and pause or recall specific actions when needed — not staying in a generic capability description. (Opinion on Deepening the Implementation of the "Artificial Intelligence+" Action)

Responsibility Boundaries Shouldn't Wait Until Problems Surface

A mature agent system may not make every judgment perfect, but it should at least not let critical actions become "the system did it on its own."

Here is a practical shift: make a few small things beyond "what the agent did" part of the product experience.

Users can see who the agent is acting for, rather than assuming it inherently holds all permissions.

Managers can trace which tools and versions an action passed through, not just get a final result.

Affected business objects are identified, so the system doesn't quietly slip broad permissions into a seemingly lightweight chat entry.

These designs may be inconspicuous, but they determine whether an organization retains enough certainty as agents move from demo to daily use. NIST's Generative AI Risk Management Profile views risk management across the full lifecycle of design, development, use, and evaluation; for agents, identity and action boundaries are the thread connecting these stages. (NIST AI RMF Generative AI Profile)

The real question worth watching may not be when agents can complete more tasks for people, but whether, with each additional task completed, we can still clearly answer: who do they represent, on what authority, and whom do they affect?

This does not mean using governance to suppress agent value. On the contrary, only when the identity and boundaries of actions are clarified will organizations dare to put agents into truly important workflows.

Sources and References

Cyberspace Administration of China: Interim Measures for the Management of Generative AI Services — for understanding public requirements on transparency, accuracy and reliability, input information, and usage record protection.

State Council: Opinion on Deepening the Implementation of the "Artificial Intelligence+" Action — for understanding the policy background of AI's deep integration with industries and safety/controllability.

OWASP: Top 10 for Agentic Applications — for understanding agent security discussions on tool misuse, identity and permission abuse.

NIST: AI RMF Generative AI Profile — for understanding the public framework for generative AI full-lifecycle risk management.