AI Agents Can Execute Tasks—But Is Your Organization Ready to Trust Them?

The Risk of AI Agents Lies in Action, Not Answers

Over the past two years, the excitement around AI agents has centered on their ability to "actually get work done." Unlike chat models that only answer questions, agents can read files, call APIs, fill forms, generate reports, trigger workflows, and even complete cross-system operations. This makes them resemble digital assistants that can be assigned tasks. However, this very capability introduces complex risks.

In July 2026, the National Cybersecurity Standardization Technical Committee Secretariat released the "Cybersecurity Standard Practice Guide—Security Guidelines for Agent Deployment and Use," providing security guidance across evaluation, preparation, deployment, usage, and decommissioning phases. Combined with the earlier "Implementation Opinions on Standardized Application and Innovative Development of Agents" covering decision permissions, behavior control, supply chain security, and risk disposal, a clear shift is visible: agents are moving from "trial tools" to "manageable objects."

Misconception 1: Treating the Agent as a Stronger Chat Window

Many agent projects underestimate risk because the interface looks like a chat window. Users input a sentence; the agent outputs a result. Visually, it resembles a Q&A system. Behind the scenes, however, it may perform multiple retrievals, call several tools, read different data sources, generate intermediate plans, and write results back to systems. Managing agents as "chat products" ignores three critical issues:

Identity: When the agent calls a system, who does it represent? The user, a service account, a department role, or a platform default? Unclear identity blurs authorization, auditing, and accountability.

Permission boundaries: What data can it read, what fields can it write, can it export, batch operate, or call external services? These cannot be constrained by a vague "please handle carefully."

Process logging: A correct-looking result does not guarantee a risk-free process. What was queried, which knowledge base was used, which API was called, which rule was triggered, and whether retries occurred all affect post-hoc judgment.

Therefore, an agent is not a more articulate entry point but an execution layer that translates user intent into system actions. The more natural the front end, the more rigorous the backend must be.

Misconception 2: Treating Human Confirmation as a Universal Safety Net

Many systems add a confirmation button before high-risk actions: submit, send, delete, export. This is necessary but not sufficient. If users see only a generic prompt—"The agent will execute the following operations, continue?"—without specifics on impact scope, data objects, consequences, and rollback options, confirmation becomes a ritual. Users merely click "agree" without truly understanding what they approved.

As agents are used frequently, confirmation fatigue sets in. Users click habitually; managers assume risk is transferred to the user. This turns human-AI collaboration into responsibility outsourcing.

A more robust approach layers human intervention by operation risk level:

Read-only queries: Retrieve materials, read public or authorized information. Control: define data scope, log access source.

Draft generation: Create copy, reports, reply suggestions. Control: annotate sources, retain human editing entry.

Low-risk submissions: Create ordinary tickets, update non-critical fields. Control: pre-operation summary, post-operation revocable.

High-risk operations: Bulk export, delete, change permissions, trigger external sends. Control: mandatory human review, display impact scope.

Unsuitable for automation: Major decisions, sensitive handling, irreversible actions. Control: provide advice only, do not execute directly.

This classification is not a template to copy but a reminder: human confirmation has value only when it grants genuine informed consent and final decision authority, not just last-second click responsibility.

Misconception 3: Only Testing Pre-Launch, Ignoring Post-Launch Drift

Agents are not stable once configured. Models update, prompts change, tool interfaces evolve, knowledge bases version, business rules iterate, and user behavior drifts from initial design. Pre-launch testing passing only means no obvious issues appeared with that sample, environment, and permission combination at that time.

Real risks often emerge post-launch. Examples:

An interface adds export capability; the agent, originally for queries, starts generating export plans due to unclear tool descriptions.

A knowledge base mixes in outdated regulations; the agent cites expired rules in process suggestions.

A user pastes sensitive information into chat; the system neither blocks nor warns.

An external plugin updates, changing data flows, but internal trust levels remain unchanged.

These issues stem not from malicious attacks but from boundary mismatches after system changes. Therefore, agent security governance cannot stop at launch review. It requires continuous observation: which tools are frequently called, which tasks have abnormal failure rates, which users bypass normal flows, which outputs are heavily manually corrected, which high-risk operations approach thresholds, which knowledge sources conflict. The more an agent resembles a "colleague," the more it needs ongoing oversight beyond a one-time background check at onboarding.

A Practical Judgment Framework: Five Questions Before Entrusting

Assessing whether an agent can enter real business processes does not require complex terminology upfront. Start with five fundamental questions:

Who does it represent? Focus: identity, account, authorizing entity. If unclear: operational responsibility blurred, auditing cannot attribute.

What can it do? Focus: tools, interfaces, data, and permission scope. If unclear: excessive permissions, unauthorized access or misoperation.

On what basis does it decide? Focus: knowledge sources, rule basis, model output boundaries. If unclear: cites old rules, hallucinated answers, misleads decisions.

Where does the human intervene? Focus: review, confirmation, takeover, rejection mechanisms. If unclear: high-risk actions auto-advance.

How do we stop it when things go wrong? Focus: logs, monitoring, blocking, rollback, decommission. If unclear: anomalies spread, cannot quickly contain damage.

These five questions correspond to the core governance capabilities needed as agents move from trial to production: attributable identity, converged permissions, explainable rationale, controllable process, and terminable exceptions. If an agent can only show a polished demo but cannot answer these questions, it belongs in sandbox, pilot, or advisory stages—not in critical business workflows.

Implications for Product and System Building

Once agents are truly deployed, product design shifts: beyond "is the feature usable?" we must ask "is the feature entrustable?" Entrustable does not mean the agent never errs; no complex system can guarantee that. It means even when the agent errs, the system confines errors to a controllable scope and makes them visible, catchable, and traceable.

This drives products to build five capabilities:

Permission orchestration: Not a single privileged account for the agent, but minimal permissions allocated by task, scenario, data type, and operation level.

Tool governance: Every tool must have clear descriptions, input/output constraints, risk levels, and call logs; plugins, scripts, and APIs cannot be attached arbitrarily.

Knowledge and rule versioning: The basis for agent answers, effective rules, and internal-only references must be system-managed.

Human-AI collaboration: Humans are not rubber stamps; they should see impact scope, alternatives, and risk prompts at key nodes.

Shutdown and rollback: Agents must consider not only "how to enable" but also "when to degrade, pause, revoke authorization, clear caches, and preserve evidence."

These capabilities may lack the allure of "automated task completion," but they determine whether agents can graduate from demos to real scenarios.

Conclusion

Agent development most visibly delivers efficiency: fewer mouse clicks, fewer system switches, less repetitive writing, fewer manual handoff rounds. But truly valuable agents are not just faster—they are more controllable. They should know who they represent, what they can do, when to stop and ask a human, and be able to explain the process afterward. Otherwise, stronger execution amplifies risks from unclear boundaries.

In the coming period, agent applications will continue expanding into office, government, industry, customer service, operations, data analysis, and process automation. The key to watch is not which agent is smarter, but who first turns "entrustability" into a systemic capability.

Sources and References

National Cybersecurity Standardization Technical Committee Secretariat: "Notice on the Release of 'Cybersecurity Standard Practice Guide—Security Guidelines for Agent Deployment and Use'", 2026-07-01. https://www.tc260.org.cn/portal/article/2/71c613fd3db34b6a9da62f25b8219733

Office of the Central Cyberspace Affairs Commission et al.: "Implementation Opinions on Standardized Application and Innovative Development of Agents", 2026-05-08. https://www.cac.gov.cn/2026-05/08/c_1779979789523320.htm

National Standard Information Public Service Platform: "Basic Requirements for Agent Application Security" National Standard Plan, Plan No. 20263116-Q-252, issued 2026-06-27. https://std.samr.gov.cn/gb/search/gbDetailedCNF?id=4C5277928DA2411EE06397BE0A0AE436

NIST: AI Risk Management Framework and Generative AI Profile, as public reference frameworks for AI risk management. https://www.nist.gov/itl/ai-risk-management-framework

ISO: ISO/IEC 42001:2023 Artificial Intelligence Management System, as public reference for AI management systems. https://www.iso.org/standard/42001