AI Can Do It, But Should It? The Responsibility Gap in Enterprise AI Adoption

Core Thesis: Capability vs. Authorization

Technical capability determines whether AI can perform a task; authorization and responsibility determine whether it may enter the business process. These two gates do not pass in the same review: every model upgrade pushes the capability frontier forward, while the authorization frontier often stays put.

Guojin Securities' 2025 annual report illustrates this duality: the firm deployed GPT-based employee assistants, intelligent Q&A, and writing tools while simultaneously establishing a hallucination-tiered prevention mechanism, monitoring abnormal inputs/outputs, and advancing permission governance.

Decide Per Action, Not Per Role

A single role (e.g., customer service) contains actions with vastly different permissions and consequences: querying orders, drafting replies, sending notifications, modifying delivery dates, approving compensation. Automation must be evaluated at the action level.

Human-factors research by Parasuraman, Sheridan, and Wickens (2000) long ago advised separating four stages: information acquisition, information analysis, decision selection, and action implementation. Information can be gathered automatically, options compared automatically, but certain decisions remain human, and approved actions can then be executed by the system.

Illustrative Scenario: Rush Order in Manufacturing

A customer requests a quantity increase while keeping the original delivery date. A digital employee pulls data from inventory, scheduling, and logistics systems and produces a feasible plan. Sales wants a quick reply; production worries about disrupting existing orders; quality must confirm whether the batch can be released.

Querying inventory, calculating gaps, and drafting a reply can run automatically within approved scope.

Booked inventory ≠ available for this order; a computed early delivery ≠ confirmed capacity.

Reallocating another customer's stock, adding overtime beyond budget, or changing prior commitments each require a person with the corresponding authority. Sales cannot approve quality release; a digital employee cannot turn "plan feasible" into "company commits."

The real boundary is at each action and its key judgment points. One action being automatable does not mean the whole role can run unattended; one judgment requiring a human does not mean the entire chain must pause for a signature.

Five Questions Before Delegating an Action

Authorization: Who approves this action? What objects, resources, scope, and duration are permitted? A system account, API access, or past success does not substitute for a current, explicit authorization. Rush-order evaluation permission allows verification and suggestion only — it does not automatically extend to inventory allocation or external commitment.

Consequences: If it goes wrong, can it be reversed? Is the loss containable? Will the impact amplify down the work chain? An internal calculation draft is editable; shipped goods and customer-adjusted production plans are far harder to undo. The harder the reversal, the earlier the corresponding judgment and risk acceptance must be placed before execution. Restoring a system configuration does not erase business consequences that have already occurred.

Rules & Evidence: What conditions count as "met"? Where does the evidence come from? Is it still valid? When inventory records conflict with sales data, the model must not pick the number it "trusts" and proceed; the person who owns the facts must verify and update the record that subsequent actions rely on.

Exceptions & Trade-offs: Is this action still within pre-approved rules, or does it require sacrificing another commitment, breaching a cost ceiling, or changing a professional standard? Routine trade-offs can be pre-authorized; major conflicts outside the rules need a human with the authority to decide — not a model masking the cost-bearer with an "optimal" solution.

Reserved Matters: Do laws, industry regulations, or valid enterprise policies require this judgment to be made by a qualified, authorized person? For significant rights, safety, or major customer commitments, verify the specific requirement. Model accuracy cannot substitute for these conditions, nor can a "high-risk" label alone justify blanket human review of every action.

Enterprises must codify these questions into business-specific judgment criteria: business and professional owners define what counts as low risk, how reversibility is proven, what independence requires, and list the needed evidence. The person with approval authority then confirms. Participating in drafting the criteria does not grant approval rights.

Model improvements change reliability and monitoring methods, but they do not self-grant permissions. Previously manual routine actions may shift to automated paths once evidence, controls, and formal authorization are in place. Boundaries are adjustable, but every adjustment must have a documented basis and an accountable person.

Verification, Acceptance, and Accountability Are Distinct

After AI takes over a multi-person workflow, the process often looks smooth: it generates a result, self-checks, and reports "done." The enterprise must still ask what that report proves.

Verification checks whether the result matches agreements and evidence is sufficient.

Acceptance is the authorized entity deciding, based on that verification, whether the delivery can be recognized and whether any conditions remain unmet.

Accountability follows role, behavior, and causality when something goes wrong.

These three are linked but cannot be collapsed into a single "check passed."

In the rush-order example, the digital employee's quantity matching the order supports verification only. It cannot declare capacity confirmed or the customer's acceptance of the new date. Self-checks by the executor can catch errors, but items requiring independent verification must still be checked by a qualified separate node or an approved independent rule/assessment mechanism. Swapping the model name while sharing the same faulty data and judgment gap does not establish independence.

Automatic acceptance is permissible only when: explicit authorization exists, risk is low, the action is reversible, judgment rules are clear, and applicable independence requirements are satisfied. Judgments reserved by law or policy remain with authorized humans.

Example: An internal gap-calculation result with a clear data standard, re-calculable for correction, and passed through an approved independent check can be auto-accepted within its defined range — no manager click needed. A major customer delivery-date change, however, must first obtain the corresponding judgment and valid agreement; a passed calculation does not make it externally binding. Sending a standard notification per approved content is managed separately from making a new commitment.

Automatic execution, independent verification, and authorized acceptance each have their own enabling conditions. Completion of one does not confer the authority of the next; these conditions should not be bypassed just to leave a human signature on the record.

Before Removing a Review Step, Know What It Guarded

A single review may simultaneously serve professional judgment, defect interception, conflict-of-interest isolation, and talent development. Before eliminating or merging it, determine: what risk it mitigates, whether any mandatory retention requirements exist, whether the replacement can reliably cover the original function, and who has the authority to approve the change. Adding a new sign-off demands the same scrutiny: what gap does it fill, why can this signer fill it, and does it duplicate existing controls?

If a reviewer only sees the model's conclusion without raw evidence, judgment timing, or veto power, the organization must first fix that control point — either by restoring information and authority, or by verifying the alternative and obtaining approval. One cannot infer that the original risk has vanished simply because "signing off is now ineffective."

Professional review effectiveness is measured by its actual ability to identify and block problems. Pre-delivery quality release, if it requires a qualified person's standards-based judgment, cannot be cancelled just because scheduling suggestions are accurate. Conversely, an internal gap calculation that has already passed approved verification and auto-acceptance should not receive an extra, baseless per-order signature merely because a manager feels uneasy.

The EU AI Act Article 14 on human oversight for high-risk systems offers a useful reference: oversight intensity must be proportionate to risk, autonomy, and context; the overseer must be able to understand the output, intervene, or stop the system. This does not set Chinese compliance specifics, but highlights a management priority — retained humans must genuinely possess oversight capability.

Cost-benefit must be calculated across the full delivery chain. Removing a review that shifts more reconciliation, rework, and remediation downstream may not save cost. Adding a sign-off that adds no detection capability only increases wait time. The proper comparison covers end-to-end delivery time, quality, risk, expense, and personnel burden.

Exception Handling: Who Can Actually Take Over?

An authorization granted at approval time does not guarantee its premises hold forever. Data-definition changes, model/rule upgrades, resource failures, or recurring similar errors can all demand re-verification of the automation boundary.

Returning to the rush order: if production confirms a make-up run but logistics cannot secure a truck, the system must pause downstream actions and unsent notifications that depend on that transport arrangement, preserve records of already-effective actions, and clarify resource consumption and pending consequences. Logistics personnel verify alternative capacity; the order owner organizes cost and schedule comparison. Budget breaches or customer commitment changes go to the relevant authorizers and customer-facing contacts. Unaffected queries and checks within valid authorization continue.

If the revised delivery date was already communicated to the customer, the actual status of that notice and agreement must be verified, and necessary re-communication and remediation arranged. Flipping a local status back to "pending" cannot erase the customer's downstream actions; an external operation with unknown result must be investigated, not simply retried by another digital-employee instance.

Human takeover also requires preconditions: who is on duty, what facts they can see, whether they have stop and reallocation authority, escalation paths and timeouts. An alert is not a handover; takeover is complete only when the receiver acknowledges the current situation, scope, and open items. Until then, the original result owner must coordinate and escalate.

Takeover capacity is constrained by headcount, expertise, and time. If a single guardian cannot handle simultaneous exceptions, the enterprise must throttle, shrink the automation scope, or pause affected actions. Adding more digital employees does not substitute for adding human takeover capacity.

In emergencies, pre-approved protective actions may be executed first, then reported and reviewed. Handling beyond the playbook follows applicable emergency authority. Ex-post ratification must not become a blank check for the digital employee to self-expand its mandate.

Authorization expansion should be managed similarly: the authorized person decides based on representative run records, errors and consequences, verification effectiveness, and takeover capacity. A 90-day trial is an evaluation milestone, not an automatic green light. Long periods without takeover require checking whether the system is truly stable or exceptions are simply going undetected.

Redistributing Human Responsibility and Growth

Post-automation, human work cannot be summarized by "how many approvals remain." People are needed to design rules and grant permissions, make professional judgments, continuously orchestrate delivery, and maintain digital capabilities while catching exceptions. These responsibilities exist before the task starts, run through execution and closure — they are not an afterthought once the machine finishes.

The Mycelium Organization distinguishes four responsibility types that sequence within a single task. They do not necessarily require new roles; separation depends on professional qualifications, workload, and independence requirements. The enterprise's legal liability to customers and stakeholders cannot be transferred via an internal responsibility matrix.

Assigning responsibility to a person means giving them the necessary facts, judgment time, commensurate authority, and support. Quality staff who find a non-conforming batch must have the power to stop release; an order owner tasked with meeting the full delivery date must be able to secure resource confirmations and conflict resolutions from other departments. A system defect cannot be dumped entirely on the last person who clicked "approve." Post-incident analysis should trace who introduced the issue, who was responsible for intercepting it, and who allowed a necessary safeguard to remain missing — combining facts with actual control capacity.

Enterprises must also preserve a growth path for judgment. Lisanne Bainbridge's 1983 "ironies of automation" warned that after routine work is automated, humans may be asked to immediately take over rare, difficult exceptions while daily practice has atrophied. Today, when removing checks, the cost of cultivating judgment must be factored in.

Developing judgment does not require repetitive per-order stamping. Structured exception retrospectives, mentored hands-on work, simulation takeovers, and measured job rotation can all serve — provided the organization verifies that staff can truly judge independently and allocates time, resources, and rewards for this work. One cannot eliminate existing training opportunities and then burden a few experts with both exception handling and novice development.

Roles may shift, merge, or transform. Cutting an action does not auto-solve staffing. The enterprise must simultaneously clarify who moves to new tasks, how capability gaps are closed, and how the transition is supported. A person's place is justified by their capacity and conditions to carry valuable responsibility.

Start with One Real Work Chain: Task Contracts and Iterative Rollout

Once boundaries are clear, they must be maintained. Customer conditions, business rules, and digital capabilities evolve; if authorization lives only in a manager's memory, succession or upgrades easily erase the rationale.

The Mycelium Organization advocates "strong core, live front": the organization holds common rules, professional baselines, and permissions; frontline people and digital employees act within explicit scope. This can be operationalized through Task Contracts (TC) — for a given outcome, define the result, boundaries, key judgments, evidence, and escalation conditions, linked to named responsible persons and valid authorizations.

For the rush order, the evaluation contract specifies clarifying deliverable conditions; resource confirmation, external commitment, and fulfillment execution each activate under their own conditions. Standard work references approved templates — no need to refill forms every time. Only when facts, scope, or risk change are the affected contracts re-verified. Responsibility thus flows with the task, without relying on people to re-guess at every handoff.

When tasks span multiple business systems and these contracts need continuous invocation, a collaboration layer that can reference facts, identify authorizations, preserve versions, and support exception continuity is required. Miduo's MOOS (Mycelium Organization Operating System) is designed in this direction: business facts stay in source systems, while cross-node task accountability, execution evidence, and verification-acceptance links are connected. Whether these designs sustain in production and yield improvement still requires corresponding operational and acceptance evidence.

Implementation approach:

A business owner assembles the actual handlers, relevant professionals, and IT staff.

Pick a work type already using AI but plagued by recurring waits or exceptions.

Use existing records to list actions, apply the five questions, and pinpoint specific gaps: unclear authorization, insufficient evidence, failed professional gates, duplicate sign-offs.

The relevant authority holders decide adjustments. Map these to current roles and valid authorizations — no preset generic titles, no new per-item sign-offs. Every adjustment must document handover, recovery, and residual consequence handling.

Pilot, then compare with similar tasks on full delivery cycle, rework, risk, total cost, guardian burden, and customer outcomes.

If improved, another qualified group replicates under normal support, independently verifies results, and the authorized person decides adoption scope with continuous monitoring, correction, or retirement.

One adjustment solves one problem; validated approaches are then codified to standardize a class of problems.

Use existing platforms if they can carry the responsibility arrangements; only when a cross-system governance gap is confirmed should new products and investment be considered. Systems must embody the responsibility arrangements the enterprise has already clarified; managers still must decide rules, authorizations, and trade-offs.

Conclusion

After AI enters the business, enterprises need evidence-based delegation of execution and concrete assignment of judgment and responsibility to people. Technical performance, valid authorization, business consequences, rule-based evidence, and professional requirements jointly determine how far each action can go. Qualified routine work runs automatically; human-judgment items reach the right people with full backing.

The most actionable starting point is a live work chain: which step still repeats signatures, which step exceeds its mandate, who can stop it, who can take over, how is it fixed. Debating these divergences inside a single de-identified business process yields more effective change than arguing abstractly about "whether AI can replace humans."

As more work becomes eligible for AI, the authorizations, support, and responsibilities the enterprise provides for that work must become equally clear.