Why Your AI ROI Calculation Is Wrong: Three Value Paths You Must Separate

The article opens with a scenario: an after-sales ticket assistant reportedly saves 3 minutes per ticket across 6,000 monthly tickets, yielding 300 hours. The project team translates this to ¥18,000 at ¥60/hour, but the business lead sees no overtime reduction and finance asks whether outsourcing bills dropped and whether knowledge-base maintenance, review, and exception handling costs were included. The core problem is that each party measures a different thing — released time, work outcomes, or cash flow — and ROI discussions must first align benefits, costs, and evaluation period.

Three Distinct Value Paths

Drawing on iResearch's 2025 layered evaluation approach, the author splits value into three paths that must be tracked separately to avoid double-counting:

Process Efficiency : faster record compilation, policy lookup, reply drafting. Verify end-to-end manual effort, throughput, and rework. Count monetary benefit only when overtime or outsourcing spend actually falls, or new capacity drives attributable contribution.

Knowledge Enhancement : more accurate identification of warranty boundaries, policy exceptions, historical commitments. Verify judgment correctness, error compensation, reopen and escalation outcomes. Record monetary benefit only when loss reduction or disposal cost reduction is evidenced; otherwise keep quality metrics.

Product Innovation : turning internal assist into a customer-purchasable diagnostic service. Verify actual payment, sustained usage, service quality, and delivery cost. Count incremental revenue over the original business minus incremental costs.

The same assistant can improve both speed and judgment, but a given benefit (e.g., fewer reopens saving handling time) must be assigned to only one path. The paths are not a maturity ladder; a stable internal tool that cuts rework may already justify its cost.

Regardless of path, benefits must be classified into three realization states — realized cash , released capacity , and future opportunities — and recorded separately. An image illustrates this two-dimensional tracking.

Why 300 Saved Hours ≠ ¥18,000 Cash Savings

First, the 3-minute figure must cover the full task: preparation, verification, editing, and subsequent reopens, not just draft generation. After verification, assume the complete task average drops from 12 to 9 minutes with no quality regression. Under comparable volume and staffing, 300 hours/month is recorded as released capacity.

Released capacity has three possible fates:

Convert to less cash spent : if rosters adjust and overtime or variable outsourcing actually decreases, that portion enters cash benefits. Fixed salaries unchanged means hours saved ≠ wage savings.

Convert to more work done : if used for backlog clearance, serving more customers, or callbacks, track whether backlog shrinks and extra service creates value. Downstream bottlenecks may block conversion.

Remain as available time : reduced burden has value but fragmented time may not enable re-rostering or role elimination. Keep hours and experience improvement without forcing cash conversion.

The ¥60/hour rate can be used for resource valuation if the source is disclosed, but it cannot be booked as cash savings without business records proving the conversion. An image shows the three fates.

Measuring Knowledge Enhancement and Product Innovation

Knowledge enhancement is harder because correct suggestions are intermediate. Example: the assistant flags a ticket as ineligible for free repair. Expert confirmation of the suggestion is not enough; the original process might have caught it, the customer might supply missing proof, or the company might honor an exception. The comparison must be against the comparable baseline: how much inappropriate payout decreased, while monitoring false denials and complaints. For low-frequency high-loss events, compare expected loss under old vs. new, estimating probability and impact with uncertainty ranges. Absence of incidents does not prove the model prevented them; without a reliable baseline, record detection capability and control effect, not hypothetical savings.

Long-term assets from knowledge enhancement include verified interaction samples, corrections, and annotations for knowledge maintenance, independent evaluation, or private model training. The data flywheel — operational feedback improving capability, validated by subsequent business results — requires use authorization, quality confirmation, evaluation isolation, and controlled release; it does not form automatically from log accumulation. Track actual reuse, improvement effects, and maintenance costs; do not value by sample count alone or add future value to current benefits. An image depicts the feedback loop.

Product innovation must pass the customer-willingness-to-pay gate. A paid remote diagnostic service may show trial users, demo praise, and LOIs, but these are demand signals, not revenue. After launch, track refunds, retention, human fallback, and delivery cost, and ensure it is not merely renaming existing service revenue. Calculate incremental revenue (excluding refunds and cannibalization) and incremental costs (new delivery, sales, support). If the business provides a contribution margin that already deducts some costs (e.g., support staff but not model calls), unpack what was deducted to align with the project cost table and avoid double deduction. Early exploration that validates willingness to pay, identifies cost ceilings, or falsifies a direction has learning value but stays separate from cash benefits.

Baseline Is Not a Pre-Launch Average

The real question: what would have happened without the AI? Post-launch speedups may stem from policy simplification, staff proficiency, fewer product defects, or seasonality. A simple pre/post one-month average is insufficient. Where feasible, use randomized comparable teams with similar training and process conditions, accounting for cross-team contamination (group by team if needed). When randomization is impossible, use phased rollout as a reference or historical trend analysis, but record task difficulty, proficiency, concurrent policy/system changes, sample size, and observation window. If evidence only supports correlation, do not state causal attribution. The NBER working paper Generative AI at Work (Brynjolfsson, Li, Raymond, 2023) shows differential effects by agent experience, reinforcing the need for segmented observation but not providing a portable productivity ratio.

Two measurement scopes are often overlooked:

Project-level benefit must cover all agreed applicable tasks, including non-adopted, timed-out, failed, and human-takeover cases. Analyzing only successful AI-assisted tickets excludes applicability and operational issues.

Clarify whether the evaluation measures model capability or the full solution (knowledge cleanup, process redesign, training). If the launch bundles these, the measured benefit and cost belong to the whole package. Isolating AI's marginal value over search or rules requires additional comparison arms.

An image summarizes comparable baseline design.

Full Lifecycle Costs Beyond Model Bills

Agree on an evaluation period (e.g., build through first full year) so that build spend, monthly operating costs, and benefit realization share the same timeline. The article provides a six-category cost checklist:

Business & Data Preparation : expert confirmation, material authorization, knowledge curation, cleansing, evaluation samples. Hidden: internal expert time is not free.

Development & Integration : product development, permission access, interface modification, testing, launch. Hidden: original system team and on-site joint debugging effort.

Model & Infrastructure : API, licenses, compute, storage, retrieval, monitoring. Hidden: retries, peak capacity, idle resources.

Human & Business Operations : review, takeover, knowledge maintenance, quality sampling, issue handling. Hidden: labor already counted as a benefit offset must not be double-counted as cost.

Organizational Change : training, communication, trial coaching, process adjustment. Hidden: learning-curve productivity dip, manager support time.

Risk & Exit : security hardening, compliance checks, recovery, migration, decommissioning. Hidden: shutdown does not mean zero cost.

Each line item notes one-time vs. recurring, actual vs. forecast, evidence owner; recurring costs split variable vs. fixed. Cash outlay and internal resource consumption are shown separately — the former for payback analysis, the latter for opportunity-cost judgment. The same labor cannot be counted as both wage and opportunity cost. Shared platform costs are allocated per agreement; expansion decisions look at incremental calls, capacity, and support. The sunk cost fallacy warns that past irrecoverable build costs stay in the full-cycle retrospective but must not drive additional budget; go-forward decisions compare future incremental benefits and costs, including salvage value, exit fees, and migration costs. An image illustrates the cost coverage.

Worked Example: Separating Capacity and Cash Accounts

Returning to the ticket assistant: verified 300 hours/month, of which 100 hours enable overtime reduction via roster changes, 200 hours absorb backlog (not yet reliably monetizable). Overtime rate ¥60/hour. Upfront cash investment ¥120,000; monthly recurring cash operating cost ¥8,000. Internal resource usage recorded separately.

Projected annual overtime savings: 100 hrs × ¥60 × 12 = ¥72,000. First-year new cash outlay: ¥120,000 + ¥8,000 × 12 = ¥216,000. First-year net cash benefit: ¥72,000 − ¥216,000 = −¥144,000. First-year cash ROI: −¥144,000 ÷ ¥216,000 ≈ −66.7%.

This is a conditional projection, not a realized financial result; it excludes tax and time value, and multi-year projects need discounted cash flow analysis. Crucially, monthly overtime savings (¥6,000) are below monthly operating cost (¥8,000), so without new value or cost reduction the project never breaks even on cash. The −66.7% answers whether cash savings cover new cash outlay; it does not capture total value. Unmonetized capacity and future opportunities are recorded separately. The remaining 200 hours may close the gap via backlog clearance or quality gains but must be verified independently — no double-counting. Sensitivity analysis should vary applicable volume, actual adoption, review burden, and operating costs to see when benefits turn negative; scaling volume also raises review and compute costs. An image shows the example numbers.

Seven-Point Checklist for Any AI ROI Report

These seven questions catch the most common errors. Apply them to any internal or vendor ROI claim:

Benefit Layering : demand separate lines for realized cash, released capacity, future opportunities.

Capacity Destination : require business records of roster changes, outsourcing reduction, backlog movement.

Baseline Comparability : require control group, phased rollout, or trend analysis with concurrent policy/system changes documented.

Sample Completeness : must cover non-adopted, timeout, failure, and human takeover cases.

Cost Completeness : must include data prep, integration, review, training, risk, exit.

Evaluation Period Consistency : pre-agree period, align costs and benefits on same timeline.

Re-evaluation Plan : define observation window, re-evaluation date, failure signals.

Value Evidence Card for Continuous Re-evaluation

A single living card aligns scope, benefits, costs, and responsibilities across business, finance, and tech. Example fields for the ticket assistant:

Object & Scope : standard warranty ticket evidence compilation and reply assistance; high-risk exceptions excluded.

Value Hypothesis : reduce full-task manual effort, reduce overtime via roster adjustment.

Evaluation Period & Targets : build period + 12 months post-launch, plus trial observation window; pre-confirm baseline, improvement target, rework/error floor.

Comparison Method : phased verification in comparable teams on task difficulty and personnel; record concurrent changes.

Benefit Status : record realized cash, released capacity, future opportunities separately; actuals with records, forecasts with assumptions/ranges, no amount without reliable basis.

Long-term Assets : verified reusable ticket correction samples under future opportunities; track actual reuse for knowledge/evaluation improvements, effects, maintenance cost; keep training and independent evaluation isolated.

Cost Evidence : procurement and operating bills, internal hours, allocation basis, maintenance and exit arrangements.

Responsibility : business owner confirms outcomes and capacity absorption; finance verifies amounts; capability owner provides version and operating records.

Re-evaluation Schedule : record observation window, current decision, next date; early re-eval on major policy/model/review changes.

Failure Signals : quality out of bounds, review rework offsets speed gains, released capacity long without actual destination.

An image shows the card structure. At each re-evaluation, focus on which assumption changed: e.g., model unit price fell but context lengthened, retries and reviews rose, unit business cost increased; or overall hours didn't drop but high-cost errors fell significantly — decide based on original targets and evidence. Continue within verified scope if main benefits are evidenced and risks controlled; assign owner and limited fix period for capacity absorption issues; shrink scope, change approach, or stop if key assumptions persistently fail. Portfolio allocation is a separate discussion.

Scenarios Not Requiring Full Accounting

One-off, low-risk personal writing and document organization only need cost, time, and usability notes. Applications needing sustained budget, cross-functional collaboration, or affecting customer outcomes warrant systematic baseline, cost, and benefit tracking. Mandatory compliance and foundational capability building must first prove necessity and control effectiveness, then compare compliant solution costs. Strategic exploration may temporarily lack financial return but needs clear budget, validation goals, and stop conditions.

Conclusion

AI benefits do not automatically enter the corporate ledger. Time needs absorption, judgment needs action change, innovation needs recurring purchase, and all incur operating and organizational costs. A credible value assessment shows what is already realized, what is merely capacity, what remains opportunity, and what the next investment will verify. The next article will ask: when value measurement moves to daily Agent operations, how to define an "effective task completion" so that impressive task counts don't mask quality, risk, and human remediation.

References

[1] iResearch: 2025 China Enterprise AI Application Industry Research Report , Dec 2025, pp. 11, 30. The report uses "process efficiency, knowledge amplification, value innovation"; the article adapts its layered dynamic evaluation logic and reorganizes value paths for the after-sales scenario. Tables, checklists, and worked examples are the author's methodological design, not reported empirical results. URL: https://report.iresearch.cn/report/202512/4779.shtml

[2] Erik Brynjolfsson, Danielle Li, Lindsey R. Raymond: Generative AI at Work , NBER Working Paper 31161 (Nov 2023 revision). Cited for its single-enterprise customer-service finding that effects vary by employee experience and skill; the productivity ratio is not extrapolated to other projects. URL: https://www.nber.org/papers/w31161