Agent Rewrites Keep Coming: What Enterprises Must Retain for Lasting AI Value

Orthogonal Growth: General Intelligence vs. Business Depth

Agent development has undergone rapid paradigm shifts — from custom Agent loops to frameworks like LangChain and LangGraph, and now to model-provided SDKs and managed runtimes. Each model upgrade triggers architectural changes. The article proposes orthogonal growth : model vendors advance general intelligence (Memory, Planner, Managed Runtime, Agent collaboration) horizontally, while enterprises must grow vertically in business depth — accumulating business facts, object relationships, unified semantics, concrete processes, and industry knowledge. These vertical assets are not replaced by model upgrades; they are amplified. A key litmus test: after the next model upgrade, will an investment be replaced or amplified?

Business Context Must Come from Continuously Running Data

Business facts have three characteristics: (1) they originate from actual system operations, not post-hoc documentation; (2) a single answer often requires joining multiple systems (CRM, ERP, inventory, logistics, etc.); (3) business state constantly changes — a one-time export quickly becomes stale. Therefore, Agents need a continuous data supply pipeline: continuous ingestion, multi-source convergence, incremental updates, cross-system composition, and evolution with business state.

Message Services as the Natural Backbone for Real-Time Data Integration

Message services (RocketMQ, Kafka, EventBridge, etc.) extend the Unix pipe model to distributed environments. They are inherently suited for continuous data integration because:

Append-only log enables high-throughput, decoupled producer/consumer patterns.

Flexible ingestion supports both push (Put) and pull (CDC, API polling) from heterogeneous sources (databases, SaaS, logs, IoT, APIs).

Caching, subscription, and replay absorb traffic spikes, enable fan-out, and support retry, backfill, and new consumer catch-up.

These services have evolved into enterprise real-time data integration infrastructure, connecting upstream sources (ERP, CRM, databases, IoT) to downstream consumers (data lakes, real-time warehouses, BI, risk, recommendation, AI, monitoring). Agents become one class of consumer, but the path is often indirect: raw messages → message hub (filter, clean, transform, route) → specialized data stores (relational, search, vector, analytical) → MCP servers → Agents.

Accessible ≠ Usable: Three Judgments for Agent-Ready Data

Even when data is accessible, Agents still struggle because schema, semantics, lineage, and freshness break across handoffs. The article presents three judgments:

Judgment 1: Source Semantics Must Propagate and Evolve End-to-End

Data passes through source system → message ingestion (schema registry, versioning, compatibility) → transformation (mapping, enrichment, new lineage) → storage (scenario-specific tables) → MCP/Agent layer. Every handoff must preserve meaning, schema compatibility, and traceability. This is continuous, not one-off.

Judgment 2: Storage Can Be Distributed, but Semantics and Retrieval Must Be Unified Upfront

Physical distribution is inevitable (stream systems, transactional DBs, data lakes, search, vector DBs, external SaaS). However, cognitive fragmentation is unacceptable. Before Agent invocation, the platform must continuously build a unified cognitive layer: Catalog (discoverability), Schema & Semantic (understandability), Entity & Lineage (cross-source correlation), Freshness/Quality/Policy (trustworthiness and access control). This shifts from "on-demand understanding at query time" to "continuous pre-materialized cognition" because Agents have limited latency and context windows.

Judgment 3: The Integration Pipeline Will Continuously Converge

Multi-hop pipelines increase context fragmentation, management overhead, replica proliferation, and blurred accountability. Convergence means a single service owns the end-to-end responsibility from ingestion to consumption, not necessarily centralizing all data physically.

EventHouse: One-Stop Real-Time Data Service for Agents

EventHouse implements the above principles with a message backbone + unified metadata layer spanning four modules: Processing (EventHouse or external), Storage (built-in or open lake formats like Iceberg/Parquet in customer-owned object storage), Query (native or federated via Trino), and Agentic Retrieval (API, CLI, MCP, IM). The unified metadata (Catalog, Schema, Semantic, Lineage) is shared across modules, enabling "one-stop" without lock-in.

Semantic Layer: Turning Business Definitions into Executable Specifications

The semantic layer comprises five categories:

Object semantics — column descriptions, business names, synonyms.

Scope semantics — which data sources/fields are relevant for a scenario.

Relational semantics — join conditions, cardinality.

Computational semantics — measures, filters, executable Field SQL.

Value semantics — format assistance, entity matching (e.g., "Beijing" → region_code = 'BJ', "Paid" → payment_status = 'PAID', net revenue → SUM(order_amount - discount_amount)).

Names enable matching, SQL ensures precise execution, relations and values complete context.

Balancing Semantic Construction and Usage Cost

EventHouse avoids heavy ontologies (like Semantic Web or Palantir's) because they require dedicated FDEs and high maintenance. Instead, it supports both structured semantic definitions and lightweight wiki-style "skills". Understanding and alignment are pushed to the usage phase, leveraging growing LLM context windows and caching. At query time, only relevant tables (typically ≤15) are injected. A ReAct loop corrects SQL through real execution feedback: Reason (infer from business definitions), Act (generate and execute SQL), Observe (read rows or errors), Reflect (correct and iterate).

Semantic Initialization: Inherit Facts First, Then Let AI Propose Candidates

Four steps: (1) inherit upstream schema, column descriptions, types, access scope; (2) sample representative values for format/entity matching; (3) discover certain relationships via existing PK/FK; (4) let AI suggest reusable SQL expressions or missing joins for expert review. Output is a Semantic Draft v0 awaiting certification.

Semantic Certification: Model Provides Evidence, Experts Bear Responsibility

Four responsibilities cannot be delegated to models: definition (metric formula, statistical caliber), relationship (which tables join, cardinality, double-counting risk), boundary (entity mapping, applicable data/time/scope), and publication (owner, version state, rollback authority). Machines supply evidence (metadata, sample values, PK/FK, candidate SQL, historical query patterns); experts accept/modify/reject and record decision, owner, version, and benchmark regression results.

Benchmark: Measuring Business Answers, Not Just SQL Strings

Three benchmark layers: (1) Exploration — verify problem realism and executability; AI generates candidate Q/SQL pairs, human spot-checks. (2) Correctness — core layer with Gold SQL, expected results, business rules; validates execution results (filters, aggregates, joins, sorting, metric caliber), not string equality. (3) Production Reliability — covers paraphrase robustness, ambiguity handling, graceful failure, permission/performance/security compliance. Higher layer ≠ higher value; correctness is the mainstay.

Treat Semantic Changes as Controlled Pull Requests

Semantic assets evolve. Each change must include: content + business reason, affected tables/metrics/questions, mandatory regression cases + paraphrases, expected benefits and risks. Release gates: core KPI zero regression, overall accuracy non-degrading, latency/error/cost targets met, owner approval. Full lifecycle: Draft → Review → Benchmark → Merge/Rollback.

Continuous Upgrade Loop from Real Q&A

Most valuable benchmarks come from real user questions. System records full Trace (original question, generated SQL, tool calls, results, final answer). Combined with user feedback and LLM Judge, domain experts attribute errors to definition, field/join/filter logic, synonym gaps, or multi-turn context loss. Fixes produce two artifacts: semantic repairs (new definitions, corrected relations, trusted SQL examples) and benchmark samples (failed cases added with paraphrases, boundaries, expected results). Offline regression confirms no accuracy regression, latency/error/cost compliance; then gradual rollout with continued feedback collection.

Two Scenarios: Unified Yet Open

Scenario 1: Churn Complaint Trace & Early Warning

Customer service asks: "Which customers who just requested refunds complained in the last 90 days? About what?" EventHouse correlates real-time refund messages (Kafka), MySQL customer/order/after-sales tables, and search-indexed service transcripts/complaint texts around customer_id and time windows, building a customer timeline (e.g., first complaint 12 days ago, repeated escalations 8 days ago, ticket upgrade 4 days ago, refund today). Unified metadata defines "complaint" scope (platform tickets + calls), brings catalog/schema/lineage/freshness/permissions into the query. Output shifts from post-hoc "who refunded" to "why refund" and "how to pre-warn".

Scenario 2: Manufacturing Data — One Ingestion, Multiple Open Outflows

Production/quality leads want real-time alerts on battery cell temperature anomalies, sync to quality DB, long-term archive, and repair Agent access. Data sources: IoT/PLC streams, MES/ERP, maintenance/quality systems. Traditional approach builds separate pipelines per goal, creating duplicate copies and divergent definitions. EventHouse keeps a single primary pipeline but opens each stage: processing can use EventHouse or external stream processors; storage can use built-in or open lake formats (Iceberg/Parquet) in enterprise object storage; query can federate existing PG/MySQL via Trino; unified metadata exposed via API/CLI/MCP/IM to Copilot, ChatGPT, Claude, Gemini, Dify, custom Agents. No full replacement required; processing, storage, query, and semantics are all reusable.

Conclusion: Follow Intelligence, Accumulate Context; Swappable Foundation, Retained Assets

Continuously track general intelligence advances, but long-term investment belongs in proprietary business context — its value grows as models improve.

Message services are mature real-time data pipes. The next step is ensuring schema, semantics, lineage, and freshness travel with the data.

Pipelines must converge for end-to-end accountability, while capabilities remain open: external processing, open storage formats, federated query, multi-Agent connectivity.

An Agent's lasting competitiveness depends not only on model and infra strength, but on its ability to continuously perceive real business, understand enterprise definitions, trust data, and act on that foundation.