AI Search Evolution: Alibaba Cloud ES AI Engine Launch for Enterprise AI Scenarios

AI Search as a Real‑Time Context Engine

AI search is moving beyond a simple query endpoint to become a live context layer that connects enterprise data with autonomous agents. The speaker outlines a full upgrade path from the underlying engine to Agentic capabilities such as Memory, self‑evolving Skills, and an enterprise Knowledge Engine.

Changes in the AI Era

Unstructured data is growing 30% annually and is projected to exceed 180 ZB worldwide by 2025. Agent‑driven queries generate bursty QPS that can vary 5‑20× between peaks and valleys, making traditional peak‑based provisioning wasteful and scaling with large data migrations impractical.

Five Core Product Capabilities

AI Engine Version : A stateless, OSS‑based multi‑tenant architecture compatible with the Elasticsearch ecosystem, supporting billions of tenants and vectors. Benchmarks with 10 000 namespaces and 93.7 M documents achieve recall@10 = 0.95, hot‑query P50 = 1 ms, cold‑query P99 = 605 ms, and a total cost about 70% lower than self‑built solutions (OSS storage 0.15 RMB/GB·month).

OpenStore : Decouples storage and compute using OSS for logs and Pangu DFS for search, enabling elastic scaling without data movement, 25× faster change speed, 40% lower storage cost, and ~30% overall cost reduction for top Web3 retrieval customers.

FalconSeek : A C++ native kernel built on Havenask that accelerates the hot query path while preserving ES API compatibility. Benchmarks show Rally big5 queries dropping from 48.609 ms to 7.076 ms (6.87× faster) and Tantivy Search benchmarks improving 2×, with vector search leading VectorDBBench and saving ~30% cloud compute.

LakeSearch : ES × Paimon lake‑wide multimodal search that builds a global index on the lake, keeping source data in place. It cuts lake storage by 50%, reduces PB‑scale data ingestion from days to minutes, and supports combined BM25, kNN, filters, sorting, and aggregation across text, image, video, and vector modalities.

Log Full‑Chain : Consolidates collection, buffering, processing, and delivery into a managed endpoint (OTel/Beats/Logstash). Reported benefits include >30% lower collection cost, 60% lower write‑compute cost, >40% storage savings, and 50% latency reduction in the recall stage.

ES Agent for Intelligent Operations

ES Agent covers alert response, periodic inspection, performance optimization, and capacity planning, offering one‑click root‑cause diagnosis, automated inspections, performance insights, and capacity forecasts, thus closing the loop from alert to optimization.

From Search Engine to Context Infrastructure

The roadmap extends the engine upward: multimodal search → model integration → API/MCP/Skills → Agent Builder → workflow orchestration. Agentic Memory persists context, extracts interests via LLMs, resolves conflicts, and creates “Golden Facts.” Skill self‑evolution records real tasks, automatically creates, merges, or discards skills after idle periods, limiting changes to agent‑generated skills only.

Enterprise Knowledge Engine

Provides trustworthy row/column/document‑level permission push‑down, visual knowledge‑graph representation, and observability via Trajectory replay for audit and continuous tuning. It integrates with agents so that Memory accumulates context, Skills enable reuse, and the Knowledge Engine ensures reliable, observable data flow.

Conclusion

The five product capabilities address AI‑search constraints in scale, elasticity, performance, lake‑data access, and logging. Agentic Memory, self‑evolving Skills, and the Knowledge Engine further enrich context accumulation, reuse, and governance, turning search from a simple result fetcher into the real‑time bridge between data and agents.