Managing Multimodal Data for Agents: OceanBase’s Multimodal Table Solution

At the recent OceanBase Hours online launch, OceanBase introduced a lake‑house AI database whose core innovation is the multimodal table, designed to meet the data needs of Agent‑centric workloads.

Why multimodal tables are needed : Agents often require simultaneous access to user profiles (unstructured), dialogue summaries (text), PDF originals, vector embeddings and real‑time business state. These data sources are currently scattered across object storage, search systems, vector stores and transactional databases, lacking unified transactions, consistent snapshots and lineage. When a PDF is updated but its embedding still points to the old version, the Agent may produce incorrect answers without any indication of the error.

Four fundamental changes introduced by multimodal data in the production pipeline :

Unstructured data enters the core production line – it becomes a key input for Agent decisions rather than a mere attachment.

Data access patterns become hybrid search – a single request may involve structured queries, full‑text search, vector recall and filter analysis.

Real‑time requirements surge – Agents need 24/7 online decision making, which traditional batch reporting cannot satisfy.

Features are dynamically generated – many fields are produced asynchronously by models from raw multimodal data.

Core capabilities of multimodal tables :

Unified management of multimodal objects (images, audio/video, PDF, web snapshots, vectors, JSON, structured fields) with full transactional, consistency, high‑availability, hybrid search, analytical and online service support.

Three storage forms for large objects:

In‑row storage : small objects or critical metadata stored directly within the row.

Out‑of‑row storage : large objects stored separately but still referenced and bounded by database transactions.

External storage : the object body resides in external object stores while the database keeps address, version and visibility metadata.

Dynamic feature columns that allow new AI‑derived attributes (embeddings, OCR results, quality scores, tags) to be added without DDL changes, while preserving SQL queryability and search optimization.

Column‑group level consistency: a column group becomes visible only after all its derived columns are completed, ensuring queries always read a consistent snapshot rather than a partially computed record.

Asynchronous near‑real‑time computation workflow :

Original data is written first, guaranteeing fast online write response.

Background tasks are triggered by lineage relationships to extract features, build indexes and back‑fill derived columns.

The system makes the derived data visible within a near‑real‑time window.

Read requests retrieve data from a consistent snapshot that includes only fully materialized column groups.

This design accepts that strong synchronous completion of all derived columns is unrealistic; instead it balances real‑time visibility, throughput, storage efficiency and consistency.

Store‑compute separation and hot‑cold integration : recent data (e.g., within 90 days) is cached on SSD for low‑latency access, while older data is tiered to low‑cost shared storage. Both hot and cold data are accessible through the same query, analysis and search engine, eliminating the need for manual data stitching across systems.

Unified catalog and metadata services provide lineage tracking, versioning and permission management for raw objects, structured fields, dynamic feature columns and model tasks. The three key abilities are:

Trigger : automatic background task initiation based on lineage.

Completeness judgment : determining whether a column group or snapshot has finished all derived computations before making it visible.

Traceability : enabling the system to report which original documents, images, business records and derived features an Agent’s answer relied on.

Integrated workload support : multimodal tables serve as a data foundation for three workloads—online transaction (writes, ACID guarantees, high availability), analytical processing (large‑scale scans, aggregations, joins, feature analysis) and hybrid search (combined structured filters, full‑text, vector recall, tag and quality‑score ranking). This unified approach is essential for Agent scenarios where a single request may involve multiple search modalities.

In summary, OceanBase’s multimodal table addresses the fundamental question of how a database can continue to manage complete business objects when Agents become its primary users, moving beyond merely storing images or vectors to providing a consistent, governed, and searchable multimodal data platform.