Hologres: One SQL Engine for Multimodal Data, AI Functions & Agent Memory

Hologres began as a unified real-time data warehouse engine for OLAP and online serving. With growing multimodal demands, it added full-text and vector search, and in the agent era further extended into data infrastructure for agents and models. Its capabilities now form two layers: a full-modal real-time data warehouse engine covering structured, semi-structured, and unstructured data with OLAP, KV, vector, full-text, graph, and spatial queries; and an agent/model infrastructure layer including long-term memory service and observable data foundation.

1. Unifying Multimodal Data in One System

Hologres connects three data source categories: structured/semi-structured data stored natively; lake/offline warehouse data (MaxCompute, Paimon, Iceberg) via external access; and unstructured files (images, PDFs, videos) on OSS mapped through Object Tables. This brings originally scattered data into a single downstream processing pipeline.

Processing relies on two components: AI Functions call models for parsing, chunking, embedding, speech-to-text, and generation; Dynamic Tables handle incremental computation and warehouse layering. For example, when an OSS directory continuously receives new images, Dynamic Tables process only increments, avoiding full recomputation. Results stay in Hologres for point lookup, OLAP, full-text, vector, and hybrid search.

2. AI Function: Multimodal Processing, Model Calls & Hybrid Search in One SQL Chain

AI Function usage takes three steps: purchase Hologres model resources, deploy models in console, then call via standard SQL. This consolidates work that previously required Python, model calls, and multiple data pipelines into the SQL development system. Cost varies by scenario: lightweight tasks (classification, summarization) ~100K–500K tokens/day; semantic search, real-time risk control ~500K–2M tokens; large-scale vectorization or complex generation ~2M–10M tokens.

AI Functions wrap common AI data capabilities as SQL functions: ai_parse_document, ai_extract, ai_transcribe for document parsing, info extraction, speech-to-text; ai_fix_grammar, ai_similarity, ai_analyze_sentiment, ai_classify, ai_summarize, ai_translate, ai_rank for text correction, similarity, sentiment, classification, summarization, translation, ranking; ai_gen for generation; ai_embed, ai_chunk, ai_mask for vectorization, chunking, and PII masking. Thus file parsing, text processing, embedding, and generation embed directly into SQL workflows without separate AI pipelines.

Image Retrieval Example

Images stored in OSS; after model deployment, SQL creates Object Table and Dynamic Table to write image paths and vectors. Downstream supports text-to-image (e.g., "a red car in the rain") and image-to-image search. Previously requiring multiple systems and pipelines, now unified in SQL.

SaaS Similar Invoice Recommendation

Invoices/contracts parsed by ai_parse_document, chunked via ai_chunk, embedded via ai_embed, then dual-recall ranking via vector + full-text search returning Top-1. Hybrid search single-concurrency avg latency 72.8ms (40% better than other retrieval systems); at 40 concurrency, 140K eval set total 1080.43s, avg latency 7.6ms, query accuracy 84.46%. Data no longer exported to external search systems, eliminating sync dev/ops; warehouse and algorithm teams collaborate on one data stack.

Retail Shelf Compliance

A large convenience-store chain uses multimodal retrieval for shelf compliance: structured data (shelf base, template, product info) plus unstructured (shelf photos, inspection photos, template mockups, product images). After ingestion, templates are structured, images stored and vectorized, then Qwen3-VL performs multimodal comparison to generate discrepancy lists. Joint retrieval/analysis via SQL; discrepancy detection accuracy >90%. The existing data platform now handles multimodal warehousing.

Marketing Video Generation

Object Table reads raw assets; ai_gen creates new materials and storyboards, video models generate final clips, human review then deployment, performance data flows back for analysis. Assets, generation, and deployment linked; role variants, storyboards, videos produced in seconds/minutes with batch support.

Speech Analytics

Industries (customer service, gaming, autonomous driving, healthcare, SaaS, cross-border) accumulate call/meeting/remote-support recordings. Object Table reads raw audio from OSS; Bailian LLM performs speaker ID, language ID, ASR transcription; results written to speech tables for downstream sentiment analysis, ticket generation. Previously manual listening/organizing becomes queryable, analyzable data assets.

3. AI Assistant: Warehouse Expertise as Executable Skills, Not Just Q&A

AI Assistant is "AI for Hologres" — lowering barriers to warehouse onboarding, development, ops, and analysis. It provides knowledge Q&A, warehouse development, and ops agents, combined with Holo Skills, meta-warehouse, and knowledge bases.

After enabling AI Assistant and linking target instances, users trigger tasks via natural language in console or HoloWeb. For CPU diagnosis: select instance and time range; model follows built-in Skill: collect data, output diagnostic report covering basic info, health judgment, actionable recommendations. Slow-query diagnosis follows same pattern: collect data, apply accumulated experience and Skills to output root-cause analysis and tuning directions.

Beyond diagnostics, capabilities cover pre-sales consulting, spec recommendation, feature Q&A, warehouse setup, SQL generation/tuning, cost governance, natural-language data access, report visualization, and data interpretation. Key: diagnostic flows and warehouse experience organized as agent-callable steps, not just generic Q&A.

Deeper internal component or source-code diagnosis is a goal, but Hologres is closed-source; source-level info involves security. AI Assistant will expand automated diagnosis scope within safety boundaries.

4. Long-Term Memory Service: Turning Agent Interactions into Sustainably Recallable Data

Long-term memory service ("Hologres for AI") solves cross-device/session forgetting and enterprise knowledge silos. It extracts effective info from human-agent interactions, persists to a searchable data layer. Typical apps: shopping agents, AI tutors, financial advisors, enterprise collaboration, AI coding.

Pipeline: "extract–store–recall–inject" four steps. Interaction context → LLM extracts memory points → embedding model vectorizes → memory text + vectors written to Hologres multimodal memory store. Each new interaction: current question first retrieves relevant memory → recalled memory injected into model call → cross-session info returns to current task. Service layer exposes Mem0 standard API; OpenClaw, Hermes, AI Coding agents can integrate. Stack includes LLM, Embedder, Rerank, Hologres HGraph/vector engine.

Product features: fast integration (Mem0 compatible, API-only), fully managed (no multi-component ops), all-in-one (memory refinement, vector/full-text/graph/scalar fused search), highly scalable (cloud-native distributed, supports 10B+ memories), high performance (real-time write, write-visible, vector/full-text/graph search in milliseconds). Goals: zero memory loss, agent sharing, zero ops.

AI Tutoring Example

Teachers, students, parents have different memory access needs. Parents input student basics and learning goals; teachers continuously write dictation, quizzes, homework feedback — real-time memory沉淀. When parents ask recent status, model combines history to summarize performance, suggest next actions, and create study plans from same memory batch.

Enterprise AI Coding

Traditionally only code persists; with agent-written code, "why this way, how step-by-step" remains in human-agent context. Extracting and persisting this turns the development process itself into enterprise assets.

Manufacturing & IoT Foundations

Manufacturing first needs data collection/沉淀. Connected vehicles: sensor signals, driving video/images collected/stored, then used for model training, fault diagnosis. Industrial scenes similar: identify valuable data, choose hardware, continuously沉淀 — AI enhancement builds on that foundation.

From multimodal processing/retrieval to AI Assistant to agent long-term memory, Hologres's AI-facing capabilities extend from data processing to data usage and agent runtime. Capabilities extend, foundation unchanged: unified storage, real-time processing, retrieval at bottom; upper layers carry growing agent-centric scenarios.