How KingbaseES Handles Unstructured Data with Full‑Text Search and Large Objects

1. Unstructured Data Processing Needs

Enterprises are seeing a rapid increase in unstructured data—information that does not follow a predefined schema, such as documents, multimedia, maps, satellite and medical images, and web content like HTML. Depending on how the data is created and used, management approaches differ widely.

Massive volumes reside on desktop office systems (documents, spreadsheets, presentations) and specialized workstations (geospatial analysis, medical imaging).

Governments, academia, and enterprises store terabytes of document archives and digital libraries.

Life‑science and pharmaceutical research maintain image banks and repositories.

Public‑sector, defense, telecom, utilities, and energy sectors run geospatial data‑warehouse applications.

Integrated operational systems in retail, insurance, healthcare, government, and public safety store business or health records, location data, and associated audio, video, and image information.

2. Advantages of KingbaseES for Unstructured Data

2.1 Full‑Text Indexing and Retrieval

Traditional SQL operators like LIKE cannot meet modern full‑text search requirements because they lack language support, ranking, and efficient indexing. KingbaseES addresses these gaps by providing:

Rich data‑type support : Full‑text search works on CHAR, VARCHAR, TEXT, and CLOB fields.

Extensibility : Users can create custom dictionaries, tokenizers, or ranking functions.

Built‑in ranking functions : Generic rank functions based on term similarity, co‑occurrence, and importance are supplied out‑of‑the‑box.

Pre‑processing of query text : Tokenization, case normalization, stemming, stop‑word removal, and rank calculation are performed before searching.

Efficient inverted index : A generalized inverted index (GII) is built on the search text to accelerate queries.

Dual search modes : Traditional exact‑match SQL and fuzzy full‑text search can be combined—first narrow results with fuzzy matching, then refine with precise SQL.

2.2 Large‑Object (LOB) Types

Modern information systems store massive semi‑structured or unstructured media (images, reports, audio, video). KingbaseES provides dedicated large‑object types to handle such data:

BLOB (Binary Large Object) and CLOB (Character Large Object) with a maximum size of 2 GB per object.

Comprehensive external functions for creation, closing, deletion, reading, writing, truncating, importing, and exporting LOBs.

Special handling for storage, lock resource usage, transaction management, and logical backup/restore compared with ordinary string types.

3. Summary

Unstructured data—including documents, multimedia, geospatial information, satellite and medical images, and web content—can be effectively managed in KingbaseES using its BLOB and CLOB large‑object types (up to 2 GB each) together with a full‑text indexing and retrieval engine that supports rich data types, extensible ranking, and efficient query processing.