Tagged articles
data indexing
20 articles · Page 1 of 1
The article explains Lance's stable_id (stable row ID) and address_id concepts, why stable IDs are needed, how they are encoded and mapped via RowIdIndex, the trade‑offs of this design, how to enable them, and how to convert between row address and stable ID.
LanceRowIdIndexStable Row ID
0 likes · 12 min read

The article analyzes the design philosophies, key components, strengths, and weaknesses of LangChain and LlamaIndex, explains their distinct core scenarios—complex multi‑step agent orchestration versus private‑data RAG—and shows how they can be combined in real projects while outlining emerging ecosystem trends.
AgentLLMLangChain
0 likes · 13 min read

Elasticsearch does not store a built‑in write timestamp, so to trace when a document was indexed you must add the field during ingest, using either an Ingest Pipeline, Logstash/Beats configuration, or application‑side code, with guidance on advantages, caveats, and handling historical data.
BeatsElasticsearchIngest Pipeline
0 likes · 5 min read

This article provides a comprehensive guide to Pandas indexing in Python, covering basic loc and iloc selection, Boolean indexing, setting and resetting indices, multi‑level indexing, index alignment, sorting, dropping, and advanced methods such as at, iat, and query, with complete code examples.
Pandasboolean indexingdata indexing
0 likes · 9 min read

This article traces the evolution of Ctrip’s travel product log platform—from a single‑table DB approach to a platform‑wide ES + HBase solution—detailing the challenges of massive data volume, the architectural decisions, RowKey design, write and query flows, and the subsequent extensions that enabled billion‑scale log storage and fast retrieval.
CtripESHBase
0 likes · 17 min read

This article presents a comprehensive overview of Apache Paimon, covering the concept of real‑time lakehouses, the underlying technologies such as LSM and merge‑on‑write, practical application cases across enterprises, and the latest frontier features like tags, branches, and advanced indexing, illustrating how Paimon bridges batch and streaming workloads in modern big‑data ecosystems.
Apache PaimonLSMReal-time Lakehouse
0 likes · 16 min read

This article details Bilibili's implementation of an Iceberg‑based lakehouse platform that unifies storage and analytics, addressing Hive’s performance and latency issues through multidimensional sorting, various file‑level indexes, cube pre‑aggregation, star‑tree structures, and an automated Magnus service for intelligent optimization, achieving near‑second query responses.
OLAPQuery Accelerationbig data
0 likes · 14 min read

The article outlines the background, design, challenges, and future roadmap of a business search platform within the Weimob Operating System, detailing its architecture, event ingestion, index building, and retrieval services that enable low‑cost, high‑performance search across multiple business domains.
Search Platformbackend architecturedata indexing
0 likes · 9 min read

A senior engineer accidentally issued a DELETE request on an ElasticSearch index holding 17 million product records, triggering a massive data loss incident, and the team’s subsequent recovery strategies, scaling challenges, and process improvements are detailed to guide backend developers.
data indexingincident responsemicroservices
0 likes · 14 min read

This article details the design and implementation of JD Retail’s product selection platform, covering its business background, core data retrieval capabilities, domain model, system architecture—including frontend configurability, backend query engine, ClickHouse indexing, and both offline and real-time data processing pipelines.
Product Selectionbig datadata indexing
0 likes · 14 min read

This guide explains how to install the Elasticsearch Python client, build a wrapper class for index management and CRUD operations, import data from MongoDB, use a Celery‑based crawler to harvest Baidu Baike content, and expose search functionality through Flask or other Python web frameworks.
ElasticsearchFlaskMongoDB
0 likes · 7 min read

This article introduces Elasticsearch as a distributed search and analytics engine, outlines its common application scenarios, provides step‑by‑step installation commands, explains core concepts such as documents and indices, and demonstrates basic indexing, retrieval, bulk processing, and aggregation operations.
ElasticsearchLog Analyticsbulk API
0 likes · 4 min read


ITPUBSep 1, 2019 · Databases
This article explains the fundamental architecture of Elasticsearch and its underlying Lucene engine, detailing the data model, index hierarchy, and the step‑by‑step write, refresh, flush, and merge processes that enable near‑real‑time search and data durability.
Lucenedata indexingsearch engine
0 likes · 8 min read


21CTOJun 10, 2019 · Databases
This tutorial introduces Elasticsearch, explains its architecture and use cases, walks through installation, index creation, mapping, CRUD operations, and demonstrates how to integrate and query Elasticsearch from Python using both the REST API and the official client library.
ElasticsearchNoSQLPython
0 likes · 13 min read

This tutorial walks you through installing Elasticsearch, creating indices, inserting and updating documents, performing searches via REST API, and integrating Elasticsearch into Python applications using both raw HTTP requests and the official Python client, illustrated with practical examples and screenshots.
ElasticsearchNoSQLREST API
0 likes · 11 min read


21CTOOct 19, 2017 · Backend Development
This tutorial walks you through installing Elasticsearch, configuring Java and network settings, understanding core concepts like nodes, clusters, indices and documents, setting up Chinese analyzers, performing CRUD operations, and executing powerful full‑text queries using the Elasticsearch REST API.
Chinese AnalyzerElasticsearchREST API
0 likes · 13 min read

Facing the challenges of searching across 60 million users and 1.3 billion products, Weidian’s engineering team designed a dump‑based indexing pipeline—Ergate—that consolidates, transforms, version‑controls, and monitors data from MySQL to HBase, enabling fast, flexible, and reliable search across massive datasets.
HBasedata indexingdump architecture
0 likes · 7 min read

This article introduces Druid as an open‑source, distributed column‑store OLAP engine, explains its architecture and indexing mechanisms, discusses real‑time and batch data ingestion for order analytics at Qunar, compares it with other engines, and shares practical tips and pitfalls.
CaravelDruidOLAP
0 likes · 8 min read

This article explains Druid’s purpose as a real‑time, distributed, column‑store OLAP engine, details its architecture and indexing techniques, discusses practical use cases and limitations, and shows how Caravel can complement Druid for visual analytics and detailed data access.
CaravelDruidOLAP
0 likes · 8 min read

This article explains how Alibaba's Jushita platform leverages Apache Solr with a wide‑table data model and a custom QParser plugin to achieve real‑time, multi‑dimensional buyer filtering that traditional relational databases cannot handle efficiently in big‑data scenarios.
Real-time QuerySolrbig data
0 likes · 10 min read
