Tagged articles
segment
25 articles · Page 1 of 1
This article breaks down ElasticSearch's performance advantages across three layers—data structures (inverted index, FST, compressed posting lists), storage (immutable segments, Doc Values), and architecture (shard parallelism, near-real-time writes, multi-level caching)—with concrete examples and interview-focused explanations.
CachingElasticsearchInterview questions
0 likes · 16 min read

A Milvus delete command looks simple, but the primary key is first stored in an L0 delta segment, then triggers compaction that merges the delete mark into L1/L2 segments, passes through four distinct segment states, and involves multiple seal strategies, rollback fields, and version‑specific behaviours, illustrating why deletion is the system's most intricate operation.
L0L1L2
0 likes · 20 min read

An INT auto‑increment primary key can exhaust its 2‑31‑1 limit in just 80 days under 10 M QPS, so the article explains how ID consumption is amplified, the high cost of migrating to BIGINT, the single‑point bottleneck of auto‑increment, and compares Snowflake, segment, UUIDv7, and Redis distributed ID solutions.
BIGINTMySQLRedis
0 likes · 18 min read

When sharding databases, independent auto‑increment IDs can collide; this article explains why the conflict occurs and evaluates five concrete solutions—UUID, MySQL auto‑increment step, segment allocation, Snowflake algorithm, and Meituan Leaf—detailing their mechanisms, trade‑offs, performance, and suitability.
Meituan LeafMySQLSharding
0 likes · 14 min read

This article explains how Kafka achieves extremely high throughput by using a Reactor‑based non‑blocking I/O model, zero‑copy data transfer, sequential disk writes, memory‑mapped files, sparse indexing, partition load‑balancing, compression, batch processing, and a lock‑free offset design.
CompressionKafkaPageCache
0 likes · 27 min read

This article dissects Guava Cache's source code, explaining its segment‑based locking, data structures, put/get implementations, cleanup and eviction mechanisms, and then contrasts its performance and design choices with the more modern Caffeine cache, highlighting why Guava falls short.
CacheCaffeineConcurrency
0 likes · 39 min read

This article explains how Kafka achieves ultra‑high throughput and low latency despite being disk‑based, covering its Reactor I/O network model, zero‑copy techniques, partitioning strategies, segment logs with sparse indexes, sequential disk writes, page cache usage, compression, batch processing, and lock‑free offset management.
KafkaReactor I/Ooffset management
0 likes · 27 min read

This article explains how Elasticsearch transforms simple keyword matching into a high‑performance, scalable search engine by using inverted indexes, term dictionaries, posting lists, term indexes, stored fields, doc values, segments, and distributed node architectures to achieve fast, reliable full‑text search on massive data sets.
ElasticsearchLuceneinverted index
0 likes · 16 min read

Since 2018 Youzan has migrated all online services to Kylin4, addressing long cube rebuilds, single‑point cache, CPU spikes, and throttling gaps by adding batch segment builds, low‑priority concurrency controls, Redis‑based query caching, parquet skew mitigation, range‑query acceleration, and class‑loader optimizations, which together doubled query‑per‑second capacity to 150, cut latency by up to 50 % and reduced CPU usage.
CubeKylinSpark
0 likes · 17 min read

This article examines various distributed ID generation techniques—including UUID, database auto‑increment, segment mode, Redis INCR, Snowflake, Meituan Leaf, Baidu UidGenerator, and Didi TinyID—detailing their principles, advantages, drawbacks, and code examples to help developers select the most suitable solution for their systems.
RedisSnowflakeUUID
0 likes · 16 min read

This article explains where MySQL keeps its data files, how InnoDB organizes tablespaces into segments, extents, pages and rows, details the Compact row format—including variable‑length field length lists, NULL‑value lists, and hidden fields—and clarifies the limits of VARCHAR and row‑overflow handling.
Database StorageExtentInnoDB
0 likes · 22 min read

This article describes how to identify and fix Greenplum standby segment failures by inspecting segment status, comparing missing files, generating a recovery configuration, executing gprecoverseg commands, and verifying that all segments return to normal operation without service interruption.
GreenplumRecoverySQL
0 likes · 17 min read

This article explains why distributed ID generation is needed, outlines business requirements such as global uniqueness and monotonicity, and compares eight practical schemes—including UUID, database auto‑increment, segment mode, Redis, Snowflake, Baidu UidGenerator, Meituan Leaf, and Didi TinyID—detailing their advantages, disadvantages, and implementation examples.
BackendSnowflakeUUID
0 likes · 16 min read

The article surveys Go‑based distributed ID generation techniques—from UUID, short‑UUID, XID, KSUID, ULID, and Snowflake to database auto‑increment, Redis, ZooKeeper, and segment/leaf approaches—detailing their characteristics, trade‑offs, implementation snippets, and strategies for ordering, uniqueness, and clock‑rollback handling.
GoRedisSnowflake
0 likes · 12 min read

This article explains how ElasticSearch achieves near real‑time search by using immutable inverted indexes, segment merging, shard distribution, and a translog for durability, while also offering practical guidance on how to study the system effectively.
Distributed SearchNear Real-Timeinverted index
0 likes · 8 min read

This article explains Kafka's storage architecture—including topics, partitions, segments, and their naming rules—along with how data is read, and details the system's reliability features such as ISR/OSR replication, leader election, producer acknowledgment levels, and delivery guarantees.
At-Least-OnceLeader ElectionStorage
0 likes · 10 min read

Elasticsearch routes client write requests to the primary shard, buffers documents in memory, logs operations to a translog, periodically refreshes to create segment files, and uses flush and merge processes to ensure data durability, optimize I/O, and consolidate segments for efficient search.
Elasticsearchrefreshsegment
0 likes · 7 min read

This article explains Elasticsearch's internal mechanisms for writing, reading, and searching data, covering the roles of coordinating nodes, primary and replica shards, buffers, translog, segment files, refresh cycles, commit and flush operations, as well as Lucene's inverted index and how deletions and updates are handled.
ElasticsearchLuceneNear Real-Time
0 likes · 10 min read

This article explains how Kafka partitions data into segments, stores each segment as paired index and log files, and uses sparse indexing to enable efficient queries, illustrating the process with examples and diagrams of segment layout and offset lookup.
IndexKafkaStorage
0 likes · 4 min read

This article explains Kafka’s workflow, detailing how topics, partitions, and segments are organized, the structure of index and log files, message composition, offset-based retrieval, and the overall data directory layout, providing a comprehensive overview of Kafka’s storage architecture.
IndexKafkaOFFSET
0 likes · 8 min read

This article explains Kafka's internal storage architecture—including topics, partitions, segments, .log and .index files—how data is read, and the various reliability mechanisms such as ISR/OSR, LEO/HW, producer acknowledgment levels, leader election strategies, and delivery semantics.
KafkaLeader ElectionProducer acks
0 likes · 9 min read

The article explains how Elasticsearch’s use of Lucene’s flush mechanism, concurrent shard writes, and IndexWriter buffering lead to an excess of small segments, outlines the flush conditions, and offers guidance on managing write concurrency for better performance.
ElasticsearchIndexWriterLucene
0 likes · 10 min read

This article reviews various distributed ID generation strategies—including database auto‑increment tables, dual‑master MySQL setups, segment allocation, the Snowflake algorithm, and Redis INCR—detailing their designs, SQL examples, scalability trade‑offs, and open‑source implementations.
Snowflakedatabasedistributed-id
0 likes · 13 min read

This article explains how Oracle's logical structures—blocks, segments, tablespaces, and rowids—affect SQL performance, presents detailed experiments on block capacity, row migration, segment handling, tablespace behavior, and provides practical optimization guidelines based on observed logical read reductions.
Block SizeLogical StructureOracle
0 likes · 11 min read

Kafka stores each topic as partitions that are divided into sequential segment files containing paired .log data and .index files, using global offsets and sparse memory‑mapped indexes to enable fast offset‑based lookups, efficient deletions, and minimal disk I/O in real‑world deployments.
KafkaMessage Queuefile storage
0 likes · 9 min read
