BestHub
Sign in
Tag

Segment

0 views collected around this technical thread.

Sanyou's Java Diary
Sanyou's Java Diary
Oct 10, 2024 · Big Data

Why Kafka Handles Millions of Messages Per Second: Inside Its High‑Performance Architecture

This article explains how Kafka achieves ultra‑high throughput and low latency despite being disk‑based, covering its Reactor I/O network model, zero‑copy techniques, partitioning strategies, segment logs with sparse indexes, sequential disk writes, page cache usage, compression, batch processing, and lock‑free offset management.

Big DataKafkaOffset Management
0 likes · 27 min read
Why Kafka Handles Millions of Messages Per Second: Inside Its High‑Performance Architecture
IT Services Circle
IT Services Circle
Jun 24, 2024 · Databases

Understanding Elasticsearch Architecture: Inverted Index, Term Dictionary, Segments, and Distributed Search

This article explains how Elasticsearch transforms simple keyword matching into a high‑performance, scalable search engine by using inverted indexes, term dictionaries, posting lists, term indexes, stored fields, doc values, segments, and distributed node architectures to achieve fast, reliable full‑text search on massive data sets.

ElasticsearchInverted IndexLucene
0 likes · 16 min read
Understanding Elasticsearch Architecture: Inverted Index, Term Dictionary, Segments, and Distributed Search
Youzan Coder
Youzan Coder
Aug 8, 2023 · Big Data

Kylin4 Deployment and Performance Optimizations at Youzan

Since 2018 Youzan has migrated all online services to Kylin4, addressing long cube rebuilds, single‑point cache, CPU spikes, and throttling gaps by adding batch segment builds, low‑priority concurrency controls, Redis‑based query caching, parquet skew mitigation, range‑query acceleration, and class‑loader optimizations, which together doubled query‑per‑second capacity to 150, cut latency by up to 50 % and reduced CPU usage.

Big DataCubeKylin
0 likes · 17 min read
Kylin4 Deployment and Performance Optimizations at Youzan
Aikesheng Open Source Community
Aikesheng Open Source Community
Dec 5, 2022 · Databases

Greenplum Standby Failure Diagnosis and Recovery Procedure

This article describes how to identify and fix Greenplum standby segment failures by inspecting segment status, comparing missing files, generating a recovery configuration, executing gprecoverseg commands, and verifying that all segments return to normal operation without service interruption.

DatabaseGreenplumRecovery
0 likes · 17 min read
Greenplum Standby Failure Diagnosis and Recovery Procedure
Top Architect
Top Architect
Nov 29, 2022 · Backend Development

Distributed ID Generation Strategies: UUID, Auto‑Increment, Segment Mode, Redis, Snowflake, Baidu UidGenerator, Meituan Leaf, Didi TinyID

This article explains why distributed ID generation is needed, outlines business requirements such as global uniqueness and monotonicity, and compares eight practical schemes—including UUID, database auto‑increment, segment mode, Redis, Snowflake, Baidu UidGenerator, Meituan Leaf, and Didi TinyID—detailing their advantages, disadvantages, and implementation examples.

RedisSegmentUUID
0 likes · 16 min read
Distributed ID Generation Strategies: UUID, Auto‑Increment, Segment Mode, Redis, Snowflake, Baidu UidGenerator, Meituan Leaf, Didi TinyID
Tencent Cloud Developer
Tencent Cloud Developer
Aug 24, 2022 · Backend Development

Go Distributed ID Generation: UUID, ShortUUID, XID, KSUID, ULID, Snowflake, and Segment Modes

The article surveys Go‑based distributed ID generation techniques—from UUID, short‑UUID, XID, KSUID, ULID, and Snowflake to database auto‑increment, Redis, ZooKeeper, and segment/leaf approaches—detailing their characteristics, trade‑offs, implementation snippets, and strategies for ordering, uniqueness, and clock‑rollback handling.

DatabaseGoRedis
0 likes · 12 min read
Go Distributed ID Generation: UUID, ShortUUID, XID, KSUID, ULID, Snowflake, and Segment Modes
Top Architect
Top Architect
Nov 7, 2021 · Backend Development

Understanding Near Real‑Time Search and Core Architecture of ElasticSearch

This article explains how ElasticSearch achieves near real‑time search by using immutable inverted indexes, segment merging, shard distribution, and a translog for durability, while also offering practical guidance on how to study the system effectively.

ElasticsearchInverted IndexSegment
0 likes · 8 min read
Understanding Near Real‑Time Search and Core Architecture of ElasticSearch
IT Architects Alliance
IT Architects Alliance
Aug 29, 2021 · Big Data

Kafka Storage Mechanism and Reliability Guarantees

This article explains Kafka's storage architecture—including topics, partitions, segments, and their naming rules—along with how data is read, and details the system's reliability features such as ISR/OSR replication, leader election, producer acknowledgment levels, and delivery guarantees.

KafkaLeader ElectionReliability
0 likes · 10 min read
Kafka Storage Mechanism and Reliability Guarantees
Big Data Technology Architecture
Big Data Technology Architecture
Jun 29, 2020 · Fundamentals

Kafka Storage Mechanism and Reliability Guarantees

This article explains Kafka's internal storage architecture—including topics, partitions, segments, .log and .index files—how data is read, and the various reliability mechanisms such as ISR/OSR, LEO/HW, producer acknowledgment levels, leader election strategies, and delivery semantics.

KafkaLeader ElectionProducer Acks
0 likes · 9 min read
Kafka Storage Mechanism and Reliability Guarantees