BestHub
Sign in
Tagged articles
65 articles
Page 1 of 1
James' Growth Diary
James' Growth Diary
May 3, 2026 · Artificial Intelligence

How Claude Code Handles max_output_tokens and Model Downgrade to Keep Agents Running

The article explains Claude Code's multi‑level fault‑tolerance for max_output_tokens errors, detailing dynamic token allocation, automatic model downgrade, environment‑variable controls, StopFailure hooks, and their coordination with compaction to prevent agents from getting stuck during long‑running tasks.

AI AgentClaude CodeEnvironment Variables
0 likes · 13 min read
How Claude Code Handles max_output_tokens and Model Downgrade to Keep Agents Running
AI Tech Publishing
AI Tech Publishing
Apr 27, 2026 · Artificial Intelligence

Context Window Strategies in Agent Harnesses: Pi, OpenClaw, Claude Code, Letta, Alyx

The article analyzes how five Agent Harness frameworks—Pi, OpenClaw, Claude Code, Letta, and Alyx—handle context windows, file pagination, tool result limits, session pruning, and sub‑agent isolation, revealing convergent design patterns that treat the context as a managed memory system.

Agent HarnessContext managementFile Pagination
0 likes · 21 min read
Context Window Strategies in Agent Harnesses: Pi, OpenClaw, Claude Code, Letta, Alyx
ByteDance SE Lab
ByteDance SE Lab
Apr 17, 2026 · Industry Insights

How DisCoGC Cuts Storage Costs by 20%: A Deep Dive into ByteStore’s New GC Paradigm

This article analyzes the DisCoGC algorithm introduced by ByteDance, explaining how its discard‑centric garbage collection eliminates the write‑amplification vs. space‑amplification trade‑off in log‑structured storage, details the engineering challenges of multi‑layer deployment, and presents production results showing up to 20% TCO reduction without impacting latency.

Cost reductionGarbage CollectionPerformance Optimization
0 likes · 19 min read
How DisCoGC Cuts Storage Costs by 20%: A Deep Dive into ByteStore’s New GC Paradigm
Architect
Architect
Apr 16, 2026 · Artificial Intelligence

Mastering Claude Code: Session Management Strategies for 1M Context Windows

This article analyzes Anthropic's Claude Code session‑management features, explaining how context rot limits effective token usage, what the 1 M‑token window actually stores, and when to use the five built‑in actions—Continue, /rewind, /clear, Compact and Subagent—to keep long‑running AI tasks reliable and efficient.

AI agentsClaude CodeContext Window
0 likes · 18 min read
Mastering Claude Code: Session Management Strategies for 1M Context Windows
AI Tech Publishing
AI Tech Publishing
Mar 18, 2026 · Artificial Intelligence

How Context Engineering Turns AI Agents from ‘Usable’ to ‘Highly Effective’

The article explains how organizing the prompt, tool schemas, dialogue history, and retrieved documents—collectively the context window—affects an AI agent’s decisions, introduces the concepts of Lost‑in‑the‑Middle, Thinking Tokens, tool‑response caching, compaction versus SubAgent strategies, and shows a step‑by‑step evolution that raised accuracy from 60 % to over 95 %.

AI agentsContext EngineeringLLM
0 likes · 9 min read
How Context Engineering Turns AI Agents from ‘Usable’ to ‘Highly Effective’
Tencent Cloud Developer
Tencent Cloud Developer
Mar 4, 2026 · Artificial Intelligence

How OpenClaw Uses a Multi‑Layer Defense System to Prevent LLM Context Overflow

The article provides a detailed technical walkthrough of OpenClaw's three‑stage context‑management framework—including pre‑emptive pruning, LLM‑driven compaction, and overflow‑recovery truncation—showing how each layer protects long‑running AI agent sessions from exceeding token windows while preserving essential information.

Context managementLLMOpenClaw
0 likes · 27 min read
How OpenClaw Uses a Multi‑Layer Defense System to Prevent LLM Context Overflow
Architect
Architect
Feb 27, 2026 · Artificial Intelligence

Turning AI Agents into Deliverable Workflows: Skills, Shell, and Compaction Explained

The article explains why writing code alone does not guarantee delivery, outlines three core challenges for long‑running agents—process reuse, execution, and context continuity—and presents a practical framework of Skills, Shell, and Compaction together with ten actionable recommendations, security guidelines, and implementation steps for teams.

AI agentsShellSkills
0 likes · 18 min read
Turning AI Agents into Deliverable Workflows: Skills, Shell, and Compaction Explained
StarRocks
StarRocks
Dec 11, 2025 · Databases

How StarRocks Redesigns Bulk Import to Cut Small Files and Boost Throughput

This article explains how StarRocks mitigates the hidden risks of massive one‑time data imports in a storage‑compute separated architecture by redesigning the write path to spill to local disk, merge centrally, and write to object storage, resulting in fewer small files, higher write throughput, and more stable query performance.

Bulk ImportS3StarRocks
0 likes · 12 min read
How StarRocks Redesigns Bulk Import to Cut Small Files and Boost Throughput
Full-Stack DevOps & Kubernetes
Full-Stack DevOps & Kubernetes
May 28, 2025 · Operations

How to Fix etcd “NOSPACE” Errors in Kubernetes Clusters

When a Kubernetes cluster’s etcd reaches its default 2 GB quota, it triggers a “NOSPACE” alarm that blocks all write operations, causing critical services to fail; this guide explains the root cause, how to diagnose the issue with etcdctl, and step‑by‑step remediation including compaction, defragmentation, and quota expansion.

KubernetesNOSPACEcompaction
0 likes · 7 min read
How to Fix etcd “NOSPACE” Errors in Kubernetes Clusters
DataFunSummit
DataFunSummit
Dec 27, 2024 · Big Data

Tencent Real-time Lakehouse Intelligent Optimization Practice

This presentation describes Tencent's real-time lakehouse architecture, including data lake compute, management, and storage layers, and details the intelligent optimization services—such as compaction, indexing, clustering, and auto-engine—designed to improve query performance, storage cost, and operational efficiency for large-scale data processing.

AutoEngineFlinkIceberg
0 likes · 11 min read
Tencent Real-time Lakehouse Intelligent Optimization Practice
Tencent Advertising Technology
Tencent Advertising Technology
Dec 6, 2024 · Big Data

Building a High‑Performance Advertising Feature Data Lake with Apache Iceberg at Tencent

Tencent's advertising team replaced a traditional HDFS‑Hive warehouse with an Apache Iceberg‑based data lake, adding primary‑key tables, multi‑stream merging, adaptive compaction, and Spark SPJ optimizations to achieve minute‑level feature update latency, 10× back‑fill speed, and up to 60% storage savings.

Big DataCDCData Lake
0 likes · 25 min read
Building a High‑Performance Advertising Feature Data Lake with Apache Iceberg at Tencent
Big Data Technology & Architecture
Big Data Technology & Architecture
Oct 28, 2024 · Big Data

Key Considerations for Using Paimon Primary Key Tables

This article explains the characteristics of Paimon primary key tables, covering bucket selection, cross‑partition update issues, recommended record‑level expiration settings, and two approaches to handle file compaction, including configuration tweaks and dedicated compaction tasks.

Big DataBucketFlink
0 likes · 6 min read
Key Considerations for Using Paimon Primary Key Tables
Linux Kernel Journey
Linux Kernel Journey
Oct 17, 2024 · Fundamentals

Inside Linux Memory Compaction: A Source‑Code Walkthrough of Memory Management

The article explains how Linux manages memory page watermarks, when the allocator falls back to kswapd, and the exact conditions that trigger direct compaction via __alloc_pages_direct_compact, then walks through the core compaction functions—try_to_compact_pages, compact_zone_order, compact_zone, and the page‑migration helpers—illustrated with flow diagrams and real kernel code.

CKernelLinux
0 likes · 37 min read
Inside Linux Memory Compaction: A Source‑Code Walkthrough of Memory Management
Linux Kernel Journey
Linux Kernel Journey
Sep 12, 2024 · Fundamentals

Understanding Linux Memory Allocation: Fast Path vs. Slow Path in the Source Code

This article dissects the Linux kernel's page allocation mechanisms, explaining how alloc_pages() follows a fast‑path using low watermarks and falls back to a slow‑path that triggers kswapd, direct reclaim, and compaction, while also detailing the corresponding page‑freeing functions and their internal data structures.

Linux kernelbuddy allocatorcompaction
0 likes · 30 min read
Understanding Linux Memory Allocation: Fast Path vs. Slow Path in the Source Code
Sohu Tech Products
Sohu Tech Products
Sep 11, 2024 · Big Data

Tencent Real-time Lakehouse Intelligent Optimization Practice

Tencent’s real‑time lakehouse combines Spark, Flink, StarRocks and Presto compute layers with Iceberg‑based management and HDFS/COS storage, and its Intelligent Optimize Service—comprising Compaction, Expiration, Cleaning, Clustering, Index and Auto‑Engine modules—automatically reduces merge time, improves query performance, enables secondary indexing, and dynamically routes hot partitions, while future plans target cold/hot separation, materialized view acceleration, and AI‑driven optimizations.

Big DataLakehousePyIceberg
0 likes · 12 min read
Tencent Real-time Lakehouse Intelligent Optimization Practice
Big Data Technology & Architecture
Big Data Technology & Architecture
Jul 26, 2024 · Databases

Apache Doris Architecture and Common Q&A: Read/Write Flow, Replication Consistency, Storage, and High Availability

This article provides a comprehensive overview of Apache Doris, explaining its frontend and backend nodes, storage structures such as tablets, rowsets, and segments, replication mechanisms, partitioning versus bucketing, indexing types, compaction processes, and high‑availability strategies through a detailed Q&A format.

Apache DorisBig DataDatabase Architecture
0 likes · 22 min read
Apache Doris Architecture and Common Q&A: Read/Write Flow, Replication Consistency, Storage, and High Availability
Big Data Technology & Architecture
Big Data Technology & Architecture
Jul 25, 2024 · Big Data

Fundamental Concepts and File Layout of Paimon: Snapshots, Partitions, Buckets, Consistency, and Compaction

This article explains Paimon's core concepts—including snapshots, partitions, buckets, consistency guarantees, file layout, LSM‑tree organization, and compaction strategies—while also covering table management tasks such as snapshot expiration, rollback, partition expiration, and small‑file mitigation techniques.

Big DataBucketsLSM‑Tree
0 likes · 12 min read
Fundamental Concepts and File Layout of Paimon: Snapshots, Partitions, Buckets, Consistency, and Compaction
StarRocks
StarRocks
Jun 18, 2024 · Databases

How StarRocks Compaction Boosts Query Performance: Mechanics, Tuning, and Best Practices

This article explains StarRocks' compaction process that merges multiple data versions into larger files to reduce I/O, details the scheduler and executor roles, shows how to monitor and control compaction via SQL commands, and provides tuning parameters and best‑practice recommendations for optimal performance.

Data ManagementSQLStarRocks
0 likes · 21 min read
How StarRocks Compaction Boosts Query Performance: Mechanics, Tuning, and Best Practices
Big Data Technology & Architecture
Big Data Technology & Architecture
Feb 18, 2024 · Big Data

Understanding Apache Paimon Table Modes and Their Use Cases

Apache Paimon provides multiple table modes—including primary key tables with fixed or dynamic buckets, Append scalable and queue tables—each with specific configurations, compaction behavior, and suitable scenarios, and the article explains their structures, performance considerations, and how to use them with Flink.

Apache PaimonAppend TableBig Data
0 likes · 12 min read
Understanding Apache Paimon Table Modes and Their Use Cases
Huolala Tech
Huolala Tech
Dec 27, 2023 · Big Data

How HBase Compaction Tuning Boosts Performance at Scale

This article explains LSM‑Tree based HBase compaction concepts, compares Minor and Major compactions, and shares practical tuning steps—including disabling automatic major compactions, controlling merge size, leveraging off‑peak windows, and improving merge efficiency—to reduce I/O, CPU usage, and latency in production environments.

Big DataDatabase OptimizationHBase
0 likes · 11 min read
How HBase Compaction Tuning Boosts Performance at Scale
Deepin Linux
Deepin Linux
Dec 9, 2023 · Fundamentals

Linux Page Reclaim Mechanism and Memory Compaction: Detailed Source Code Analysis

This article explains the Linux page‑reclaim mechanism, its goals, common techniques, the allocation paths, LRU data structures, and provides an in‑depth walkthrough of the kernel source code for slow‑path reclaim, direct reclaim, and memory compaction, including all relevant functions and code snippets.

Linuxcompactionpage reclaim
0 likes · 80 min read
Linux Page Reclaim Mechanism and Memory Compaction: Detailed Source Code Analysis
Alibaba Cloud Native
Alibaba Cloud Native
Dec 6, 2023 · Cloud Native

How RocketMQ Implements Random Indexing for Cloud‑Native Storage

This article explains RocketMQ's random indexing mechanism, detailing its on‑disk three‑segment hash table structure, the compact format conversion process, multi‑threaded write and query workflows, layered system design, crash‑recovery strategy, and comparisons with RocksDB and InnoDB storage engines.

Distributed SystemsMessage IndexingRocketMQ
0 likes · 16 min read
How RocketMQ Implements Random Indexing for Cloud‑Native Storage
Aikesheng Open Source Community
Aikesheng Open Source Community
Nov 8, 2023 · Databases

Analyzing OceanBase Freeze Dump Process via Log Parsing

This article explains how to parse OceanBase logs to trace the tenant freeze dump workflow, detailing the roles and log sequences of the freeze check thread, LSFreeze, Flush, DagScheduler, and MiniMerge threads, and illustrating each step with actual log excerpts and code snippets.

DAGFreeze ProcessOceanBase
0 likes · 16 min read
Analyzing OceanBase Freeze Dump Process via Log Parsing
DataFunTalk
DataFunTalk
Aug 30, 2023 · Big Data

Design and Implementation of Baidu Cloud Block Storage EC System for Large‑Scale Data

This article presents Baidu Cloud's block storage architecture, comparing replication and erasure‑coding fault‑tolerance methods, detailing the challenges of applying EC to mutable block data, and describing a two‑layer append‑engine solution with selective 3‑replica caching, cost‑benefit compaction, and performance optimizations for low‑cost, high‑throughput storage.

Big Dataappend engineblock storage
0 likes · 14 min read
Design and Implementation of Baidu Cloud Block Storage EC System for Large‑Scale Data
Sohu Tech Products
Sohu Tech Products
Aug 16, 2023 · Big Data

Understanding HBase Compaction: Principles, Process, Throttling Strategies and Real‑World Optimizations

This article explains HBase’s LSM‑Tree compaction fundamentals—including minor and major compaction triggers, file‑selection policies, dynamic throughput throttling, and practical tuning examples that show how adjusting size limits, thread pools, and off‑peak settings can dramatically improve read latency and cluster stability.

Big DataHBaseJava
0 likes · 35 min read
Understanding HBase Compaction: Principles, Process, Throttling Strategies and Real‑World Optimizations
vivo Internet Technology
vivo Internet Technology
Jul 26, 2023 · Big Data

Understanding HBase Compaction: Principles, Process, Throttling Strategies, and Optimization Cases

Understanding HBase compaction involves knowing its minor and major merge types, trigger mechanisms, file‑selection policies such as RatioBased and Exploring, throttling controls based on file count, and practical tuning of key parameters to avoid latency spikes, as illustrated by real‑world production cases.

Big DataHBasecompaction
0 likes · 36 min read
Understanding HBase Compaction: Principles, Process, Throttling Strategies, and Optimization Cases
Huolala Tech
Huolala Tech
May 25, 2023 · Big Data

How Huolala Solved HBase Bulkload Challenges: A Practical Guide

This article details Huolala’s experience building a unified Hive‑to‑HBase pipeline, addressing low development efficiency, lack of monitoring, and HBase instability by evaluating two architectures, implementing a generic Transform tool, optimizing compaction and DistCp, and establishing stability and data‑validation mechanisms.

DistcpHBasebulkload
0 likes · 12 min read
How Huolala Solved HBase Bulkload Challenges: A Practical Guide
Aikesheng Open Source Community
Aikesheng Open Source Community
Mar 9, 2023 · Databases

In‑Depth Exploration of OceanBase Hierarchical Dump and Compaction Mechanisms

This article explains the LSM‑Tree foundation of OceanBase, details its tiered and leveled compaction strategies, and presents two experiments that observe Mini and Minor compactions under different configuration parameters, revealing how minor freeze and trigger settings affect data movement between L0 and L1 layers.

Database StorageLSM‑TreeMini Compaction
0 likes · 13 min read
In‑Depth Exploration of OceanBase Hierarchical Dump and Compaction Mechanisms
DataFunSummit
DataFunSummit
Feb 28, 2023 · Big Data

Iceberg Technology Overview and Its Application at Xiaomi: Practices, Stream‑Batch Integration, and Future Plans

This article introduces the Iceberg table format, explains its core architecture and advantages such as transactionality, implicit partitioning and row‑level updates, details Xiaomi's practical deployments—including CDC pipelines, partition strategies, compaction services, and stream‑batch integration—and outlines future development directions.

Data LakeFlinkIceberg
0 likes · 20 min read
Iceberg Technology Overview and Its Application at Xiaomi: Practices, Stream‑Batch Integration, and Future Plans
Big Data Technology Architecture
Big Data Technology Architecture
Aug 13, 2022 · Big Data

Apache Doris at Xiaomi: Architecture Evolution, Performance Optimizations, and Production Practices

This article details Xiaomi's three‑year journey of adopting Apache Doris across dozens of internal services, describing the transition from a Spark‑SQL‑based Lambda architecture to a unified MPP database, performance benchmarks, data ingestion pipelines, compaction tuning, two‑phase commit, single‑replica writes, monitoring, and community contributions.

Apache DorisData WarehouseMPP
0 likes · 19 min read
Apache Doris at Xiaomi: Architecture Evolution, Performance Optimizations, and Production Practices
StarRocks
StarRocks
May 12, 2022 · Databases

How StarRocks’ Primary Key Model Delivers 3‑5× Faster Real‑Time Queries

This article explains the design and implementation of StarRocks 2.x Primary Key tables, covering real‑time update mechanisms, write and commit workflows, in‑memory primary indexing, compaction, read‑path optimizations, performance benchmarks, and upcoming features such as partial and conditional updates.

OLAPStarRockscompaction
0 likes · 19 min read
How StarRocks’ Primary Key Model Delivers 3‑5× Faster Real‑Time Queries
NetEase Cloud Music Tech Team
NetEase Cloud Music Tech Team
Mar 16, 2022 · Databases

RDB: Cloud Music's Customized Algorithm Feature KV Storage System Based on RocksDB

To meet Cloud Music’s massive algorithm‑feature KV storage needs, the team built RDB—a RocksDB‑based engine within Tair—adding bulk‑load, dual‑version imports, KV‑separation, in‑place sequence appends and protobuf field updates, cutting storage cost, write amplification and latency while scaling to billions of records and millions of QPS.

Algorithm FeaturesKV SeparationKV storage
0 likes · 16 min read
RDB: Cloud Music's Customized Algorithm Feature KV Storage System Based on RocksDB
DataFunTalk
DataFunTalk
Feb 25, 2022 · Big Data

Tencent's Application of Apache Iceberg for Real‑Time Data Lake Ingestion, Governance, and Query Optimization

This article explains how Tencent leverages Apache Iceberg together with Flink to build a real‑time data lake pipeline, covering data ingestion, Iceberg's snapshot‑based read/write model, compaction and governance services, Z‑order based query optimization, performance results, and future roadmap.

Apache IcebergBig DataData Lake
0 likes · 24 min read
Tencent's Application of Apache Iceberg for Real‑Time Data Lake Ingestion, Governance, and Query Optimization
Tencent Cloud Developer
Tencent Cloud Developer
Dec 10, 2020 · Databases

Understanding LevelDB Architecture, Read/Write Flow, and Compaction Process

LevelDB stores data using an in‑memory Memtable that flushes to immutable tables and disk‑based SSTables, writes are logged then batched and applied through a writer queue, reads check Memtable, immutable Memtable, then SSTables, and background compactions merge tables to improve read performance and reclaim space.

Database InternalsLSM‑TreeLevelDB
0 likes · 16 min read
Understanding LevelDB Architecture, Read/Write Flow, and Compaction Process
Big Data Technology & Architecture
Big Data Technology & Architecture
Aug 27, 2020 · Big Data

HBase Architecture, Components, and Operations Overview

This article provides a comprehensive overview of Apache HBase’s architecture, detailing its core components such as RegionServer, HMaster, ZooKeeper, WAL, MemStore, and HFiles, and explains key processes including read/write paths, compaction, region splitting, load balancing, and recovery mechanisms.

Big DataDatabase ArchitectureDistributed Systems
0 likes · 17 min read
HBase Architecture, Components, and Operations Overview
Big Data Technology & Architecture
Big Data Technology & Architecture
Jun 10, 2020 · Databases

Understanding HBase Compaction: Types, Triggers, Algorithms, and Impact on Read/Write Performance

This article explains HBase compaction—a key operation in the Log‑Structured Merge‑Tree model—covering minor and major compaction differences, trigger conditions, configuration parameters, selection algorithms, thread‑pool handling, and the effects on read and write performance in a big‑data database environment.

HBaseLSMbigdata
0 likes · 10 min read
Understanding HBase Compaction: Types, Triggers, Algorithms, and Impact on Read/Write Performance
Big Data Technology Architecture
Big Data Technology Architecture
May 22, 2020 · Databases

HBase Compaction Types and Parameter Tuning Guide

This article explains how HBase uses WAL and MemStore to create HFiles, describes the two compaction types (Minor and Major), and provides detailed recommendations for tuning key compaction-related configuration parameters to improve query performance and reduce HDFS impact.

HBaseTuningcompaction
0 likes · 4 min read
HBase Compaction Types and Parameter Tuning Guide
Big Data Technology & Architecture
Big Data Technology & Architecture
Mar 30, 2020 · Databases

HBase Optimization: JVM Tuning, Region Split Policies, BlockCache, and Compaction Strategies

This guide explains how to optimize HBase performance by adjusting JVM memory settings, selecting appropriate garbage collectors, configuring MSLAB and in‑memory compaction, choosing region split policies, tuning BlockCache implementations, and applying suitable compaction policies for different workloads.

Big DataBlockCacheHBase
0 likes · 18 min read
HBase Optimization: JVM Tuning, Region Split Policies, BlockCache, and Compaction Strategies
DataFunTalk
DataFunTalk
Mar 24, 2020 · Databases

ByteDance’s Enhancements to RocksDB: LazyBuffer, Adaptive Map, KV Separation, Multi‑Index, Extreme Compression, and New Hardware Support

This article describes ByteDance’s extensive improvements to the RocksDB storage engine—including LazyBuffer, Adaptive Map‑based lazy compaction, KV separation, adaptive multi‑index support, extreme compression techniques, and hardware acceleration—to reduce amplification, improve performance, and lower costs for large‑scale database workloads.

Hardware accelerationKV SeparationRocksDB
0 likes · 14 min read
ByteDance’s Enhancements to RocksDB: LazyBuffer, Adaptive Map, KV Separation, Multi‑Index, Extreme Compression, and New Hardware Support
Big Data Technology & Architecture
Big Data Technology & Architecture
Jan 7, 2020 · Big Data

Why Small Files Are a Problem in Big Data and How Delta Lake Compaction Solves It

This article examines the root causes and performance impact of massive small-file proliferation in traditional data warehouses, explains why HDFS metadata limits scalability, and details how Delta Lake’s custom compaction process can safely merge these files for append-only tables without disrupting reads or writes.

Delta LakeHDFSSmall Files
0 likes · 5 min read
Why Small Files Are a Problem in Big Data and How Delta Lake Compaction Solves It
58 Tech
58 Tech
Dec 2, 2019 · Databases

Optimizing RocksDB Compaction Rate Limiting to Reduce IO Spikes in WTable

This article analyzes RocksDB's compaction rate‑limiting source code and presents practical tuning methods—both fixed and auto‑tuned—to mitigate IO spikes in the distributed KV store WTable, improving real‑time read/write latency and stability.

IO optimizationRocksDBcompaction
0 likes · 7 min read
Optimizing RocksDB Compaction Rate Limiting to Reduce IO Spikes in WTable
Big Data Technology Architecture
Big Data Technology Architecture
Aug 16, 2019 · Big Data

In‑Depth Overview of HBase Architecture

This article provides a comprehensive, illustrated explanation of Apache HBase's architecture, covering its master‑slave components, region management, Zookeeper coordination, data flow for reads and writes, storage structures, compaction processes, fault recovery, and the system's strengths and limitations within the Hadoop ecosystem.

Distributed SystemsHBaseHadoop
0 likes · 21 min read
In‑Depth Overview of HBase Architecture
Alibaba Cloud Developer
Alibaba Cloud Developer
Jun 24, 2019 · Databases

How X‑Engine Redefines LSM Storage for High‑Performance E‑Commerce

This article provides an in‑depth technical overview of Alibaba's X‑Engine storage engine, explaining its LSM‑based architecture, compaction optimizations, transaction pipeline, caching strategies, and how these innovations enable low‑cost, high‑throughput OLTP for large‑scale e‑commerce workloads.

LSMPolarDB-XStorage Engine
0 likes · 18 min read
How X‑Engine Redefines LSM Storage for High‑Performance E‑Commerce
Big Data Technology Architecture
Big Data Technology Architecture
May 27, 2019 · Databases

Understanding HBase Compaction: Types, Triggers, Parameters, and Performance Impact

This article explains HBase's compaction mechanism, covering why it is needed, the differences between minor and major compaction, the conditions that trigger compaction, key configuration parameters, thread‑pool handling, compaction policies, and how compaction influences read and write performance in a large‑scale NoSQL database.

HBasebigdatacompaction
0 likes · 12 min read
Understanding HBase Compaction: Types, Triggers, Parameters, and Performance Impact
Qunar Tech Salon
Qunar Tech Salon
Apr 18, 2018 · Databases

FPGA-Accelerated X-Engine Storage Engine for High‑Performance OLTP

This article presents the design, implementation, and evaluation of X‑Engine, a next‑generation LSM‑Tree based storage engine that offloads compaction to FPGA, achieving up to 50% KV‑interface and 40% SQL‑interface performance gains for write‑intensive OLTP workloads.

FPGALSM‑TreeStorage Engine
0 likes · 19 min read
FPGA-Accelerated X-Engine Storage Engine for High‑Performance OLTP
Alibaba Cloud Developer
Alibaba Cloud Developer
Apr 9, 2018 · Databases

How FPGA Acceleration Supercharges X-Engine’s Compaction for 10× MySQL Performance

This article introduces Alibaba’s X‑Engine storage engine, the foundation of the next‑generation distributed database X‑DB, and explains how FPGA‑accelerated compaction and asynchronous scheduling dramatically improve write‑intensive OLTP performance, reduce CPU contention, and achieve up to 50 % throughput gains while maintaining fault tolerance.

FPGAHardware accelerationLSM‑Tree
0 likes · 21 min read
How FPGA Acceleration Supercharges X-Engine’s Compaction for 10× MySQL Performance
Taobao Frontend Technology
Taobao Frontend Technology
Jul 6, 2017 · Databases

Understanding LevelDB: Architecture, Interfaces, and New Features

LevelDB, Google's high-performance key‑value store built on LSM trees, uses an in‑memory skip‑list, immutable memtables, and sstable files organized in multi‑level compaction, offering interfaces for creation, reads, writes, snapshots, and new features like fuzzy search and JSON storage, all explained with diagrams.

Database ArchitectureLSM‑TreeLevelDB
0 likes · 11 min read
Understanding LevelDB: Architecture, Interfaces, and New Features