BestHub
Sign in
Tagged articles

High Throughput

67 articles · Page 1 of 1
Cloud Architecture
Cloud Architecture
Aug 7, 2026 · Backend Development

Building a Trillion‑Message Queue: Kafka‑Level Architecture and Implementation

This article explains why and how to build a Kafka‑grade message‑queue kernel from scratch, detailing functional and non‑functional goals, core design principles, storage layout, replication, consumer‑group coordination, performance optimizations, deployment on Kubernetes, and a step‑by‑step roadmap to production‑grade reliability.

Distributed LogHigh ThroughputKafka
0 likes · 40 min read
Building a Trillion‑Message Queue: Kafka‑Level Architecture and Implementation
21CTO
21CTO
Aug 4, 2026 · Backend Development

How Zalando Achieved 1M RPS with an In‑Process Client‑Side Load Balancer

Zalando’s engineering team redesigned its high‑throughput product‑read API by moving 100‑fold internal fan‑out routing into an in‑process client‑side load balancer, cutting tail latency, reducing infrastructure costs by over 75%, and improving observability while keeping the external Skipper edge router unchanged.

High ThroughputKubernetesLoad Balancing
0 likes · 7 min read
How Zalando Achieved 1M RPS with an In‑Process Client‑Side Load Balancer
Coder Trainee
Coder Trainee
Jun 28, 2026 · Fundamentals

Java Performance Tuning Part 5: Hands‑On GC Optimization from G1 to ZGC

This article walks through Java GC tuning by defining low‑latency and high‑throughput goals, comparing major collectors, presenting G1 and ZGC configuration examples, and demonstrating a real‑world payment system case where pause times were reduced from 150‑200 ms to under 50 ms.

G1GCHigh Throughput
0 likes · 8 min read
Java Performance Tuning Part 5: Hands‑On GC Optimization from G1 to ZGC
Cloud Architecture
Cloud Architecture
Jun 7, 2026 · Cloud Native

Deep Dive into Kubernetes Scheduler: From Scheduling Cycle to Managing Ten‑Thousands of Pods

This guide explains why the kube‑scheduler becomes a bottleneck in large clusters, walks through its architecture, scheduling cycle, queue mechanics, constraint handling, plugin framework, multi‑scheduler designs, production‑grade tuning, observability, and real‑world case studies for achieving stable, high‑throughput pod placement.

High ThroughputSchedulingkube-scheduler
0 likes · 37 min read
Deep Dive into Kubernetes Scheduler: From Scheduling Cycle to Managing Ten‑Thousands of Pods
Su San Talks Tech
Su San Talks Tech
May 19, 2026 · Interview Experience

Designing a Hundred‑Billion‑Scale Message Queue: A ByteDance Interview Walkthrough

This article walks through the interview question of designing a message queue that handles billions of messages daily and peaks at millions of QPS, covering traffic calculations, core roles, storage and throughput techniques, scalability, high availability, observability, framework comparisons, a real‑world case study, and key follow‑up interview topics.

High ThroughputKafkaMessage Queue
0 likes · 12 min read
Designing a Hundred‑Billion‑Scale Message Queue: A ByteDance Interview Walkthrough
Machine Heart
Machine Heart
May 7, 2026 · Artificial Intelligence

Photo‑Level Simulation Bridges Vision Gap for Robot Learning (GS‑Playground, RSS 2026)

GS‑Playground is a next‑generation visual‑high‑fidelity robot simulator that cuts photo‑level rendering cost, automates asset creation, and narrows the Sim2Real gap, achieving up to 10,000 FPS on RTX 4090 and outperforming MuJoCo by 32× while supporting full‑stack parallel physics, 3DGS rendering, and end‑to‑end Real2Sim pipelines.

3D Gaussian SplattingHigh ThroughputSimulation
0 likes · 10 min read
Photo‑Level Simulation Bridges Vision Gap for Robot Learning (GS‑Playground, RSS 2026)
LuTiao Programming
LuTiao Programming
Mar 23, 2026 · Backend Development

Stop Using Cron: From Single-Node to Distributed Scheduling to Reach >1k Tasks/s

The article explains why traditional Cron‑based scheduling fails under high load, walks through three architectural stages—from database polling to Redis delay queues and finally a full distributed scheduler—showing code examples, design trade‑offs, and performance numbers that enable thousands of tasks per second.

CronHigh ThroughputRedis
0 likes · 8 min read
Stop Using Cron: From Single-Node to Distributed Scheduling to Reach >1k Tasks/s
Code Mala Tang
Code Mala Tang
Feb 17, 2026 · Backend Development

How to Build a High‑Throughput HTTP Server with Node.js

This article explains what high‑throughput means for an HTTP server, why Node.js’s event‑driven, non‑blocking architecture makes it ideal, and provides step‑by‑step code examples—including a basic server, blocking vs. async I/O, clustering, keep‑alive, JSON optimization, and streaming—to help developers design scalable, low‑latency services.

HTTPHigh ThroughputNode.js
0 likes · 8 min read
How to Build a High‑Throughput HTTP Server with Node.js
Alibaba Cloud Observability
Alibaba Cloud Observability
Jan 26, 2026 · Cloud Native

How LoongCollector Delivers 10× Throughput and 80% Resource Savings in Cloud‑Native Observability

LoongCollector, the open‑source cloud‑native collector behind Alibaba Cloud's Simple Log Service, achieves ten‑fold higher throughput, up to 80% lower CPU and memory usage, near‑linear scaling, zero‑copy processing, lock‑free event pools and adaptive concurrency, while guaranteeing enterprise‑grade reliability for petabyte‑scale log and metric ingestion.

High ThroughputLock-Freeadaptive-concurrency
0 likes · 16 min read
How LoongCollector Delivers 10× Throughput and 80% Resource Savings in Cloud‑Native Observability
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Dec 3, 2025 · Big Data

Kafka High‑Throughput Tricks: Sequential Writes, Zero‑Copy, Partitioning

The article explains how Kafka achieves high throughput by writing messages sequentially to disk, leveraging OS page cache and zero‑copy system calls, using partitioned topics for parallelism, batching and compressing records on both producer and broker sides, and employing asynchronous replication with configurable persistence strategies.

High ThroughputKafkaPartitioning
0 likes · 4 min read
Kafka High‑Throughput Tricks: Sequential Writes, Zero‑Copy, Partitioning
Su San Talks Tech
Su San Talks Tech
Nov 9, 2025 · Backend Development

Designing a Scalable Real‑Time Video Ranking System for Billions of Views

This article walks through the complete system‑design process for a Douyin‑style video leaderboard, covering requirement analysis, functional and non‑functional specs, scale estimation, single‑machine baseline, fault‑tolerance, sharding, time‑window handling with a double‑pointer method, and caching strategies to achieve low latency, high throughput, and precise results at massive scale.

Distributed ArchitectureHigh ThroughputSliding Window
0 likes · 26 min read
Designing a Scalable Real‑Time Video Ranking System for Billions of Views
Architect Chen
Architect Chen
Sep 10, 2025 · Big Data

How Kafka Achieves Million‑Message Throughput: Sequential Writes, Page Cache, Batching & Zero‑Copy

The article explains how Kafka attains high‑throughput performance by using sequential disk writes, leveraging the OS page cache, employing producer and consumer batching with configurable parameters, and utilizing zero‑copy sendfile to minimize CPU and memory overhead, enabling stable million‑message per second rates.

Big DataHigh ThroughputKafka
0 likes · 5 min read
How Kafka Achieves Million‑Message Throughput: Sequential Writes, Page Cache, Batching & Zero‑Copy
Tencent Cloud Developer
Tencent Cloud Developer
Aug 13, 2025 · Backend Development

How to Build a Streaming Reconciliation Engine for Billions of Transactions per Hour

This article explains the fundamentals of financial reconciliation, outlines the two‑step reconciliation process, and details a streaming reconciliation architecture capable of handling 30 billion USD per hour (≈27 777 TPS) with high concurrency, distributed parsing, and efficient result aggregation.

FinancialHigh ThroughputReconciliation
0 likes · 8 min read
How to Build a Streaming Reconciliation Engine for Billions of Transactions per Hour
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Jul 24, 2025 · Big Data

Boost Kafka Consumer Throughput: Multi‑Threading, Consumer Groups & Config Tuning

This guide explains why Kafka consumer throughput matters, then details practical techniques—including multi‑threaded consumption, scaling with consumer groups, client parameter tuning, and batch processing—to dramatically increase throughput while maintaining reliability in high‑concurrency, large‑scale data pipelines.

High ThroughputKafkaconsumer
0 likes · 5 min read
Boost Kafka Consumer Throughput: Multi‑Threading, Consumer Groups & Config Tuning
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Jul 7, 2025 · Big Data

How Kafka Achieves Billion-Message Throughput: Sequential Disk Writes, Page Cache, and Zero‑Copy

This article explains how Kafka sustains massive traffic by writing logs sequentially to disk, leveraging the operating system’s page cache for fast in‑memory writes, employing zero‑copy techniques like sendfile to avoid user‑space copying, and batching messages to reduce network overhead, thereby delivering high‑throughput, low‑latency streaming.

High ThroughputKafkabatching
0 likes · 4 min read
How Kafka Achieves Billion-Message Throughput: Sequential Disk Writes, Page Cache, and Zero‑Copy
Code Mala Tang
Code Mala Tang
Jul 3, 2025 · Databases

How We Achieved 50,000 TPS with PostgreSQL & Node.js: 6 Proven Optimizations

By systematically applying connection pooling, batch processing, WAL tuning, table partitioning, query refactoring, and OS-level tweaks, we transformed a PostgreSQL‑Node.js stack to sustain 50,000 transactions per second with sub‑100 ms latency, while maintaining data consistency on modest hardware.

High ThroughputNode.jsPostgreSQL
0 likes · 10 min read
How We Achieved 50,000 TPS with PostgreSQL & Node.js: 6 Proven Optimizations
Instant Consumer Technology Team
Instant Consumer Technology Team
Jun 30, 2025 · Operations

How 3FS Revolutionizes AI Storage with High‑Throughput Distributed Filesystem

3FS, DeepSeek’s high‑performance parallel file system, is engineered for AI workloads, offering ultra‑low latency, high‑throughput storage via RDMA, CRAQ consistency, and seamless cloud‑native integration, with detailed architecture, deployment steps, performance benchmarks, and cost‑saving strategies for large‑scale model training and inference.

AI storageDistributed File SystemHigh Throughput
0 likes · 28 min read
How 3FS Revolutionizes AI Storage with High‑Throughput Distributed Filesystem
Senior Tony
Senior Tony
Jun 8, 2025 · Backend Development

Why Kafka Handles Millions of Messages Per Second: Batch, Partition, Zero‑Copy, and Compression Explained

This article breaks down the core techniques that give Kafka its high‑throughput capability, including producer batch settings (batch.size, linger.ms), broker append‑only writes, consumer poll configuration, partition distribution, zero‑copy data transfer, dual‑thread processing, and configurable compression algorithms.

Batch ProcessingHigh ThroughputPartitioning
0 likes · 8 min read
Why Kafka Handles Millions of Messages Per Second: Batch, Partition, Zero‑Copy, and Compression Explained
Raymond Ops
Raymond Ops
Jun 1, 2025 · Backend Development

RocketMQ vs Others: Key Features, Reliability, and Performance

RocketMQ, an Apache‑licensed distributed message queue from Alibaba, offers low latency, high reliability, and high throughput through features like distributed architecture, synchronous disk flushing, master‑slave replication, and flexible messaging models, and it compares favorably against Kafka, RabbitMQ, and ActiveMQ in various scenarios.

Distributed ArchitectureHigh ThroughputHigh reliability
0 likes · 11 min read
RocketMQ vs Others: Key Features, Reliability, and Performance
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
May 22, 2025 · Big Data

Optimizing Kafka Producer for High Throughput: Batching, Asynchronous Sending, Compression, and Concurrency

This article details practical techniques for boosting Kafka producer throughput, covering batch sending optimization, linger.ms tuning, asynchronous send handling, compression options, and concurrent sending strategies, while providing code examples and performance considerations for high‑throughput streaming applications.

AsynchronousHigh ThroughputKafka
0 likes · 4 min read
Optimizing Kafka Producer for High Throughput: Batching, Asynchronous Sending, Compression, and Concurrency
Ma Wei Says
Ma Wei Says
Mar 28, 2025 · Backend Development

Choosing the Right Message Queue: Kafka vs RocketMQ vs RabbitMQ Explained

This article compares Kafka, RocketMQ, and RabbitMQ, detailing their architectures, performance characteristics, strengths, and ideal use‑cases to help engineers select the most suitable message‑queue solution for high‑throughput, fault‑tolerant, and real‑time processing scenarios.

Event StreamingHigh ThroughputKafka
0 likes · 11 min read
Choosing the Right Message Queue: Kafka vs RocketMQ vs RabbitMQ Explained
Huolala Tech
Huolala Tech
Nov 8, 2024 · Backend Development

How Huolala Built a Scalable Real‑Time Reconciliation Platform for Millions of Daily Transactions

Huolala’s real‑time reconciliation platform tackles massive daily transaction volumes by addressing distributed system consistency, high‑throughput data ingestion, dynamic cluster scaling, and security safeguards, enabling sub‑second settlement verification across hundreds of services.

Backend ArchitectureData ConsistencyDynamic Scaling
0 likes · 10 min read
How Huolala Built a Scalable Real‑Time Reconciliation Platform for Millions of Daily Transactions
Tongcheng Travel Technology Center
Tongcheng Travel Technology Center
Apr 17, 2024 · Backend Development

In-Depth Analysis of Apache RocketMQ Architecture, Operation Principles, and High‑Throughput Mechanisms

This article provides a comprehensive overview of Apache RocketMQ, detailing its core components, producer and consumer workflows, storage strategies, master‑slave synchronization, Raft‑based half‑write and leader election mechanisms, and best‑practice recommendations for high‑throughput, fault‑tolerant messaging systems.

Fault ToleranceHigh ThroughputMessage Queue
0 likes · 22 min read
In-Depth Analysis of Apache RocketMQ Architecture, Operation Principles, and High‑Throughput Mechanisms
Bilibili Tech
Bilibili Tech
Apr 12, 2024 · Backend Development

Design and Optimization of a High‑Throughput Long‑Connection Service for Live Streaming

The article details a Golang‑based high‑throughput long‑connection service for live‑streaming, describing its five‑layer architecture, multi‑protocol support, load‑balancing, message‑queue decoupling, aggregation with brotli compression, multi‑region deployment, priority channels, and future enhancements for observability and intelligent endpoint selection.

Backend ArchitectureHigh ThroughputStreaming
0 likes · 16 min read
Design and Optimization of a High‑Throughput Long‑Connection Service for Live Streaming
Programmer DD
Programmer DD
Jan 5, 2024 · Operations

Master the ‘Three Highs’: Availability, Throughput, and Scalability in System Design

This article explains the essential "three high" goals of system design—high availability, high throughput, and high scalability—detailing their meanings, common architectural patterns such as Hot‑Hot, Hot‑Warm, leader‑based clusters, and practical techniques like caching, async processing, and micro‑service isolation to build robust, scalable services.

Backend ArchitectureHigh AvailabilityHigh Throughput
0 likes · 6 min read
Master the ‘Three Highs’: Availability, Throughput, and Scalability in System Design
Java High-Performance Architecture
Java High-Performance Architecture
Aug 11, 2023 · Operations

Unlock 10M+ HTTP Connections per Second: dperf’s DPDK‑Powered Performance

dperf, an open‑source Baidu tool built on DPDK, delivers ultra‑high network performance—supporting millions of HTTP connections per second, hundreds of Gbps throughput, and detailed statistics—making it ideal for load‑balancer, cloud VM, NIC, and CPU network testing, with step‑by‑step deployment instructions.

DPDKHTTP benchmarkingHigh Throughput
0 likes · 7 min read
Unlock 10M+ HTTP Connections per Second: dperf’s DPDK‑Powered Performance
JD Tech
JD Tech
Jun 15, 2023 · Big Data

Event Bus: Architecture, Technical Challenges, and Solutions for High‑Throughput Data Standardization

This article introduces the event bus as a data pipeline for risk insight, explains its source‑transform‑sink architecture, outlines key technical challenges such as data heterogeneity and high‑throughput parsing, and presents solutions including standardized data models, plugin extensibility, low‑code hot‑loading, dynamic grouping, one‑click degradation, and traffic monitoring.

Data PipelineHigh ThroughputPlugin architecture
0 likes · 14 min read
Event Bus: Architecture, Technical Challenges, and Solutions for High‑Throughput Data Standardization
JD Cloud Developers
JD Cloud Developers
Apr 20, 2023 · Operations

How to Build a Cost‑Effective, High‑Throughput Log Collection System with ClickHouse

This article examines the challenges of scaling log storage and retrieval for high‑traffic services, analyzes the cost and performance limits of traditional ELK‑based pipelines, and presents a streamlined, UDP‑driven architecture using ClickHouse that dramatically reduces hardware expenses while handling hundreds of gigabytes per second.

ClickHouseHigh ThroughputUDP
0 likes · 16 min read
How to Build a Cost‑Effective, High‑Throughput Log Collection System with ClickHouse
ITPUB
ITPUB
Oct 4, 2022 · Big Data

How Kafka Achieves Million‑TPS with Sequential I/O, MMAP, and Zero‑Copy

This article explains how Kafka attains million‑level transactions per second by leveraging sequential disk writes, memory‑mapped files, zero‑copy data transfer, and batch processing, detailing each technique's mechanics and performance impact.

Big DataHigh ThroughputSequential I/O
0 likes · 10 min read
How Kafka Achieves Million‑TPS with Sequential I/O, MMAP, and Zero‑Copy
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Aug 18, 2022 · Big Data

How Hologres Achieves Ultra‑High‑Throughput Real‑Time Writes in Big Data Warehouses

This article explores the challenges of real‑time data ingestion at massive scale, explains Hologres' innovative upsert model, performance optimizations, and compares its capabilities with other data‑warehouse solutions, demonstrating how it delivers low‑latency, high‑throughput writes for big‑data analytics.

High ThroughputHologresReal-time Data Warehouse
0 likes · 24 min read
How Hologres Achieves Ultra‑High‑Throughput Real‑Time Writes in Big Data Warehouses
58 Tech
58 Tech
Aug 11, 2022 · Backend Development

WLock: High‑Reliability, High‑Throughput Distributed Lock Service Based on WPaxos

WLock is an open‑source distributed lock service built on the WPaxos consensus algorithm and RocksDB storage, offering multiple lock types, flexible acquisition modes, high concurrency optimizations, and strong reliability and throughput for coordinating shared resources in distributed systems.

High ThroughputRocksDBWPaxos
0 likes · 12 min read
WLock: High‑Reliability, High‑Throughput Distributed Lock Service Based on WPaxos
Laravel Tech Community
Laravel Tech Community
Apr 21, 2022 · Fundamentals

What Is Wi‑Fi 7? Features, Speeds, Frequencies, and Release Timeline

Wi‑Fi 7 (802.11be) is a forthcoming wireless standard that promises peak speeds over 40 Gbps—up to four times faster than Wi‑Fi 6—by using wider 320 MHz channels, 4096‑QAM, multi‑link operation, and advanced OFDMA and MU‑MIMO technologies across 2.4 GHz, 5 GHz, and 6 GHz bands, with devices expected to appear by late 2023 or early 2024.

802.11beHigh ThroughputMLO
0 likes · 6 min read
What Is Wi‑Fi 7? Features, Speeds, Frequencies, and Release Timeline
JD Tech
JD Tech
Apr 8, 2022 · Big Data

Designing a High‑Throughput Log Collection System with ClickHouse, UDP and Compression

The article analyses the massive cost and performance challenges of traditional log‑storage pipelines at JD.com, then proposes a streamlined architecture that eliminates disk and MQ stages, uses in‑memory buffering, UDP transport, Snappy/ZSTD compression, and ClickHouse storage to achieve multi‑gigabyte per‑second throughput with over 70% hardware cost reduction.

ClickHouseHigh Throughputdistributed systems
0 likes · 15 min read
Designing a High‑Throughput Log Collection System with ClickHouse, UDP and Compression
dbaplus Community
dbaplus Community
Dec 14, 2021 · Backend Development

How to Build a Cost‑Effective High‑Throughput Log Collection System with ClickHouse and UDP

This article analyzes the challenges of massive log storage and retrieval, calculates the bandwidth and hardware costs of traditional pipelines, and presents a streamlined architecture that uses in‑memory buffering, UDP transport, compression, and ClickHouse to achieve petabyte‑scale throughput while cutting storage costs by over 75%.

High ThroughputUDPbackend
0 likes · 16 min read
How to Build a Cost‑Effective High‑Throughput Log Collection System with ClickHouse and UDP
JD Retail Technology
JD Retail Technology
Nov 22, 2021 · Backend Development

Designing a High‑Performance Log Collection System with UDP, Compression, and ClickHouse

The article analyzes the high cost and scalability challenges of traditional log collection pipelines and proposes a streamlined architecture that uses in‑memory buffering, UDP transport, aggressive compression, and ClickHouse storage to achieve massive throughput while drastically reducing hardware and operational expenses.

ClickHouseHigh ThroughputMonitoring
0 likes · 15 min read
Designing a High‑Performance Log Collection System with UDP, Compression, and ClickHouse
Programmer DD
Programmer DD
Aug 30, 2021 · Big Data

Why Is Kafka So Fast? Unveiling the Secrets Behind Its High Throughput

This article explains how Kafka achieves remarkable speed and massive throughput by using sequential disk I/O, OS page cache, zero‑copy transfers, partitioned log segments with indexes, batch processing, and efficient compression, making it a cornerstone of modern big‑data pipelines.

Big DataHigh ThroughputKafka
0 likes · 9 min read
Why Is Kafka So Fast? Unveiling the Secrets Behind Its High Throughput
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Aug 11, 2021 · Backend Development

Building a High‑Throughput, Highly Available Message Center with RocketMQ, ES, and Kubernetes

This article outlines the technical and business goals, design architecture, technology selection, and operational considerations for creating a message center capable of handling 10,000 messages per second upstream and 1,000 downstream, while ensuring 100% high availability and easy integration with third‑party push services.

Backend ArchitectureDevOpsHigh Throughput
0 likes · 5 min read
Building a High‑Throughput, Highly Available Message Center with RocketMQ, ES, and Kubernetes
macrozheng
macrozheng
Dec 15, 2020 · Big Data

How Kafka Achieves Million‑TPS Through Sequential I/O, MMAP, and Zero‑Copy

Kafka can sustain millions of transactions per second by writing data sequentially to disk, leveraging memory‑mapped files, employing zero‑copy DMA transfers, and batching messages, each technique reducing I/O overhead and CPU involvement, which together enable its high‑throughput performance in big‑data pipelines.

Big DataHigh ThroughputKafka
0 likes · 11 min read
How Kafka Achieves Million‑TPS Through Sequential I/O, MMAP, and Zero‑Copy
ITPUB
ITPUB
Jul 23, 2020 · Databases

How to Achieve 570,000 MySQL Writes per Second with TokuDB

This article details a real‑world test that loads 200 million rows into a MySQL TokuDB table in under six minutes, achieving roughly 570 k writes per second, and compares the results with InnoDB while providing configuration tips and bulk‑loader considerations.

High ThroughputMySQLTokuDB
0 likes · 6 min read
How to Achieve 570,000 MySQL Writes per Second with TokuDB
Tencent Cloud Developer
Tencent Cloud Developer
Apr 27, 2020 · Cloud Computing

High‑Throughput Cloud File Storage (CFS) Design for Kuaishou's Spring Festival Red‑Envelope Campaign

During Kuaishou's 2020 Spring Festival red‑envelope campaign, Tencent Cloud's Cloud File Storage (CFS) provided a high‑throughput, read‑heavy NFS solution that handled up to 5 GB/s across 1 700 pods, delivering 100% availability, low latency, and massive interaction volumes for billions of users.

CFSCloud StorageDistributed File System
0 likes · 22 min read
High‑Throughput Cloud File Storage (CFS) Design for Kuaishou's Spring Festival Red‑Envelope Campaign
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Mar 7, 2020 · Big Data

How Kafka’s Reactor Thread Model Powers High‑Throughput Messaging

Kafka’s high‑throughput network architecture—built on NIO, a Reactor thread model, and a TCP‑based protocol—evolves from a simple synchronous processor design to a decoupled handler‑pool system, offering valuable lessons for designing scalable backend communication layers in big‑data applications.

High ThroughputKafkaNIO
0 likes · 7 min read
How Kafka’s Reactor Thread Model Powers High‑Throughput Messaging
Big Data Technology Architecture
Big Data Technology Architecture
Aug 3, 2019 · Big Data

Kafka Architecture: Design Principles for High Throughput and Reliability

This article explains Kafka's design background, persistence mechanisms, disk sequential I/O optimizations, network and compression strategies, and stability features such as partitioning, replication, and ISR, illustrating how these techniques enable high‑throughput, low‑latency real‑time log processing in big‑data environments.

Disk I/OHigh ThroughputKafka
0 likes · 9 min read
Kafka Architecture: Design Principles for High Throughput and Reliability
Architecture Digest
Architecture Digest
Aug 2, 2019 · Backend Development

Design and Implementation of a High‑Throughput WeChat Red‑Envelope System Simulating 10 Billion Requests

This article describes how a single‑machine prototype was built in Go to simulate a WeChat red‑envelope service capable of handling up to 6 × 10⁴ QPS, supporting one million concurrent connections, and processing 10 billion shake‑red‑envelope requests through careful load‑balancing, monitoring, and performance analysis.

High Throughputdistributed-systemsgolang
0 likes · 17 min read
Design and Implementation of a High‑Throughput WeChat Red‑Envelope System Simulating 10 Billion Requests
Big Data Technology Architecture
Big Data Technology Architecture
Jul 9, 2019 · Big Data

Kafka Best Practices for High Throughput: 20 Recommendations

This article presents New Relic's 20 best‑practice recommendations for operating Apache Kafka at high throughput, covering partitions, consumers, producers, and brokers, and explains key concepts, configuration tuning, monitoring, and architectural considerations to ensure reliable, scalable streaming pipelines.

BrokersConsumersHigh Throughput
0 likes · 14 min read
Kafka Best Practices for High Throughput: 20 Recommendations
ITPUB
ITPUB
Jul 1, 2019 · Big Data

Why Kafka Handles Millions of Writes per Second: I/O Secrets Revealed

This article explains how Kafka achieves ultra‑high throughput by using sequential disk writes, memory‑mapped files, zero‑copy sendfile reads, and batch compression, while also describing its data retention policies and the trade‑offs of synchronous versus asynchronous producer modes.

Batch CompressionHigh ThroughputMemory Mapped Files
0 likes · 9 min read
Why Kafka Handles Millions of Writes per Second: I/O Secrets Revealed
Programmer DD
Programmer DD
May 5, 2019 · Operations

20 Proven Kafka Best Practices to Scale High‑Throughput Streams

This article presents New Relic’s 20 best‑practice recommendations for Apache Kafka, covering partitions, consumers, producers, and brokers, to help engineers design, configure, and monitor high‑throughput, reliable streaming pipelines at scale.

Apache KafkaHigh Throughput
0 likes · 13 min read
20 Proven Kafka Best Practices to Scale High‑Throughput Streams
dbaplus Community
dbaplus Community
Nov 20, 2018 · Backend Development

20 Proven Kafka Best Practices for High‑Throughput Clusters

This guide presents New Relic’s 20 practical best‑practice recommendations—covering partitions, consumers, producers, and brokers—to help engineers design, tune, and monitor Apache Kafka deployments for reliable, high‑throughput performance.

BrokersConsumersHigh Throughput
0 likes · 15 min read
20 Proven Kafka Best Practices for High‑Throughput Clusters
Alibaba Cloud Infrastructure
Alibaba Cloud Infrastructure
Aug 29, 2018 · Artificial Intelligence

Alibaba's FPGA-Based Ultra‑Low Latency, High‑Throughput Machine Learning Processor

Alibaba unveiled an FPGA‑designed machine‑learning accelerator that achieves sub‑millisecond inference latency and thousands of frames‑per‑second throughput, demonstrating how integrated hardware‑software optimizations can deliver real‑time AI performance surpassing conventional GPU and ASIC solutions.

AI acceleratorFPGAHigh Throughput
0 likes · 5 min read
Alibaba's FPGA-Based Ultra‑Low Latency, High‑Throughput Machine Learning Processor
Architecture Digest
Architecture Digest
Apr 11, 2017 · Backend Development

Design and Practice of a High‑Throughput Spring Festival Red‑Packet System Supporting One Million Connections

This article describes how to design, implement, and evaluate a backend system that simulates the Spring Festival red‑packet service, achieving up to 6 × 10⁴ QPS on a single server while handling one million concurrent connections, and discusses the hardware, software, architecture, and performance results.

High ThroughputPerformance TestingQPS
0 likes · 18 min read
Design and Practice of a High‑Throughput Spring Festival Red‑Packet System Supporting One Million Connections
dbaplus Community
dbaplus Community
Jun 14, 2016 · Backend Development

How LeTV Scaled Its Order System to One Million Transactions per Second

The talk explains how LeTV’s BOSS platform used sharding, a custom distributed ORM called Mango, and Snowflake‑style ID generation together with cluster‑level request routing to expand its order processing from hundreds of thousands to over a million transactions per second.

High Throughputbackenddistributed-orm
0 likes · 5 min read
How LeTV Scaled Its Order System to One Million Transactions per Second
21CTO
21CTO
Nov 25, 2015 · Backend Development

Why Alipay Chose MetaQ: Inside Its High‑Performance Distributed Messaging System

Metamorphosis (MetaQ), Alipay’s open‑source Java‑based distributed messaging middleware, offers high throughput, ordered storage, transaction support, and HA replication, positioning itself as a Kafka‑like solution tailored for massive log, broadcast, and sequential data flows, now powering billions of daily messages for Alipay and Taobao.

High ThroughputKafka AlternativeMetaQ
0 likes · 6 min read
Why Alipay Chose MetaQ: Inside Its High‑Performance Distributed Messaging System