BestHub
Sign in
Random Bulletin
Author

Random Bulletin

17-year internet software developer specializing in AI applications, networking, architecture, and open source. Led the delivery of network services handling hundreds of millions of concurrent devices and tens of millions of QPS, and has three years of experience designing and building an agent platform. Follow to stay updated.

115
Articles
0
Likes
311
Views
0
Comments
Recent Articles

Latest from Random Bulletin

100 recent articles max
Random Bulletin
Random Bulletin
Jun 27, 2026 · Operations

Cross‑Data‑Center Replication at Ten‑Million QPS: From Async to Semi‑Sync and How to Choose

The article examines why cross‑datacenter replication must evolve from simple asynchronous copying to semi‑synchronous and layered strategies at the ten‑million‑QPS scale, detailing latency, RPO, bandwidth costs, failover complexities, and practical selection guidelines for each business tier.

Message Queueasynchronous replicationbandwidth optimization
0 likes · 17 min read
Cross‑Data‑Center Replication at Ten‑Million QPS: From Async to Semi‑Sync and How to Choose
Random Bulletin
Random Bulletin
Jun 26, 2026 · Operations

Message Duplication Is Inevitable: Building a Multi‑Layer Idempotency Middleware for Ten‑Million QPS

Message queues guarantee at‑least‑once delivery, making duplicate messages a normal feature; the article examines a real coupon‑distribution incident, critiques business‑level idempotency approaches, and outlines a layered platform‑wide middleware design—including unique keys, state machines, storage choices, and TTL strategies—to achieve reliable processing at ten‑million QPS scale.

IdempotencyMessage QueueMiddleware
0 likes · 19 min read
Message Duplication Is Inevitable: Building a Multi‑Layer Idempotency Middleware for Ten‑Million QPS
Random Bulletin
Random Bulletin
Jun 25, 2026 · Operations

Scaling Message Queues to 10M QPS: From Downtime to Seamless Online Expansion

At the 10‑million‑QPS scale, expanding a message‑queue cluster no longer hinges on simply adding brokers; it requires coordinated online upgrades of metadata, data migration with dynamic throttling, cooperative consumer rebalance, shadow‑traffic warm‑up, and rollback snapshots, making the act of adding machines the hardest part.

10M QPSMessage QueueRebalance
0 likes · 27 min read
Scaling Message Queues to 10M QPS: From Downtime to Seamless Online Expansion
Random Bulletin
Random Bulletin
Jun 24, 2026 · Operations

Rack Awareness: From Zero to High‑QPS – Boost Availability, Cut Cross‑AZ Traffic

A real‑world rack‑power outage showed that three‑replica Kafka clusters can lose all replicas when brokers share a failure domain, prompting a deep dive into rack awareness—how fault‑domain tags are injected, replica‑placement and leader‑distribution algorithms, consumer‑proximity reads, bandwidth costs, failure scenarios, and the stepwise evolution from hundred‑thousand to ten‑million QPS deployments.

KafkaPulsarRack Awareness
0 likes · 26 min read
Rack Awareness: From Zero to High‑QPS – Boost Availability, Cut Cross‑AZ Traffic
Random Bulletin
Random Bulletin
Jun 23, 2026 · Backend Development

Choosing the Right Replication Factor for 10M QPS Message Queues: From Dual to Multi‑Replica

The article walks through why a single replica is insufficient for million‑scale message queues, explains the latency and data‑loss trade‑offs of dual‑replica sync and async modes, shows how three‑replica majority voting becomes the sweet spot, and then details the engineering considerations for scaling to five or more replicas across availability zones and regions.

AZ awarenessHigh AvailabilityKafka
0 likes · 18 min read
Choosing the Right Replication Factor for 10M QPS Message Queues: From Dual to Multi‑Replica
Random Bulletin
Random Bulletin
Jun 20, 2026 · Backend Development

Message Ordering at 10M+ QPS: From Global to Partition to Business-Key

The article explains that message ordering is a spectrum—from strict global ordering to partition-level and finally business-key ordering—and examines the trade‑offs, pitfalls, and engineering techniques needed to maintain order while scaling to tens of millions of QPS.

Kafkabusiness keymessage ordering
0 likes · 24 min read
Message Ordering at 10M+ QPS: From Global to Partition to Business-Key
Random Bulletin
Random Bulletin
Jun 18, 2026 · Operations

Zero Message Loss at 10M QPS: From Best‑Effort to Proven Guarantees

The article dissects why message loss is an end‑to‑end engineering challenge across production, broker, and consumer stages, presents real‑world failure cases at ten‑million QPS, and outlines concrete strategies—ack loops, outbox tables, broker replication settings, consumer handling rules, observability, reconciliation, and SLA‑driven compensation—to evolve from best‑effort to provable, recoverable reliability.

Message QueueOutbox PatternSLA
0 likes · 17 min read
Zero Message Loss at 10M QPS: From Best‑Effort to Proven Guarantees
Random Bulletin
Random Bulletin
Jun 17, 2026 · Backend Development

Rebalance Optimization for Million‑QPS Kafka Clusters: From Default to Custom Assignors

At a 5,000‑instance, 20,000‑partition Kafka consumer cluster, the default eager Rebalance caused a 90‑second stop‑the‑world pause, illustrating how a single GC‑induced heartbeat miss can flood brokers with tens of millions of messages; the article dissects eager vs cooperative protocols, built‑in assignor limits, and four custom assignor patterns to scale Rebalance from 100 K to tens of millions of QPS.

AssignorCooperative ProtocolCustom Scheduler
0 likes · 28 min read
Rebalance Optimization for Million‑QPS Kafka Clusters: From Default to Custom Assignors
Random Bulletin
Random Bulletin
Jun 16, 2026 · Backend Development

Consumer Rate Limiting: From Zero to Full Control in Million‑QPS Architectures

The article explains why consumer‑side rate limiting is essential in million‑QPS systems, detailing how unchecked consumers can overwhelm downstream services, and presents practical strategies—including pause/resume, token‑bucket algorithms, adaptive thresholds, and global coordination—to safely throttle consumption without dropping messages.

KafkaMessage Queueconsumer optimization
0 likes · 16 min read
Consumer Rate Limiting: From Zero to Full Control in Million‑QPS Architectures