Author

Random Bulletin

17-year internet software developer specializing in AI applications, networking, architecture, and open source. Led the delivery of network services handling hundreds of millions of concurrent devices and tens of millions of QPS, and has three years of experience designing and building an agent platform. Follow to stay updated.

115

Articles

Likes

311

Views

Comments

Latest from Random Bulletin

100 recent articles max

Random Bulletin

Aug 26, 2026 · Operations

Log Standards at Million‑QPS Scale: From Free‑form to Strict Structured Logging

When a production outage forces a midnight investigation across five services, the lack of log standards turns a quick debug into an all‑night forensic hunt; the article explains how structured logs, a unified schema, level semantics, trace_id linking, and field‑level masking enforced by SDK, Lint and CI can eliminate these five walls and make logging scalable, searchable, and compliant.

CI enforcementhigh QPSlog masking

0 likes · 22 min read

Log Standards at Million‑QPS Scale: From Free‑form to Strict Structured Logging

Random Bulletin

Aug 24, 2026 · Operations

Scaling Log Volumes from GB to TB at Ten‑Million QPS: Cost‑Effective Strategies and Architecture

At ten‑million QPS, log data can explode from a few gigabytes to terabytes or even petabytes, triggering storage blow‑up, pipeline saturation, slow queries, runaway costs, and poor signal‑to‑noise, and the article breaks down ingest, index, and store costs while presenting edge sampling, label‑based indexing, tiered storage, and log‑to‑metric rollup as mitigation tactics.

ElasticsearchLokicost optimization

0 likes · 19 min read

Scaling Log Volumes from GB to TB at Ten‑Million QPS: Cost‑Effective Strategies and Architecture

Random Bulletin

Aug 22, 2026 · Operations

From Manual Trace Queries to Intelligent Link Analysis at Million‑QPS Scale

The article walks through how link tracing evolves from manually searching individual traces to an automated, intelligent system that aggregates massive spans, derives RED metrics and service maps, performs critical‑path and differential analysis, auto‑detects anomalies, and ties together traces, metrics, and logs for rapid root‑cause identification.

RED metricsTail SamplingTracing

0 likes · 19 min read

From Manual Trace Queries to Intelligent Link Analysis at Million‑QPS Scale

Random Bulletin

Aug 21, 2026 · Operations

Scaling Link Tracing Storage: From Centralized Elasticsearch to Tiered Architecture

The article analyzes why storing massive tracing spans in a single Elasticsearch cluster fails at high QPS, outlines the four key challenges of trace data, and presents a three‑step engineering solution—sampling, hot‑warm‑cold tiered storage, and separating indexes from span payloads—while comparing major back‑ends such as Elasticsearch, Cassandra, ClickHouse, and Tempo.

ClickHouseElasticsearchTempo

0 likes · 19 min read

Scaling Link Tracing Storage: From Centralized Elasticsearch to Tiered Architecture

Random Bulletin

Aug 20, 2026 · Cloud Native

TraceID Propagation: From Manual Hand‑coding to Automatic OpenTelemetry

The article explains why propagating TraceID and parent SpanID across services, processes, and asynchronous boundaries is essential for complete distributed tracing, outlines the pain points of manual header handling, and walks through the evolution toward standardized W3C Trace Context, automatic instrumentation, and OpenTelemetry‑based solutions while highlighting async and service‑mesh pitfalls.

W3C Trace Contextasynchronous contextdistributed-tracing

0 likes · 18 min read

TraceID Propagation: From Manual Hand‑coding to Automatic OpenTelemetry

Random Bulletin

Aug 19, 2026 · Operations

Sampling at Ten‑Million QPS: From Full to Intelligent Adaptive Sampling

The article explains how sampling serves as the cost‑fidelity knob in distributed tracing, why full sampling collapses at ten‑million QPS, compares head‑based, tail‑based, and intelligent adaptive strategies, and shows how tools like OpenTelemetry, Jaeger, and AWS X‑Ray implement these approaches.

Jaegeradaptive samplingdistributed-tracing

0 likes · 19 min read

Sampling at Ten‑Million QPS: From Full to Intelligent Adaptive Sampling

Random Bulletin

Aug 18, 2026 · Operations

From Single‑Node to Distributed Monitoring Storage: Scaling TSDB for Million‑QPS

The article explains how a single‑machine Prometheus TSDB initially works well but eventually hits five scalability walls—capacity, single‑point failure, short retention, lack of global view, and throughput limits—and then details the step‑by‑step evolution to remote_write with object‑storage‑backed Thanos and finally to native distributed TSDBs such as Cortex, Mimir, and VictoriaMetrics, including their trade‑offs, costs, and practical selection guidance.

CortexPrometheusTSDB

0 likes · 21 min read

From Single‑Node to Distributed Monitoring Storage: Scaling TSDB for Million‑QPS

Random Bulletin

Aug 17, 2026 · Operations

Push vs Pull for Million‑QPS Monitoring: Hybrid Edge Pull and Central Push

At massive scale, the choice between push‑based and pull‑based metric collection determines monitoring system survivability; the article examines StatsD’s push model, Prometheus’s pull model, their trade‑offs, and proposes a hybrid edge‑pull, central‑push architecture to balance flow control, health checks, and network topology.

Hybrid ArchitecturePrometheusStatsD

0 likes · 18 min read

Push vs Pull for Million‑QPS Monitoring: Hybrid Edge Pull and Central Push

Random Bulletin

Aug 16, 2026 · Operations

Real‑Time Alerting at Million‑QPS: From Static Thresholds to Intelligent Detection

At massive scales of millions of QPS and hundreds of metrics, static alert thresholds become noisy and hard to maintain; the article walks through a stepwise evolution—adding sustained‑duration and multi‑condition rules, adopting dynamic baselines, leveraging anomaly detection, and applying correlation, RCA and SLO burn‑rate techniques—to transform alerts from simple threshold breaches into precise, user‑experience‑focused notifications while combating alert fatigue.

AIOpsSLOalerting

0 likes · 18 min read

Real‑Time Alerting at Million‑QPS: From Static Thresholds to Intelligent Detection

Random Bulletin

Aug 14, 2026 · Operations

Why Metric Counts Explode from Thousands to Millions in High‑QPS Systems

A single high‑cardinality label can cause a monitoring system to crash as metric series jump from hundreds of thousands to millions, overwhelming storage, queries, collection, cost, and signal‑to‑noise; the article explains the root cause, impact dimensions, and practical mitigation strategies such as cardinality control, distributed TSDBs, downsampling, pre‑aggregation, and proper division of metrics, traces, and logs.

PrometheusTSDBhigh cardinality

0 likes · 17 min read

Why Metric Counts Explode from Thousands to Millions in High‑QPS Systems