Tagged articles
Storm
59 articles · Page 1 of 1
This article introduces STORM, a Stanford‑developed large‑language‑model‑based knowledge‑management platform that automates topic research, outline generation, citation‑rich article writing, and iterative refinement through perspective‑guided questioning and simulated conversations, dramatically improving technical investigation efficiency.
AI toolsArticle GenerationAutomated Research
0 likes · 7 min read

Facing a four‑hour coupon‑calculation bottleneck at eLong, I explored Storm’s stream‑processing model, identified the data‑extraction stage as the weak link, refactored the hotel‑pull service, leveraged RocketMQ for threading, and ultimately reduced full‑batch processing time to just over an hour.
Coupon SystemJavaRocketMQ
0 likes · 12 min read


21CTOJan 1, 2022 · Fundamentals
The article introduces “Suffering‑Oriented Programming,” a risk‑averse development approach that prioritizes only truly painful problems, follows the three maxims “first make it possible, then make it beautiful, finally make it fast,” and illustrates its application in building Storm’s distributed stream‑processing system.
Stormdistributed systemsprogramming methodology
0 likes · 10 min read

Meituan Waimai’s data intelligence team outlines a universal real‑time data‑warehouse methodology that combines a production platform with an interactive analytics engine, detailing scenarios, technology choices, architectural designs, platformization, SLA management, and a practical Lambda‑style case study.
DorisFlinkKappa architecture
0 likes · 18 min read

This article introduces a universal method for building a real-time data warehouse at Meituan Waimai, covering streaming technologies, architecture choices such as Lambda and Kappa, component design, feature production, SLA management, and practical OLAP solutions using Flink, Storm, and Doris.
DorisFlinkKappa architecture
0 likes · 15 min read

This article details how Dada’s SRE team rebuilt its logging platform—replacing ELK with Filebeat and Storm, deploying Elasticsearch hot and cold nodes on Kubernetes, and optimizing log collection, parsing, and storage to handle over 130 billion daily log entries and 300 TB of data.
Stormlog management
0 likes · 13 min read

This article introduces Meituan Waimai's approach to constructing a universal real‑time data warehouse, covering streaming technology choices, Lambda/Kappa architectures, layered design, platformization, SLA management, and a practical Lambda‑style use case for real‑time analytics.
Doris OLAPFlinkKappa architecture
0 likes · 16 min read

Tencent’s game marketing analysis system “EAS” evolved from inefficient TDW HiveSQL jobs and file‑heavy real‑time pipelines to a scalable ClickHouse‑based deduplication service that processes hundreds of thousands of daily activity counts in sub‑second time, offering fast, reliable, and maintainable participant deduplication for massive marketing campaigns.
ClickHouseLevelDBMPP
0 likes · 10 min read

This article presents a modular architecture for real‑time ETL log analysis that combines Flume for log collection, Kafka as a buffering layer, Storm for stream processing, Drools for rule‑based data transformation, and Redis for fast storage, detailing installation, configuration, and code integration steps.
DroolsFlumeKafka
0 likes · 23 min read

This article details how Dada built a Kubernetes‑mixed log platform that handles over 130 billion logs per day, stores more than 14 TB daily, and maintains a 300 TB total volume by automating collection with Filebeat, parsing with Storm, and optimizing Elasticsearch with hot‑cold nodes.
ElasticsearchKubernetesOperations
0 likes · 12 min read

This article investigates intermittent disk IOWait spikes on Storm machines, identifies the kjournald and pdflush processes as primary contributors, explains ext3 journaling modes, and offers practical mount‑option tweaks like adding noatime to improve disk performance.
Ext3Performance TuningStorm
0 likes · 5 min read

This article details the design, installation, and modular integration of Flume, Kafka, Storm, Drools, and Redis to build a real‑time log analysis pipeline for ETL systems, discussing architecture, configuration, code examples, and practical considerations for scalability and fault tolerance.
DroolsFlumeKafka
0 likes · 24 min read

The article describes how Dada rebuilt its log processing pipeline using Kubernetes mixed deployment, Filebeat for automated collection, Storm for efficient parsing, and Elasticsearch cold/hot nodes to handle over 130 billion daily log entries and 300TB storage.
ElasticsearchKubernetesLogging
0 likes · 9 min read

This article presents a modular architecture for real‑time ETL log collection and analysis, detailing the installation and configuration of Flume, Kafka, Storm, Drools, and Redis, and explains how their integration improves fault tolerance, scalability, and processing speed.
DroolsFlumeStorm
0 likes · 22 min read

The article details the six‑layer big‑data architecture of an online school, chronicles its migration from Storm to Spark Streaming and finally to Flink, and showcases concrete real‑time applications such as gateway monitoring, user‑profile tagging, renewal reporting, and advertising analysis, while outlining future development directions.
AnalyticsFlinkSpark Streaming
0 likes · 14 min read


21CTOAug 20, 2019 · Big Data
This article explains how Mogu’s advertising system designs and evolves a real‑time data pipeline—covering merchant and operation needs, data collection, cleaning, processing with Storm, Flink, and Kylin, and service guarantees—to enable high‑quality, low‑latency analytics for advertisers and the platform.
AdvertisingFlinkKylin
0 likes · 12 min read

This article describes how the 58 real‑time computing platform optimized the Flink‑Storm beta tool and implemented large‑scale, smooth migration of Storm jobs to Flink, covering background, architecture, platform‑level enhancements, YARN runtime support, deployment, and user‑side integration.
FlinkStormYARN
0 likes · 9 min read

Based on data transmission and reliability metrics, this article compares Apache Storm and Apache Flink in stream processing, presenting benchmark designs, test environments, results for synthetic and Kafka data, and offers practical recommendations such as operator chaining, object reuse, and checkpoint strategies to maximize Flink performance.
FlinkStormbig data
0 likes · 13 min read

This article presents a detailed performance benchmark comparing Apache Storm and Apache Flink in stream processing, focusing on data transmission methods, reliability mechanisms, operator chaining, and both self‑generated and Kafka‑sourced workloads, and provides practical optimization recommendations based on the results.
Data TransmissionFlinkStorm
0 likes · 10 min read

This article examines Ele.me’s big‑data platform evolution, comparing Storm, Spark Streaming, Structured Streaming, and Flink, detailing their architectures, consistency semantics, performance trade‑offs, and why Flink became the preferred real‑time computation engine for the company.
FlinkSparkStorm
0 likes · 15 min read

The article introduces Meteor, a Storm‑based real‑time computation platform developed by Ctrip Marketing to simplify topology management, automate deployment, and improve resource efficiency for complex marketing scenarios, highlighting its architecture, features, and measurable business impact.
Real-time ComputingStormmarketing platform
0 likes · 10 min read

Meituan‑Dianping’s User Action System unifies disparate user‑behavior events with a 5W1H format, ingests them via a proprietary MAPI channel into Kafka, processes them in real‑time using Storm and a Lambda batch‑speed architecture, and delivers millisecond‑level responses for billions of daily events while offering flexible, modular query and storage options.
KafkaLambda architectureStorm
0 likes · 17 min read


ITPUBJun 19, 2018 · Big Data
The article examines Hadoop's relevance by contrasting it with PostgreSQL and Storm, discussing when each technology fits big‑data challenges such as volume, velocity, and variety, and highlighting cost, complexity, and use‑case considerations for enterprises.
HadoopPostgreSQLStorm
0 likes · 8 min read

This article explains how Tencent’s ZhiYun multi‑dimensional monitoring system evolves from the Mobile Monitor platform, outlines its design principles, data‑factory capabilities, storage choices, and intelligent features, and demonstrates how it enables real‑time, multi‑dimensional analysis and alerting for large‑scale business operations.
DruidStormbig data
0 likes · 11 min read

This article examines Ctrip Ticket's real-time data processing ecosystem, comparing batch and streaming frameworks such as Hadoop, Spark, Storm, Flink, and Spark Streaming, detailing Kafka deployment and configuration, and describing how these technologies are applied in production for log analysis, seat‑occupancy detection, and anti‑crawling.
FlinkSpark StreamingStorm
0 likes · 12 min read

Ctrip’s real‑time user behavior service, a core foundation for recommendations, ads, and user profiling, was redesigned with a Java‑based stack (Kafka, Storm, Redis, MySQL) to achieve millisecond‑level latency, high availability, and ten‑fold scalability across processing and output flows.
KafkaMySQLStorm
0 likes · 12 min read

The article describes the evolution of Ctrip's wireless APM platform from the early UBT-based monitoring to a globally‑oriented, metric‑rich system that processes over 100 billion data points daily using Storm and Elasticsearch, detailing its design, key performance dimensions, data‑volume trade‑offs, and implementation choices.
APMCtripElasticsearch
0 likes · 12 min read

Meituan designed a Storm‑based real‑time data processing platform that guarantees at‑least‑once delivery and high availability, employs a custom spout, regression‑driven traffic smoothing, and a low‑latency KV store with atomic operations, persisting results in Kafka, MySQL and Cellar to power merchant dashboards and heat‑tag analytics, while planning broader real‑time analytics expansion.
Data EngineeringStormbig data
0 likes · 10 min read

To cope with Meituan Hotel’s exploding micro‑service complexity, the infrastructure team built the Satellite System—combining MTrace and a selective, zero‑intrusion Log4j2‑based logging pipeline that streams enriched logs through Kafka, Storm, Redis and Elasticsearch, delivering second‑level trace‑log queries and six‑month retention, dramatically speeding up debugging.
Distributed TracingElasticsearchKafka
0 likes · 11 min read

This report presents a systematic performance evaluation of Apache Flink and Apache Storm across multiple real‑time processing scenarios, measuring throughput, latency, message‑delivery semantics, and state‑backend effects, and provides recommendations for selecting the most suitable engine based on the observed results.
FlinkStormbig data
0 likes · 21 min read

This article presents a comprehensive performance evaluation of Apache Flink versus Apache Storm across multiple real‑time processing scenarios, measuring throughput, latency, and the impact of different configurations and delivery semantics to guide framework selection and optimization.
FlinkStormexactly-once
0 likes · 16 min read


21CTONov 11, 2017 · Big Data
This article explains the design and implementation of a unified seller‑operation logging platform that uses Kafka for ingestion, Storm for real‑time processing, Elasticsearch for hot‑data search, and HBase for cold‑data storage, detailing the challenges faced and the optimizations applied.
ElasticsearchHBaseKafka
0 likes · 12 min read

This article describes the design and implementation of a seller log system, detailing the use of Kafka for high‑throughput messaging, Storm for real‑time stream processing, Elasticsearch for hot‑data search, and HBase for cold‑data storage, along with challenges faced and optimization solutions.
KafkaStormlog system
0 likes · 12 min read

This article details JD's end‑to‑end seller log system architecture, explaining why Kafka, Storm, Elasticsearch and HBase were chosen, the challenges faced during scaling, and the practical solutions implemented to achieve a unified, high‑throughput logging platform for merchants and operations.
ElasticsearchHBaseKafka
0 likes · 13 min read

This guide explains Storm's architecture, the roles of spouts and bolts, the hierarchy of nodes, workers, executors, and tasks, and shows how to configure concurrency by adding workers, adjusting executor and task counts, and handling special cases for optimal stream processing performance.
ConcurrencyJavaStorm
0 likes · 9 min read


21CTOAug 23, 2017 · Big Data
This article explains the design of a real-time customer behavior collection platform that uses NSQ for messaging, Storm for streaming processing, and HBase for storage, covering architecture, data flow, reliability guarantees, and deployment considerations.
HBaseNSQStorm
0 likes · 11 min read


21CTOJul 8, 2017 · Big Data
This article details Ctrip’s redesign of its real‑time user behavior service, covering the new architecture, data flow, use of Java, Kafka, Storm, Redis, and MySQL, and how it achieves high real‑time performance, availability, scalability, and fault‑tolerance to support massive travel‑industry traffic.
KafkaMySQLReal-time
0 likes · 12 min read

This article examines Apache Flink's stream‑processing architecture, compares its native streaming model, fault‑tolerance, performance and SQL capabilities with Spark and Storm, and concludes that Flink offers a more powerful and efficient solution despite some maturity gaps.
Apache FlinkSparkStorm
0 likes · 12 min read

The article describes how Ctrip rebuilt its real‑time user behavior platform using a Java‑based stack (Kafka, Storm, Redis, MySQL) to achieve millisecond‑level latency, high availability, scalable performance, and robust handling of traffic spikes, failures, and data back‑pressure.
KafkaMySQLReal-time
0 likes · 12 min read

This article introduces a real‑time computation platform built on Apache Storm, explains its low‑latency, high‑throughput design, details the integration of Continuous Query Language (CQL) via StreamCQL, showcases development workflows, code examples, and two typical business use cases, and outlines future directions.
CQLStormStreamCQL
0 likes · 13 min read

The article describes how Ctrip redesigned its real-time user behavior service using a Java‑Kafka‑Storm stack with Redis and MySQL, detailing the architecture, real‑time processing, availability, performance, scalability, and deployment strategies to handle billions of events daily.
MySQLStormreal-time processing
0 likes · 13 min read

The Sentinel system was built to provide real‑time, zero‑modification monitoring of airline ticket business services by consuming Tianwang logs through a Storm cluster, offering flexible rule configuration, addressing performance pitfalls, and planning future enhancements such as custom monitoring scripts and visual dashboards.
KafkaReal-timeStorm
0 likes · 6 min read

This article presents a detailed case study of Vipshop's log platform during the 419 sales event, analyzing the 2013 architecture, bottlenecks in RabbitMQ and Storm, and the subsequent redesign using Kafka, Impala, and HBase to achieve scalable, reliable big‑data processing.
ImpalaKafkaStorm
0 likes · 16 min read

The article analyzes Vipshop’s 419 flash‑sale log platform, detailing the 2013 architecture using Flume, RabbitMQ, Storm, Redis and MySQL, diagnosing bottlenecks in RabbitMQ and Storm during traffic spikes, and presenting practical scaling and monitoring solutions for high‑throughput backend systems.
RabbitMQStormlog processing
0 likes · 8 min read

The article describes how Yihaodian built a low‑latency, highly available, and easily scalable streaming computation platform using Storm, Kafka, Linux containers and a custom CGroup management framework to handle massive e‑commerce traffic spikes and real‑time analytics.
ContainerizationKafkaStorm
0 likes · 9 min read

This article details Ctrip's journey building a unified real-time data platform—covering business motivations, architectural requirements, technology choices like Kafka and Storm, implementation of Avro schemas, monitoring, alerting, operational lessons, and future explorations such as Streaming CQL and JStorm.
KafkaStormalerting
0 likes · 15 min read

The article provides a comprehensive overview of Twitter's Heron stream processing system, comparing its architecture, design principles, back‑pressure mechanisms, resource utilization, and performance test results with Storm/JStorm, and concludes with practical insights for large‑scale deployments.
HeronStormarchitecture
0 likes · 23 min read

This article describes Ctrip's development of a unified real-time data platform, detailing its motivations, architectural choices such as Kafka and Storm, implementation of shared schemas, resource control, monitoring, and operational lessons, as well as experiences with Storm, JStorm, and Streaming CQL.
CtripKafkaStorm
0 likes · 15 min read

This article introduces a beginner-friendly architecture for real-time order analytics in a big‑data environment, detailing how Flume collects logs, Kafka buffers them, Storm processes streams, and Redis stores results, while also covering configuration, code snippets, deployment steps, and troubleshooting tips.
FlumeKafkaStorm
0 likes · 26 min read

This article examines the design considerations of real-time distributed streaming systems, outlines their background and characteristics, compares the architectures of Spark Streaming and Storm, discusses primitives, message passing, high availability, storage models, and integration with production environments, providing practical insights for architects.
High AvailabilitySparkStorm
0 likes · 20 min read

This article explains how to build a large‑scale, real‑time vehicle monitoring system using Apache Storm and Kafka on Alibaba Cloud, covering the challenges of big‑data ingestion, system architecture, deployment steps, performance testing, and practical lessons learned.
Alibaba CloudKafkaStorm
0 likes · 12 min read


21CTODec 4, 2015 · Big Data
This article details how the e‑commerce company 1号店 selected the Storm framework to create a low‑cost, highly available, and easily scalable distributed stream‑processing system, covering architecture design, resource isolation with CGroup, custom UI improvements, and operational lessons for handling massive traffic spikes.
Stormcgroupresource management
0 likes · 9 min read


21CTONov 4, 2015 · Big Data
To monitor hundreds of web apps on UAE’s PaaS platform in near‑real time, we combined Storm with lightweight log transport, a memcached‑based fqueue, and adaptive cardinality counting to efficiently compute PV, UV, response times, and custom metrics while handling cross‑cluster log aggregation.
Cardinality countingStormUAE platform
0 likes · 9 min read

The article explains the need for low‑latency, high‑performance, distributed real‑time processing, outlines the challenges such systems must address, and introduces Storm as a Hadoop‑like framework for stream processing, detailing its architecture, fault‑tolerance mechanisms, transactional topology, and large‑scale deployment at Taobao.
Stormbig datadistributed systems
0 likes · 14 min read

This article provides a concise overview of several open‑source real‑time stream processing platforms—including S4, Storm, StreamBase, HStreaming, Esper/NEsper, Kafka, Scribe, and Flume—highlighting their main features, programming languages, and project links for further reference.
KafkaReal-timeStorm
0 likes · 5 min read

The article explains how Storm complements batch‑oriented Hadoop by providing fault‑tolerant, low‑latency stream processing for massive, continuously generated data such as Twitter’s tweet firehose, and demonstrates its architecture, key features, and a simple MapReduce‑style code example.
ApacheClojureJava
0 likes · 14 min read

The article describes how a mobile analytics service processes billions of events daily using a Lambda‑style architecture that combines Kafka, Storm, Amazon EMR, and S3 to achieve scalable, fault‑tolerant batch and real‑time computation, while ensuring reliable event ingestion and graceful degradation.
AWSKafkaStorm
0 likes · 8 min read

This article chronicles Suning’s journey from early SAP‑based data warehouses to a modern, open‑source big data platform featuring real‑time collection, Hadoop‑Hive offline processing, Storm‑based streaming, and a visual development environment, highlighting how each layer addresses growing data volume, variety, and business demands.
HadoopStormdata architecture
0 likes · 5 min read

This article discusses various log collection and analysis architectures, comparing solutions such as Flume‑Kafka‑Storm pipelines, Sentry, MongoDB, ELK stack, and Hadoop, and shares practical experiences, advantages, drawbacks, and deployment tips from multiple engineers.
FlumeKafkaLog Collection
0 likes · 7 min read
