BestHub
Sign in
Tagged articles
59 articles
Page 1 of 1
JD Tech Talk
JD Tech Talk
Aug 5, 2024 · Artificial Intelligence

An Introduction to STORM: An LLM‑Powered Knowledge Management System for Automated Research and Writing

This article introduces STORM, a Stanford‑developed large‑language‑model‑based knowledge‑management platform that automates topic research, outline generation, citation‑rich article writing, and iterative refinement through perspective‑guided questioning and simulated conversations, dramatically improving technical investigation efficiency.

AI toolsLLMStorm
0 likes · 7 min read
An Introduction to STORM: An LLM‑Powered Knowledge Management System for Automated Research and Writing
21CTO
21CTO
Jan 1, 2022 · Fundamentals

Why ‘Suffering‑Oriented Programming’ Can Reduce Risk in Large‑Scale Projects

The article introduces “Suffering‑Oriented Programming,” a risk‑averse development approach that prioritizes only truly painful problems, follows the three maxims “first make it possible, then make it beautiful, finally make it fast,” and illustrates its application in building Storm’s distributed stream‑processing system.

Distributed SystemsStormprogramming methodology
0 likes · 10 min read
Why ‘Suffering‑Oriented Programming’ Can Reduce Risk in Large‑Scale Projects
Meituan Technology Team
Meituan Technology Team
Aug 26, 2021 · Big Data

How Meituan Built a Scalable Real‑Time Data Warehouse: Architecture & Lessons

Meituan Waimai’s data intelligence team outlines a universal real‑time data‑warehouse methodology that combines a production platform with an interactive analytics engine, detailing scenarios, technology choices, architectural designs, platformization, SLA management, and a practical Lambda‑style case study.

FlinkKappa architectureLambda architecture
0 likes · 18 min read
How Meituan Built a Scalable Real‑Time Data Warehouse: Architecture & Lessons
Tencent Cloud Developer
Tencent Cloud Developer
Sep 9, 2020 · Big Data

Tencent Game Marketing Deduplication Service: Technical Evolution from TDW to ClickHouse

Tencent’s game marketing analysis system “EAS” evolved from inefficient TDW HiveSQL jobs and file‑heavy real‑time pipelines to a scalable ClickHouse‑based deduplication service that processes hundreds of thousands of daily activity counts in sub‑second time, offering fast, reliable, and maintainable participant deduplication for massive marketing campaigns.

LevelDBMPPOLAP
0 likes · 10 min read
Tencent Game Marketing Deduplication Service: Technical Evolution from TDW to ClickHouse
Java Architect Essentials
Java Architect Essentials
Aug 21, 2020 · Big Data

Design and Integration of Flume, Kafka, Storm, Drools, and Redis for Real‑Time ETL Log Analysis

This article presents a modular architecture for real‑time ETL log analysis that combines Flume for log collection, Kafka as a buffering layer, Storm for stream processing, Drools for rule‑based data transformation, and Redis for fast storage, detailing installation, configuration, and code integration steps.

Big DataDroolsFlume
0 likes · 23 min read
Design and Integration of Flume, Kafka, Storm, Drools, and Redis for Real‑Time ETL Log Analysis
Xueersi Online School Tech Team
Xueersi Online School Tech Team
Sep 6, 2019 · Big Data

Real-Time Data Architecture, Evolution, and Applications at an Online School

The article details the six‑layer big‑data architecture of an online school, chronicles its migration from Storm to Spark Streaming and finally to Flink, and showcases concrete real‑time applications such as gateway monitoring, user‑profile tagging, renewal reporting, and advertising analysis, while outlining future development directions.

AnalyticsBig Data ArchitectureFlink
0 likes · 14 min read
Real-Time Data Architecture, Evolution, and Applications at an Online School
360 Zhihui Cloud Developer
360 Zhihui Cloud Developer
Jun 4, 2019 · Big Data

Why Flink Outperforms Storm: Deep Dive into Stream Processing Performance

Based on data transmission and reliability metrics, this article compares Apache Storm and Apache Flink in stream processing, presenting benchmark designs, test environments, results for synthetic and Kafka data, and offers practical recommendations such as operator chaining, object reuse, and checkpoint strategies to maximize Flink performance.

Big DataFlinkPerformance Testing
0 likes · 13 min read
Why Flink Outperforms Storm: Deep Dive into Stream Processing Performance
360 Tech Engineering
360 Tech Engineering
Jun 3, 2019 · Big Data

Performance Comparison of Apache Storm and Apache Flink from Data Transmission and Reliability Perspectives

This article presents a detailed performance benchmark comparing Apache Storm and Apache Flink in stream processing, focusing on data transmission methods, reliability mechanisms, operator chaining, and both self‑generated and Kafka‑sourced workloads, and provides practical optimization recommendations based on the results.

Big DataData TransmissionFlink
0 likes · 10 min read
Performance Comparison of Apache Storm and Apache Flink from Data Transmission and Reliability Perspectives
Alibaba Cloud Developer
Alibaba Cloud Developer
Aug 13, 2018 · Big Data

How Ele.me Evolved Its Real‑Time Engine: From Storm to Flink

This article examines Ele.me’s big‑data platform evolution, comparing Storm, Spark Streaming, Structured Streaming, and Flink, detailing their architectures, consistency semantics, performance trade‑offs, and why Flink became the preferred real‑time computation engine for the company.

Big DataFlinkSpark
0 likes · 15 min read
How Ele.me Evolved Its Real‑Time Engine: From Storm to Flink
Ctrip Technology
Ctrip Technology
Jul 17, 2018 · Big Data

Meteor: A Real-Time Computation Platform Based on Storm for Ctrip Marketing

The article introduces Meteor, a Storm‑based real‑time computation platform developed by Ctrip Marketing to simplify topology management, automate deployment, and improve resource efficiency for complex marketing scenarios, highlighting its architecture, features, and measurable business impact.

Real‑Time ComputingStormmarketing platform
0 likes · 10 min read
Meteor: A Real-Time Computation Platform Based on Storm for Ctrip Marketing
Meituan Technology Team
Meituan Technology Team
Jul 5, 2018 · Big Data

Meituan Dianping User Action System (UAS): Architecture and Implementation for Real-time User Behavior Processing

Meituan‑Dianping’s User Action System unifies disparate user‑behavior events with a 5W1H format, ingests them via a proprietary MAPI channel into Kafka, processes them in real‑time using Storm and a Lambda batch‑speed architecture, and delivers millisecond‑level responses for billions of daily events while offering flexible, modular query and storage options.

KafkaLambda architectureStorm
0 likes · 17 min read
Meituan Dianping User Action System (UAS): Architecture and Implementation for Real-time User Behavior Processing
ITPUB
ITPUB
Jun 19, 2018 · Big Data

Is Hadoop Still Relevant? Comparing Hadoop, PostgreSQL, and Storm

The article examines Hadoop's relevance by contrasting it with PostgreSQL and Storm, discussing when each technology fits big‑data challenges such as volume, velocity, and variety, and highlighting cost, complexity, and use‑case considerations for enterprises.

Batch ProcessingHadoopStorm
0 likes · 8 min read
Is Hadoop Still Relevant? Comparing Hadoop, PostgreSQL, and Storm
Efficient Ops
Efficient Ops
Jun 6, 2018 · Big Data

How Tencent’s Multi‑Dimensional Monitoring Turns Big Data Into Real‑Time Business Insights

This article explains how Tencent’s ZhiYun multi‑dimensional monitoring system evolves from the Mobile Monitor platform, outlines its design principles, data‑factory capabilities, storage choices, and intelligent features, and demonstrates how it enables real‑time, multi‑dimensional analysis and alerting for large‑scale business operations.

Big DataDruidStorm
0 likes · 11 min read
How Tencent’s Multi‑Dimensional Monitoring Turns Big Data Into Real‑Time Business Insights
Ctrip Technology
Ctrip Technology
Jun 4, 2018 · Big Data

Real-Time Data Processing Frameworks and Kafka Practices at Ctrip Ticketing

This article examines Ctrip Ticket's real-time data processing ecosystem, comparing batch and streaming frameworks such as Hadoop, Spark, Storm, Flink, and Spark Streaming, detailing Kafka deployment and configuration, and describing how these technologies are applied in production for log analysis, seat‑occupancy detection, and anti‑crawling.

FlinkReal-time ProcessingSpark Streaming
0 likes · 12 min read
Real-Time Data Processing Frameworks and Kafka Practices at Ctrip Ticketing
Ctrip Technology
Ctrip Technology
Mar 8, 2018 · Big Data

Ctrip Wireless APM Platform: Architecture, Metrics, and Technical Details

The article describes the evolution of Ctrip's wireless APM platform from the early UBT-based monitoring to a globally‑oriented, metric‑rich system that processes over 100 billion data points daily using Storm and Elasticsearch, detailing its design, key performance dimensions, data‑volume trade‑offs, and implementation choices.

APMBig DataCtrip
0 likes · 12 min read
Ctrip Wireless APM Platform: Architecture, Metrics, and Technical Details
Meituan Technology Team
Meituan Technology Team
Jan 26, 2018 · Big Data

Design and Implementation of a Real-Time Data Processing System at Meituan

Meituan designed a Storm‑based real‑time data processing platform that guarantees at‑least‑once delivery and high availability, employs a custom spout, regression‑driven traffic smoothing, and a low‑latency KV store with atomic operations, persisting results in Kafka, MySQL and Cellar to power merchant dashboards and heat‑tag analytics, while planning broader real‑time analytics expansion.

Big DataDistributed SystemsStorm
0 likes · 10 min read
Design and Implementation of a Real-Time Data Processing System at Meituan
Meituan Technology Team
Meituan Technology Team
Jan 12, 2018 · Backend Development

Design and Implementation of Meituan Hotel Full-Chain Log and Trace System

To cope with Meituan Hotel’s exploding micro‑service complexity, the infrastructure team built the Satellite System—combining MTrace and a selective, zero‑intrusion Log4j2‑based logging pipeline that streams enriched logs through Kafka, Storm, Redis and Elasticsearch, delivering second‑level trace‑log queries and six‑month retention, dramatically speeding up debugging.

Distributed TracingElasticsearchKafka
0 likes · 11 min read
Design and Implementation of Meituan Hotel Full-Chain Log and Trace System
Architecture Digest
Architecture Digest
Dec 16, 2017 · Big Data

Performance Comparison of Apache Flink and Apache Storm for Real‑Time Stream Processing

This report presents a systematic performance evaluation of Apache Flink and Apache Storm across multiple real‑time processing scenarios, measuring throughput, latency, message‑delivery semantics, and state‑backend effects, and provides recommendations for selecting the most suitable engine based on the observed results.

Big DataFlinkReal-time analytics
0 likes · 21 min read
Performance Comparison of Apache Flink and Apache Storm for Real‑Time Stream Processing
21CTO
21CTO
Nov 11, 2017 · Big Data

How We Built a Scalable Seller Log System with Kafka, Storm, ES & HBase

This article explains the design and implementation of a unified seller‑operation logging platform that uses Kafka for ingestion, Storm for real‑time processing, Elasticsearch for hot‑data search, and HBase for cold‑data storage, detailing the challenges faced and the optimizations applied.

Big DataElasticsearchHBase
0 likes · 12 min read
How We Built a Scalable Seller Log System with Kafka, Storm, ES & HBase
dbaplus Community
dbaplus Community
Oct 15, 2017 · Big Data

How JD Built a Scalable Seller Log Platform with Kafka, Storm, ES & HBase

This article details JD's end‑to‑end seller log system architecture, explaining why Kafka, Storm, Elasticsearch and HBase were chosen, the challenges faced during scaling, and the practical solutions implemented to achieve a unified, high‑throughput logging platform for merchants and operations.

Big DataElasticsearchHBase
0 likes · 13 min read
How JD Built a Scalable Seller Log Platform with Kafka, Storm, ES & HBase
21CTO
21CTO
Jul 8, 2017 · Big Data

Ctrip’s Scalable Real‑Time User Behavior System with Kafka, Storm, Redis

This article details Ctrip’s redesign of its real‑time user behavior service, covering the new architecture, data flow, use of Java, Kafka, Storm, Redis, and MySQL, and how it achieves high real‑time performance, availability, scalability, and fault‑tolerance to support massive travel‑industry traffic.

KafkaReal-TimeStorm
0 likes · 12 min read
Ctrip’s Scalable Real‑Time User Behavior System with Kafka, Storm, Redis
Suning Technology
Suning Technology
May 18, 2017 · Big Data

Why Apache Flink Beats Spark and Storm in Stream Processing

This article examines Apache Flink's stream‑processing architecture, compares its native streaming model, fault‑tolerance, performance and SQL capabilities with Spark and Storm, and concludes that Flink offers a more powerful and efficient solution despite some maturity gaps.

Apache FlinkSparkStorm
0 likes · 12 min read
Why Apache Flink Beats Spark and Storm in Stream Processing
Architecture Digest
Architecture Digest
May 18, 2017 · Backend Development

Design and Architecture of Ctrip's Real‑Time User Behavior Service

The article describes how Ctrip rebuilt its real‑time user behavior platform using a Java‑based stack (Kafka, Storm, Redis, MySQL) to achieve millisecond‑level latency, high availability, scalable performance, and robust handling of traffic spikes, failures, and data back‑pressure.

Backend ArchitectureKafkaReal-Time
0 likes · 12 min read
Design and Architecture of Ctrip's Real‑Time User Behavior Service
Tongcheng Travel Technology Center
Tongcheng Travel Technology Center
Apr 25, 2017 · Big Data

Real-Time Computation Platform Based on Storm and StreamCQL: Architecture, CQL Integration, and Development Guide

This article introduces a real‑time computation platform built on Apache Storm, explains its low‑latency, high‑throughput design, details the integration of Continuous Query Language (CQL) via StreamCQL, showcases development workflows, code examples, and two typical business use cases, and outlines future directions.

CQLStormStreamCQL
0 likes · 13 min read
Real-Time Computation Platform Based on Storm and StreamCQL: Architecture, CQL Integration, and Development Guide
Ctrip Technology
Ctrip Technology
Apr 13, 2017 · Big Data

Design and Implementation of Ctrip's Real-Time User Behavior System

The article describes how Ctrip redesigned its real-time user behavior service using a Java‑Kafka‑Storm stack with Redis and MySQL, detailing the architecture, real‑time processing, availability, performance, scalability, and deployment strategies to handle billions of events daily.

Real-time ProcessingStormSystem Architecture
0 likes · 13 min read
Design and Implementation of Ctrip's Real-Time User Behavior System
Tongcheng Travel Technology Center
Tongcheng Travel Technology Center
Apr 10, 2017 · Operations

Sentinel Monitoring System: Real‑Time Business Log Monitoring and Incident Detection for an Airline Ticket Platform

The Sentinel system was built to provide real‑time, zero‑modification monitoring of airline ticket business services by consuming Tianwang logs through a Storm cluster, offering flexible rule configuration, addressing performance pitfalls, and planning future enhancements such as custom monitoring scripts and visual dashboards.

KafkaLog ProcessingReal-Time
0 likes · 6 min read
Sentinel Monitoring System: Real‑Time Business Log Monitoring and Incident Detection for an Airline Ticket Platform
Architecture Digest
Architecture Digest
Sep 14, 2016 · Backend Development

Log Platform Architecture and Scaling Lessons from Vipshop’s 419 Flash Sale

The article analyzes Vipshop’s 419 flash‑sale log platform, detailing the 2013 architecture using Flume, RabbitMQ, Storm, Redis and MySQL, diagnosing bottlenecks in RabbitMQ and Storm during traffic spikes, and presenting practical scaling and monitoring solutions for high‑throughput backend systems.

Log ProcessingRabbitMQScalability
0 likes · 8 min read
Log Platform Architecture and Scaling Lessons from Vipshop’s 419 Flash Sale
Ctrip Technology
Ctrip Technology
Aug 12, 2016 · Big Data

Ctrip's Real-Time Data Platform: Architecture, Practices, and Lessons Learned

This article details Ctrip's journey building a unified real-time data platform—covering business motivations, architectural requirements, technology choices like Kafka and Storm, implementation of Avro schemas, monitoring, alerting, operational lessons, and future explorations such as Streaming CQL and JStorm.

AlertingBig DataKafka
0 likes · 15 min read
Ctrip's Real-Time Data Platform: Architecture, Practices, and Lessons Learned
Architect
Architect
Feb 29, 2016 · Big Data

Design Principles of Real-Time Distributed Streaming Systems: A Comparison of Spark and Storm

This article examines the design considerations of real-time distributed streaming systems, outlines their background and characteristics, compares the architectures of Spark Streaming and Storm, discusses primitives, message passing, high availability, storage models, and integration with production environments, providing practical insights for architects.

Distributed SystemsReal-time ProcessingSpark
0 likes · 20 min read
Design Principles of Real-Time Distributed Streaming Systems: A Comparison of Spark and Storm
Architect
Architect
Dec 30, 2015 · Big Data

Real-Time Big Data Processing with Storm and Kafka on Alibaba Cloud

This article explains how to build a large‑scale, real‑time vehicle monitoring system using Apache Storm and Kafka on Alibaba Cloud, covering the challenges of big‑data ingestion, system architecture, deployment steps, performance testing, and practical lessons learned.

Alibaba CloudBig DataKafka
0 likes · 12 min read
Real-Time Big Data Processing with Storm and Kafka on Alibaba Cloud
21CTO
21CTO
Dec 4, 2015 · Big Data

Building a Cost‑Effective Real‑Time Stream Processing Platform with Storm

This article details how the e‑commerce company 1号店 selected the Storm framework to create a low‑cost, highly available, and easily scalable distributed stream‑processing system, covering architecture design, resource isolation with CGroup, custom UI improvements, and operational lessons for handling massive traffic spikes.

Resource ManagementStormcgroup
0 likes · 9 min read
Building a Cost‑Effective Real‑Time Stream Processing Platform with Storm
21CTO
21CTO
Nov 4, 2015 · Big Data

How We Built a Real‑Time Log Analytics Platform with Storm and Cardinality Counting

To monitor hundreds of web apps on UAE’s PaaS platform in near‑real time, we combined Storm with lightweight log transport, a memcached‑based fqueue, and adaptive cardinality counting to efficiently compute PV, UV, response times, and custom metrics while handling cross‑cluster log aggregation.

Big DataCardinality countingLog Processing
0 likes · 9 min read
How We Built a Real‑Time Log Analytics Platform with Storm and Cardinality Counting
Art of Distributed System Architecture Design
Art of Distributed System Architecture Design
Sep 25, 2015 · Big Data

Understanding Storm: A Distributed Real-Time Computation System

The article explains the need for low‑latency, high‑performance, distributed real‑time processing, outlines the challenges such systems must address, and introduces Storm as a Hadoop‑like framework for stream processing, detailing its architecture, fault‑tolerance mechanisms, transactional topology, and large‑scale deployment at Taobao.

Big DataDistributed SystemsReal-time Processing
0 likes · 14 min read
Understanding Storm: A Distributed Real-Time Computation System
Art of Distributed System Architecture Design
Art of Distributed System Architecture Design
Jun 15, 2015 · Big Data

Designing a Scalable Real‑Time Mobile Analytics Platform with Kafka, Storm, and Amazon EMR

The article describes how a mobile analytics service processes billions of events daily using a Lambda‑style architecture that combines Kafka, Storm, Amazon EMR, and S3 to achieve scalable, fault‑tolerant batch and real‑time computation, while ensuring reliable event ingestion and graceful degradation.

AWSBig DataKafka
0 likes · 8 min read
Designing a Scalable Real‑Time Mobile Analytics Platform with Kafka, Storm, and Amazon EMR
Suning Technology
Suning Technology
May 22, 2015 · Big Data

Suning’s Big Data Platform Evolution: From SAP BW to Real‑Time Streaming

This article chronicles Suning’s journey from early SAP‑based data warehouses to a modern, open‑source big data platform featuring real‑time collection, Hadoop‑Hive offline processing, Storm‑based streaming, and a visual development environment, highlighting how each layer addresses growing data volume, variety, and business demands.

Data ArchitectureHadoopReal-time Processing
0 likes · 5 min read
Suning’s Big Data Platform Evolution: From SAP BW to Real‑Time Streaming