BestHub
Sign in
Tag

storm

0 views collected around this technical thread.

JD Tech Talk
JD Tech Talk
Aug 5, 2024 · Artificial Intelligence

An Introduction to STORM: An LLM‑Powered Knowledge Management System for Automated Research and Writing

This article introduces STORM, a Stanford‑developed large‑language‑model‑based knowledge‑management platform that automates topic research, outline generation, citation‑rich article writing, and iterative refinement through perspective‑guided questioning and simulated conversations, dramatically improving technical investigation efficiency.

AI toolsLLMarticle generation
0 likes · 7 min read
An Introduction to STORM: An LLM‑Powered Knowledge Management System for Automated Research and Writing
DataFunSummit
DataFunSummit
Aug 15, 2021 · Big Data

Building a General Real-Time Data Warehouse: Methods and Practices at Meituan Waimai

This article introduces a universal method for building a real-time data warehouse at Meituan Waimai, covering streaming technologies, architecture choices such as Lambda and Kappa, component design, feature production, SLA management, and practical OLAP solutions using Flink, Storm, and Doris.

DorisFlinkKappa architecture
0 likes · 15 min read
Building a General Real-Time Data Warehouse: Methods and Practices at Meituan Waimai
Efficient Ops
Efficient Ops
Dec 21, 2020 · Operations

How Dada Scaled Log Processing to 130 Billion Entries Daily with Kubernetes and Storm

This article details how Dada’s SRE team rebuilt its logging platform—replacing ELK with Filebeat and Storm, deploying Elasticsearch hot and cold nodes on Kubernetes, and optimizing log collection, parsing, and storage to handle over 130 billion daily log entries and 300 TB of data.

Big DataElasticsearchFilebeat
0 likes · 13 min read
How Dada Scaled Log Processing to 130 Billion Entries Daily with Kubernetes and Storm
DataFunTalk
DataFunTalk
Oct 12, 2020 · Big Data

Building a General Real‑Time Data Warehouse: Methods and Practices at Meituan Waimai

This article introduces Meituan Waimai's approach to constructing a universal real‑time data warehouse, covering streaming technology choices, Lambda/Kappa architectures, layered design, platformization, SLA management, and a practical Lambda‑style use case for real‑time analytics.

Big DataDoris OLAPFlink
0 likes · 16 min read
Building a General Real‑Time Data Warehouse: Methods and Practices at Meituan Waimai
Tencent Cloud Developer
Tencent Cloud Developer
Sep 9, 2020 · Big Data

Tencent Game Marketing Deduplication Service: Technical Evolution from TDW to ClickHouse

Tencent’s game marketing analysis system “EAS” evolved from inefficient TDW HiveSQL jobs and file‑heavy real‑time pipelines to a scalable ClickHouse‑based deduplication service that processes hundreds of thousands of daily activity counts in sub‑second time, offering fast, reliable, and maintainable participant deduplication for massive marketing campaigns.

Big DataClickHouseDeduplication
0 likes · 10 min read
Tencent Game Marketing Deduplication Service: Technical Evolution from TDW to ClickHouse
Java Architect Essentials
Java Architect Essentials
Aug 21, 2020 · Big Data

Design and Integration of Flume, Kafka, Storm, Drools, and Redis for Real‑Time ETL Log Analysis

This article presents a modular architecture for real‑time ETL log analysis that combines Flume for log collection, Kafka as a buffering layer, Storm for stream processing, Drools for rule‑based data transformation, and Redis for fast storage, detailing installation, configuration, and code integration steps.

Big DataKafkaReal-time Processing
0 likes · 23 min read
Design and Integration of Flume, Kafka, Storm, Drools, and Redis for Real‑Time ETL Log Analysis
Dada Group Technology
Dada Group Technology
Mar 5, 2020 · Operations

Building a High-Throughput Kubernetes-Based Log Processing System at Dada

The article describes how Dada rebuilt its log processing pipeline using Kubernetes mixed deployment, Filebeat for automated collection, Storm for efficient parsing, and Elasticsearch cold/hot nodes to handle over 130 billion daily log entries and 300TB storage.

ElasticsearchFilebeatSRE
0 likes · 9 min read
Building a High-Throughput Kubernetes-Based Log Processing System at Dada
Xueersi Online School Tech Team
Xueersi Online School Tech Team
Sep 6, 2019 · Big Data

Real-Time Data Architecture, Evolution, and Applications at an Online School

The article details the six‑layer big‑data architecture of an online school, chronicles its migration from Storm to Spark Streaming and finally to Flink, and showcases concrete real‑time applications such as gateway monitoring, user‑profile tagging, renewal reporting, and advertising analysis, while outlining future development directions.

FlinkReal-time StreamingSpark Streaming
0 likes · 14 min read
Real-Time Data Architecture, Evolution, and Applications at an Online School
360 Zhihui Cloud Developer
360 Zhihui Cloud Developer
Jun 4, 2019 · Big Data

Why Flink Outperforms Storm: Deep Dive into Stream Processing Performance

Based on data transmission and reliability metrics, this article compares Apache Storm and Apache Flink in stream processing, presenting benchmark designs, test environments, results for synthetic and Kafka data, and offers practical recommendations such as operator chaining, object reuse, and checkpoint strategies to maximize Flink performance.

Big DataFlinkoperator chaining
0 likes · 13 min read
Why Flink Outperforms Storm: Deep Dive into Stream Processing Performance
360 Tech Engineering
360 Tech Engineering
Jun 3, 2019 · Big Data

Performance Comparison of Apache Storm and Apache Flink from Data Transmission and Reliability Perspectives

This article presents a detailed performance benchmark comparing Apache Storm and Apache Flink in stream processing, focusing on data transmission methods, reliability mechanisms, operator chaining, and both self‑generated and Kafka‑sourced workloads, and provides practical optimization recommendations based on the results.

Big DataData TransmissionFlink
0 likes · 10 min read
Performance Comparison of Apache Storm and Apache Flink from Data Transmission and Reliability Perspectives
Ctrip Technology
Ctrip Technology
Jul 17, 2018 · Big Data

Meteor: A Real-Time Computation Platform Based on Storm for Ctrip Marketing

The article introduces Meteor, a Storm‑based real‑time computation platform developed by Ctrip Marketing to simplify topology management, automate deployment, and improve resource efficiency for complex marketing scenarios, highlighting its architecture, features, and measurable business impact.

Big DataResource Managementmarketing platform
0 likes · 10 min read
Meteor: A Real-Time Computation Platform Based on Storm for Ctrip Marketing
Efficient Ops
Efficient Ops
Jun 6, 2018 · Big Data

How Tencent’s Multi‑Dimensional Monitoring Turns Big Data Into Real‑Time Business Insights

This article explains how Tencent’s ZhiYun multi‑dimensional monitoring system evolves from the Mobile Monitor platform, outlines its design principles, data‑factory capabilities, storage choices, and intelligent features, and demonstrates how it enables real‑time, multi‑dimensional analysis and alerting for large‑scale business operations.

Big DataDruidReal-time Monitoring
0 likes · 11 min read
How Tencent’s Multi‑Dimensional Monitoring Turns Big Data Into Real‑Time Business Insights
Ctrip Technology
Ctrip Technology
Jun 4, 2018 · Big Data

Real-Time Data Processing Frameworks and Kafka Practices at Ctrip Ticketing

This article examines Ctrip Ticket's real-time data processing ecosystem, comparing batch and streaming frameworks such as Hadoop, Spark, Storm, Flink, and Spark Streaming, detailing Kafka deployment and configuration, and describing how these technologies are applied in production for log analysis, seat‑occupancy detection, and anti‑crawling.

Big DataFlinkKafka
0 likes · 12 min read
Real-Time Data Processing Frameworks and Kafka Practices at Ctrip Ticketing
Ctrip Technology
Ctrip Technology
Mar 8, 2018 · Big Data

Ctrip Wireless APM Platform: Architecture, Metrics, and Technical Details

The article describes the evolution of Ctrip's wireless APM platform from the early UBT-based monitoring to a globally‑oriented, metric‑rich system that processes over 100 billion data points daily using Storm and Elasticsearch, detailing its design, key performance dimensions, data‑volume trade‑offs, and implementation choices.

APMBig DataCtrip
0 likes · 12 min read
Ctrip Wireless APM Platform: Architecture, Metrics, and Technical Details
Architecture Digest
Architecture Digest
Dec 16, 2017 · Big Data

Performance Comparison of Apache Flink and Apache Storm for Real‑Time Stream Processing

This report presents a systematic performance evaluation of Apache Flink and Apache Storm across multiple real‑time processing scenarios, measuring throughput, latency, message‑delivery semantics, and state‑backend effects, and provides recommendations for selecting the most suitable engine based on the observed results.

Big DataFlinkPerformance Benchmark
0 likes · 21 min read
Performance Comparison of Apache Flink and Apache Storm for Real‑Time Stream Processing
Qunar Tech Salon
Qunar Tech Salon
Nov 30, 2017 · Big Data

Performance Comparison of Apache Flink and Apache Storm for Real-Time Stream Processing

This article presents a comprehensive performance evaluation of Apache Flink versus Apache Storm across multiple real‑time processing scenarios, measuring throughput, latency, and the impact of different configurations and delivery semantics to guide framework selection and optimization.

FlinkPerformance Benchmarkexactly-once
0 likes · 16 min read
Performance Comparison of Apache Flink and Apache Storm for Real-Time Stream Processing
Architecture Digest
Architecture Digest
Nov 11, 2017 · Big Data

Design and Implementation of a Seller Log System Using Kafka, Storm, Elasticsearch, and HBase

This article describes the design and implementation of a seller log system, detailing the use of Kafka for high‑throughput messaging, Storm for real‑time stream processing, Elasticsearch for hot‑data search, and HBase for cold‑data storage, along with challenges faced and optimization solutions.

Big DataElasticsearchHBase
0 likes · 12 min read
Design and Implementation of a Seller Log System Using Kafka, Storm, Elasticsearch, and HBase