BestHub
Sign in
Tagged articles
92 articles
Page 1 of 1
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Apr 27, 2026 · Information Security

Real-Time Agentic Risk Detection with Flink, Fluss, and Large Language Models

The article presents a Flink‑Fluss‑LLM architecture that captures full‑link agent events via a non‑intrusive hook, combines semantic AI inference with deterministic CEP rules, and delivers millisecond‑level alerts for malicious user detection, tool result poisoning, and chain‑attack risk mitigation.

AI FunctionAgent SecurityFlink
0 likes · 41 min read
Real-Time Agentic Risk Detection with Flink, Fluss, and Large Language Models
Machine Heart
Machine Heart
Mar 31, 2026 · Artificial Intelligence

How JD’s JoyStreamer Achieves Smooth Long‑Form, Free‑Form Digital Human Live Streams

The article details how JD’s JoyStreamer and JoyStreamer‑Flash models overcome text‑control weakness, multimodal conflict, and identity drift to enable long‑duration, free‑state, real‑time interactive digital‑human video generation, surpassing current SOTA models in benchmark scores and reaching 30 FPS inference speed for e‑commerce live streaming.

Digital HumanJoyStreamerReal-time Streaming
0 likes · 12 min read
How JD’s JoyStreamer Achieves Smooth Long‑Form, Free‑Form Digital Human Live Streams
Big Data Technology & Architecture
Big Data Technology & Architecture
Dec 10, 2025 · Big Data

What’s New in Apache Spark 4.0? Deep Dive into 2025 Core Updates

The 2025 release of Apache Spark 4.0 brings a comprehensive overhaul—including default ANSI SQL mode, full SQL scripting support, a new Real‑Time streaming mode, adaptive query execution, dynamic memory management, and GPU‑accelerated MLlib—significantly boosting performance, reliability, and developer productivity across big‑data workloads.

Apache SparkBig DataGPU Acceleration
0 likes · 9 min read
What’s New in Apache Spark 4.0? Deep Dive into 2025 Core Updates
Spring Full-Stack Practical Cases
Spring Full-Stack Practical Cases
Sep 5, 2025 · Backend Development

Master Real-Time Data Streaming in Spring Boot 3: StreamingResponseBody vs SSE

This article compares Spring Boot 3's StreamingResponseBody and SseEmitter approaches for server‑side streaming, providing complete backend and frontend implementations, code samples, and performance considerations to help developers choose the right solution for real‑time data delivery.

Backend DevelopmentJava backendReal-time Streaming
0 likes · 8 min read
Master Real-Time Data Streaming in Spring Boot 3: StreamingResponseBody vs SSE
php Courses
php Courses
Aug 29, 2025 · Operations

How to Build a Real‑Time PHP Log Event Pipeline for Instant Insights

Learn how to transform PHP logs into real‑time, structured events by implementing a log event pipeline that includes JSON logging, lightweight collectors like Filebeat, streaming platforms such as Kafka or Flink, enrichment, and visualization with Grafana, enabling instant monitoring, alerting, and data‑driven decisions.

FlinkGrafanaKafka
0 likes · 7 min read
How to Build a Real‑Time PHP Log Event Pipeline for Instant Insights
Huolala Tech
Huolala Tech
Aug 20, 2025 · Backend Development

How We Rebuilt Our Code Coverage Platform for Real‑Time, Multi‑Language Insight

This article details the redesign of a code coverage platform, introducing a Kafka‑centric real‑time data stream, language‑agnostic contracts, non‑intrusive agents for Java and PHP, fine‑grained multi‑dimensional analysis, and interactive visualizations that dramatically improve testing efficiency and coverage accuracy.

Agent ArchitectureMulti-languageReal-time Streaming
0 likes · 11 min read
How We Rebuilt Our Code Coverage Platform for Real‑Time, Multi‑Language Insight
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Aug 7, 2025 · Big Data

How Flink ML Transforms Intelligent Operations: Real‑Time Anomaly Detection, Forecasting & Log Clustering

This article explains how Alibaba Cloud’s big‑data platform leverages Flink ML to build an intelligent‑operations service that tackles stability, cost and efficiency challenges through time‑series anomaly detection, forecasting and streaming log‑clustering, dramatically reducing latency, complexity and operational overhead.

FlinkIntelligent OperationsLog Clustering
0 likes · 25 min read
How Flink ML Transforms Intelligent Operations: Real‑Time Anomaly Detection, Forecasting & Log Clustering
Architecture and Beyond
Architecture and Beyond
Jun 8, 2025 · Frontend Development

Why Server‑Sent Events (SSE) Power Real‑Time Typing Effects in AI Chat Apps

Server‑Sent Events (SSE) offer a lightweight, HTTP‑native solution for real‑time, one‑way streaming, making them ideal for AI chat applications that need a typewriter‑style response, with advantages over WebSockets such as easy implementation, automatic reconnection, broad compatibility, and simple debugging.

JavaScriptPythonReal-time Streaming
0 likes · 13 min read
Why Server‑Sent Events (SSE) Power Real‑Time Typing Effects in AI Chat Apps
Architect
Architect
Apr 28, 2025 · Backend Development

Master Real-Time Log Streaming in Spring Boot with ResponseBodyEmitter

Learn how to implement efficient real-time log streaming in Spring Boot using ResponseBodyEmitter, covering its purpose, core methods, usage scenarios, step-by-step controller code, connection lifecycle management, comparison with SSE and Streaming, and important considerations for client support, timeouts, and thread safety.

HTTP Chunked TransferJavaReal-time Streaming
0 likes · 11 min read
Master Real-Time Log Streaming in Spring Boot with ResponseBodyEmitter
Spring Full-Stack Practical Cases
Spring Full-Stack Practical Cases
Mar 18, 2025 · Backend Development

Master Streaming Responses in Spring Boot 3: Real‑World Cases & Code

This article introduces Spring Boot 3’s StreamingResponseBody, explains its benefits for high‑concurrency scenarios, and provides three hands‑on examples—large file download, real‑time data streaming, and dynamic CSV export—complete with full Java code and practical tips for efficient backend development.

Backend DevelopmentCSV exportFile Download
0 likes · 8 min read
Master Streaming Responses in Spring Boot 3: Real‑World Cases & Code
Baidu Tech Salon
Baidu Tech Salon
Mar 6, 2025 · Big Data

Real-Time Anti-Fraud Streaming System Based on Flink: Architecture, Challenges, and Optimizations

The article describes a Flink‑based real‑time anti‑fraud streaming system that combines a risk‑control platform, configurable YAML‑driven pipelines, and optimized state handling—using early event‑time triggers, micro‑batch caching, and coarse‑grained key reduction—to compute multi‑dimensional features, support rapid strategy updates, simulation filtering, and seamless output to ClickHouse, Hive, and Redis for both instant monitoring and offline analysis.

ConfigurationFlinkReal-time Streaming
0 likes · 26 min read
Real-Time Anti-Fraud Streaming System Based on Flink: Architecture, Challenges, and Optimizations
Baidu Geek Talk
Baidu Geek Talk
Mar 3, 2025 · Big Data

Real-Time Anti-Cheat Streaming System Based on Flink: Architecture, Challenges, and Solutions

The article details a Flink‑based real‑time anti‑cheat streaming architecture that combines tumbling, sliding and session windows with early triggers, batch state updates cached in memory, coarse‑grained key reduction, and YAML‑driven strategy configuration to deliver millisecond‑level detection, seamless integration with ClickHouse, Hive, Redis and message queues, and self‑service analytics, achieving high throughput, low latency, and robust stability for large‑scale risk control.

Configuration ManagementFlinkPerformance Optimization
0 likes · 25 min read
Real-Time Anti-Cheat Streaming System Based on Flink: Architecture, Challenges, and Solutions
Java Architecture Diary
Java Architecture Diary
Mar 3, 2025 · Frontend Development

Boost Real-Time AI Streams in the Browser with fetch-event-source

This article explains how Server‑Sent Events (SSE) work, outlines the limitations of the native EventSource API, and demonstrates how the fetch‑event‑source library enhances SSE with POST support, custom headers, retry strategies, and visibility handling, enabling efficient real‑time AI data streams in modern web front‑ends.

AIJavaScriptReal-time Streaming
0 likes · 6 min read
Boost Real-Time AI Streams in the Browser with fetch-event-source
Open Source Tech Hub
Open Source Tech Hub
Oct 31, 2024 · Backend Development

How to Build Real-Time SSE and HTTP Chunk Streams in Webman (PHP)

This guide explains the concept of stream output, its advantages and typical scenarios, then provides step‑by‑step implementations of Server‑Sent Events and HTTP chunk streaming using the Webman framework, including PHP process code, configuration, startup commands, and front‑end testing.

HTTP ChunkPHPReal-time Streaming
0 likes · 9 min read
How to Build Real-Time SSE and HTTP Chunk Streams in Webman (PHP)
JD Retail Technology
JD Retail Technology
Oct 31, 2024 · Big Data

JDQ Kafka Bandwidth Throttling Architecture and Optimization

This article presents an in‑depth analysis of Kafka's native throttling mechanisms, identifies their limitations in large‑scale e‑commerce scenarios, and introduces JDQ's multi‑dimensional, dynamic throttling architecture that ensures stable throughput and priority‑aware bandwidth management across broker failures and traffic spikes.

Distributed SystemsJDQKafka
0 likes · 17 min read
JDQ Kafka Bandwidth Throttling Architecture and Optimization
Top Architect
Top Architect
Aug 10, 2024 · Big Data

Design and Implementation of a Scalable Real-Time Log Monitoring Platform at Baidu

This article introduces Baidu's log platform that handles billions of daily events, explains UBC logging concepts and monitoring requirements, and details a low‑cost, high‑accuracy architecture using real‑time streaming, dimension mapping, watermarking, and time‑window aggregation to achieve reliable, scalable event monitoring.

Big DataLog MonitoringReal-time Streaming
0 likes · 14 min read
Design and Implementation of a Scalable Real-Time Log Monitoring Platform at Baidu
Architect
Architect
Aug 7, 2024 · Operations

How Baidu Scales Real‑Time Log Monitoring for Billions of Events

This article explains Baidu's log‑center architecture for handling billions of UBC events per day, detailing UBC concepts, monitoring requirements, a low‑cost scalable design with dimension mapping, watermarking, data trimming and time‑window aggregation, and the resulting performance and cost benefits.

Cost OptimizationLog MonitoringReal-time Streaming
0 likes · 13 min read
How Baidu Scales Real‑Time Log Monitoring for Billions of Events
iQIYI Technical Product Team
iQIYI Technical Product Team
Jul 5, 2024 · Big Data

RiskFactor: An Integrated Real‑Time and Offline Feature Platform for Risk Control

RiskFactor unifies iQIYI’s legacy real‑time and offline feature platforms onto Opal’s DAG‑plus‑SQL engine, accelerating feature production fifteen‑fold, cutting latency from hours to minutes, streamlining development, lowering costs, and delivering more reliable, versioned risk‑control capabilities against sophisticated online threats.

Big DataDAGReal-time Streaming
0 likes · 14 min read
RiskFactor: An Integrated Real‑Time and Offline Feature Platform for Risk Control
php Courses
php Courses
May 9, 2024 · Backend Development

Real-Time Camera Monitoring with PHP

This tutorial explains how to set up a real‑time camera monitoring system using PHP, covering hardware connection, required streaming server software, a complete PHP script to fetch and stream video, and instructions for running and security considerations for local environments.

CameraPHPReal-time Streaming
0 likes · 4 min read
Real-Time Camera Monitoring with PHP
Baidu Geek Talk
Baidu Geek Talk
Apr 8, 2024 · Big Data

How RTS Platform Turns Real‑Time Data Streams into Reliable Business Value

This article analyzes the challenges of commercial real‑time data processing—such as stability, multi‑stage computation, and frequent schema changes—and explains how the RTS platform provides end‑to‑end managed solutions, auto schema handling, primary‑secondary redundancy, experiment‑first deployment, and metadata generation to unlock high‑velocity data value for advertising operations.

Big DataRTS platformReal-time Streaming
0 likes · 17 min read
How RTS Platform Turns Real‑Time Data Streams into Reliable Business Value
Architects Research Society
Architects Research Society
Jul 16, 2023 · Big Data

Four Innovation Phases of Netflix’s Trillion‑Scale Real‑Time Data Infrastructure

The article chronicles Netflix’s evolution from a failing batch pipeline to a cloud‑native, self‑service streaming platform, detailing four development phases, the technical challenges faced, the stream‑processing patterns introduced, key learnings, and future opportunities for real‑time data and machine‑learning workloads.

Data PlatformFlinkKafka
0 likes · 30 min read
Four Innovation Phases of Netflix’s Trillion‑Scale Real‑Time Data Infrastructure
MoonWebTeam
MoonWebTeam
Jun 8, 2023 · Game Development

How WebRTC Powers Cloud Gaming: A Deep Dive into Real-Time Game Streaming

Explore the fundamentals of cloud gaming and discover how WebRTC's real-time audio‑video, NAT traversal, and data channel technologies enable low‑latency game streaming across devices, with detailed architecture, protocols, and code examples for developers.

Game DevelopmentReal-time StreamingWebRTC
0 likes · 36 min read
How WebRTC Powers Cloud Gaming: A Deep Dive into Real-Time Game Streaming
Bilibili Tech
Bilibili Tech
Jun 2, 2023 · Artificial Intelligence

AI‑Driven Video Quality Enhancement and Low‑Bitrate High‑Resolution Techniques at Bilibili

Bilibili’s Cloud Multimedia team uses AI‑driven pipelines to cut bandwidth costs while delivering low‑bitrate, high‑quality video, employing a QoE‑based decision engine, real‑time 4K super‑resolution for game streams, low‑rank reconstruction for narrow‑band HD, data‑driven HDR LUTs, and explores diffusion‑based restoration for legacy content.

AIBilibiliReal-time Streaming
0 likes · 27 min read
AI‑Driven Video Quality Enhancement and Low‑Bitrate High‑Resolution Techniques at Bilibili
DataFunTalk
DataFunTalk
May 4, 2023 · Big Data

Tencent Content Ecosystem Real‑Time Signal System: Architecture, Challenges, and Optimization

This article explains how Tencent builds a trillion‑scale real‑time signal system for its content ecosystem, covering signal applications, data source and processing challenges, a layered architecture with Flink‑based streaming, dynamic topic detection, high‑throughput ID mapping, large‑window calculations, rule‑engine triggering, and future roadmap for scalability and cost reduction.

FlinkReal-time StreamingTencent
0 likes · 17 min read
Tencent Content Ecosystem Real‑Time Signal System: Architecture, Challenges, and Optimization
dbaplus Community
dbaplus Community
Apr 11, 2023 · Big Data

How Autohome Built a Flink‑StarRocks Real‑Time Ad Data Warehouse

This article details Autohome's transition from an hourly offline ad data warehouse to a Flink‑StarRocks real‑time architecture, covering background, engine and storage selection, multi‑layer design, implementation steps, encountered issues, monitoring strategies, and future roadmap to achieve second‑level data freshness and high accuracy.

AdvertisingFlinkReal-time Streaming
0 likes · 12 min read
How Autohome Built a Flink‑StarRocks Real‑Time Ad Data Warehouse
ShiZhen AI
ShiZhen AI
Mar 1, 2023 · Cloud Native

Why We Chose Kafka for Our Open‑Source Real‑Time Streaming Platform

The article explains how market trends, data‑driven enterprise needs, and internal platform experience led Didi to build Know Streaming—a zero‑intrusion, plugin‑based real‑time streaming solution built on Kafka—to address scalability, operability, and community adoption challenges.

Cloud NativeData PlatformKafka
0 likes · 12 min read
Why We Chose Kafka for Our Open‑Source Real‑Time Streaming Platform
ITPUB
ITPUB
Dec 21, 2022 · Big Data

How Bilibili Optimized Flink Runtime for Massive Real‑Time Jobs

This article details Bilibili's extensive enhancements to the Flink runtime—including checkpoint recoverability, max‑parallelism calculations, State Processor API extensions, Full and Regional Checkpoints, hybrid HA, task‑level recovery, load‑balanced partitioners, and large‑scale cluster maintenance—to improve reliability and performance of its billion‑scale streaming workloads.

Big DataCheckpointFlink
0 likes · 33 min read
How Bilibili Optimized Flink Runtime for Massive Real‑Time Jobs
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Nov 30, 2022 · Big Data

What’s New in Apache Flink 2022? Highlights from the Flink Forward Asia Summit

The 2022 Flink Forward Asia summit showcased Apache Flink’s rapid community growth, key technical breakthroughs such as distributed snapshot upgrades, cloud‑native state storage, hybrid shuffle, Flink CDC 2.0, and Flink ML 2.0, and real‑world deployments at companies like Midea, miHoYo and Disney.

Apache FlinkBig DataFlink Forward Asia
0 likes · 25 min read
What’s New in Apache Flink 2022? Highlights from the Flink Forward Asia Summit
DataFunTalk
DataFunTalk
Nov 29, 2022 · Big Data

Summary of Flink Forward Asia 2022: Keynotes, Technical Innovations, and Industry Deployments of Apache Flink

The 2022 Flink Forward Asia conference highlighted Apache Flink’s rapid growth, showcased major technical advances such as upgraded checkpointing, cloud‑native state storage, Hybrid Shuffle, Flink CDC 2.0, and Flink ML 2.0, and presented real‑world deployments from Alibaba, Midea, miHoYo, and Disney.

Apache FlinkData IntegrationReal-time Streaming
0 likes · 25 min read
Summary of Flink Forward Asia 2022: Keynotes, Technical Innovations, and Industry Deployments of Apache Flink
DaTaobao Tech
DaTaobao Tech
Nov 23, 2022 · Big Data

Real-time Log Aggregation and Monitoring with Blink (Flink) on Mobile Endpoints

The article explains how Blink, Alibaba’s optimized Flink variant, uses dynamic tables and streaming‑SQL to ingest mobile telemetry via source tables, compute per‑minute metrics such as API success rates with tumbling windows, and write results to Alibaba Cloud Log Service, enabling real‑time dashboards and extensible use cases like fraud detection.

FlinkReal-time StreamingSQL
0 likes · 10 min read
Real-time Log Aggregation and Monitoring with Blink (Flink) on Mobile Endpoints
Bilibili Tech
Bilibili Tech
Nov 11, 2022 · Backend Development

Real-Time Audio/Video System Architecture and Key Technologies Based on WebRTC

The article surveys the evolution of live streaming toward low‑latency, interactive scenarios and details WebRTC‑based real‑time audio/video system design, covering RTP/UDP transport, FEC and ARQ loss recovery, congestion control, jitter buffering, echo cancellation, edge‑node path optimization, and a multi‑layer architecture with signaling, routing, mixing services for scalable, high‑availability PK deployments.

Low latencyMedia ServerReal-time Streaming
0 likes · 16 min read
Real-Time Audio/Video System Architecture and Key Technologies Based on WebRTC
Bilibili Tech
Bilibili Tech
Sep 6, 2022 · Big Data

Lancer: Evolution of Bilibili's Real-Time Streaming Architecture

Lancer, Bilibili’s real‑time streaming backbone, has evolved from a monolithic Flume pipeline to a log‑id‑isolated, Kubernetes‑native architecture where Go edge agents feed synchronous Kafka‑proxied gateways into per‑logid topics processed by dedicated Flink‑SQL jobs, delivering exactly‑once, back‑pressured, highly scalable data ingestion for billions of daily requests.

Big DataFlinkKafka
0 likes · 29 min read
Lancer: Evolution of Bilibili's Real-Time Streaming Architecture
Alibaba Cloud Developer
Alibaba Cloud Developer
Jun 28, 2022 · Big Data

How Kuaishou Guarantees Real‑Time Data Warehouse Reliability During Billion‑Scale Events

This article details Kuaishou’s real‑time data warehouse architecture and its comprehensive assurance framework—including forward lifecycle standards, reverse fault‑injection testing, and Spring Festival event practices—highlighting challenges of massive traffic, high timeliness, accuracy, and stability, and outlining future plans for automation, batch‑stream integration, and cost reduction.

Data WarehouseFlinkReal-time Streaming
0 likes · 23 min read
How Kuaishou Guarantees Real‑Time Data Warehouse Reliability During Billion‑Scale Events
Zuoyebang Tech Team
Zuoyebang Tech Team
May 26, 2022 · Cloud Native

How ZRTC Powers Millions of Live Streams: Architecture & Scaling

ZRTC, the real‑time audio‑video platform behind 作业帮, has been refined for over three years to support massive, multi‑cloud, multi‑protocol live streaming, employing a unified SDK, intelligent scheduling, custom SFU services, and extensive performance tuning to achieve high concurrency, low latency, and robust high‑availability.

Cloud NativePerformance OptimizationReal-time Streaming
0 likes · 22 min read
How ZRTC Powers Millions of Live Streams: Architecture & Scaling
Baidu Geek Talk
Baidu Geek Talk
May 23, 2022 · Industry Insights

How Baidu Scales Real-Time Content Safety for Millions of Mini‑Programs

This article explains Baidu's evolving inspection scheduling system for its smart mini‑programs, detailing the challenges of massive page volumes, the V1.0 offline architecture, the V2.0 real‑time enhancements, resource constraints, deduplication logic, and the measurable improvements in risk detection and ecosystem health.

Big DataContent SafetyReal-time Streaming
0 likes · 17 min read
How Baidu Scales Real-Time Content Safety for Millions of Mini‑Programs
dbaplus Community
dbaplus Community
May 21, 2022 · Big Data

5 Trends for 2022: Analytics Engineers, Lakehouse Wars, Real‑Time Pipelines, Cloud Market

The article outlines five major 2022 data trends— the rise of analytics engineers, the intensifying lake‑house competition, the growth of real‑time streaming pipelines and operational analytics, the expanding cloud marketplaces for data tools, and the push toward unified data‑quality terminology—explaining their origins, market impact, and future outlook.

Data QualityLakehouseReal-time Streaming
0 likes · 21 min read
5 Trends for 2022: Analytics Engineers, Lakehouse Wars, Real‑Time Pipelines, Cloud Market
DataFunTalk
DataFunTalk
Apr 20, 2022 · Big Data

OpenMLDB Pulsar Connector: A Real‑time Data Integration Guide

This article presents a step‑by‑step tutorial on using the OpenMLDB Pulsar Connector to stream real‑time data from Apache Pulsar into OpenMLDB, covering connector architecture, key features, Docker‑based installation, sink configuration, schema registration, message production, verification queries, and future roadmap details.

Apache PulsarConnectorData Integration
0 likes · 13 min read
OpenMLDB Pulsar Connector: A Real‑time Data Integration Guide
JD Cloud Developers
JD Cloud Developers
Dec 21, 2021 · Artificial Intelligence

How JD Cloud’s Mobile Super‑Resolution SDK Boosts Video Quality and Cuts Bandwidth by 30%

JD Cloud’s new mobile super‑resolution SDK leverages deep‑learning ESPCN algorithms with ROI‑based processing to upscale video streams in real time, delivering up to 80% longer playback, 30% lower bandwidth costs, and measurable quality gains demonstrated through PSNR, VMAF, and SSIM metrics.

Bandwidth ReductionDeep LearningESPCN
0 likes · 6 min read
How JD Cloud’s Mobile Super‑Resolution SDK Boosts Video Quality and Cuts Bandwidth by 30%
Tencent Cloud Developer
Tencent Cloud Developer
Nov 19, 2021 · Artificial Intelligence

End‑to‑End Breast Cancer Prediction Solution Using Decision Tree on Tencent Cloud AI Platform

This guide details an end‑to‑end breast‑cancer prediction pipeline on Tencent Cloud, covering offline decision‑tree training with TI‑ONE, model packaging as a PMML service, real‑time feature generation via Oceanus and CKafka, and live inference stored in ClickHouse, all within a secure VPC.

AIFlinkReal-time Streaming
0 likes · 19 min read
End‑to‑End Breast Cancer Prediction Solution Using Decision Tree on Tencent Cloud AI Platform
Java Architecture Diary
Java Architecture Diary
Aug 13, 2021 · Backend Development

Streaming Real-Time Sensor Data with Grafana Live and MQTT

Learn how to capture, stream, and visualize real-time IMU sensor data using MQTT and Grafana Live’s streaming API, with step-by-step setup on a Raspberry Pi, configuration of the Grafana MQTT datasource plugin, and visualization via time-series and timeline panels.

Raspberry PiReal-time StreamingSensor Data
0 likes · 7 min read
Streaming Real-Time Sensor Data with Grafana Live and MQTT
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Aug 3, 2021 · Big Data

How BIGO Scaled Real‑Time Messaging by Migrating from Kafka to Pulsar

BIGO replaced its Kafka‑based message‑flow platform with Apache Pulsar to overcome scaling, stability, and operational cost challenges, leveraging Pulsar’s storage‑compute separation, seamless horizontal expansion, low latency, and tight integration with Flink for real‑time ETL and AB‑test pipelines, resulting in billions of messages processed daily with half the hardware cost.

Apache PulsarETLFlink
0 likes · 17 min read
How BIGO Scaled Real‑Time Messaging by Migrating from Kafka to Pulsar
Xianyu Technology
Xianyu Technology
Jul 20, 2021 · Big Data

Design and Implementation of a Content Flow Control System for Xianyu Community

The Xianyu “Play” tab flow‑control system combines task‑specific and rule‑based strategies with a dynamic strategy‑, control‑, and distribution‑chain architecture that integrates real‑time data processing into the recommendation engine, delivering guaranteed exposure, boosting daily posts by 14.4 % and paving the way for multi‑objective, zero‑code control.

Big DataFlow ControlReal-time Streaming
0 likes · 6 min read
Design and Implementation of a Content Flow Control System for Xianyu Community
DataFunTalk
DataFunTalk
May 18, 2021 · Big Data

Evolution and Architecture of Beike Real-Time Computing Platform

Beike's real-time computing platform, led by Liu Liyun, has evolved from early Spark Streaming to a Flink-based system with SQL 1.0, 2.0, and upcoming 3.0, supporting a large-scale data warehouse, event-driven processing, extensive monitoring, and diverse business scenarios across the company's operations.

Data WarehouseEvent-drivenFlink
0 likes · 14 min read
Evolution and Architecture of Beike Real-Time Computing Platform
iQIYI Technical Product Team
iQIYI Technical Product Team
Mar 26, 2021 · Big Data

Evolution of iQIYI's Real-Time Big Data Ecosystem

iQIYI transformed its data infrastructure from a traditional offline T+1 model to a comprehensive real‑time ecosystem—leveraging Kafka, Flink, a three‑layer Stream Data Service Platform, the Talos drag‑and‑drop pipeline, and a Druid‑based analytics platform—to enable low‑latency monitoring, personalized recommendations, ad targeting, and continuous machine‑learning workflows while planning future stream‑batch integration and lake‑warehouse convergence.

AnalyticsBig DataData Warehouse
0 likes · 13 min read
Evolution of iQIYI's Real-Time Big Data Ecosystem
Yuewen Technology
Yuewen Technology
Oct 16, 2020 · Artificial Intelligence

How Intelligent Traffic Distribution Boosts New Book Exposure in Reading Apps

This article describes the design and implementation of an intelligent traffic distribution system for a reading platform, detailing its background, overall architecture, sub-modules such as the small‑traffic experiment platform, near‑line computation, retrieval strategies, pacing algorithms, and how it balances user personalization with content ecosystem growth.

AIBig DataReal-time Streaming
0 likes · 8 min read
How Intelligent Traffic Distribution Boosts New Book Exposure in Reading Apps
DataFunTalk
DataFunTalk
Sep 17, 2020 · Big Data

Design and Implementation of a Scalable User Tag Production Platform

The article explains how a flexible, high‑performance user‑tagging system is built on a batch‑stream integrated architecture using big‑data technologies such as Impala, HDFS, and Flink to support both offline and real‑time label generation for precise marketing, product improvement, and operational analytics.

Big DataFlinkImpala
0 likes · 15 min read
Design and Implementation of a Scalable User Tag Production Platform
Architect
Architect
Jul 4, 2020 · Big Data

Kuaishou Flink Real‑Time Architecture and Spring Festival Gala Assurance Practices

This article details Kuaishou's Flink‑based real‑time computing architecture, its massive cluster scale, and the comprehensive strategies—including overload protection, system stability, pressure testing, and resource guarantees—implemented to ensure reliable streaming for the 2020 Spring Festival Gala and its real‑time dashboard.

Big DataFlinkKuaishou
0 likes · 12 min read
Kuaishou Flink Real‑Time Architecture and Spring Festival Gala Assurance Practices
Alibaba Cloud Developer
Alibaba Cloud Developer
Jun 16, 2020 · Databases

How Youku Scales Billions of Video Nodes with Real‑Time Graph Databases

Facing billions of video entities and edges, Youku’s engineering team replaced traditional relational stores with a graph‑based knowledge platform, leveraging Alibaba’s Blink streaming engine and Lindorm to enable real‑time, incremental updates, unified UDF logic, and scalable feature computation for search and recommendation.

Big DataKnowledge GraphReal-time Streaming
0 likes · 10 min read
How Youku Scales Billions of Video Nodes with Real‑Time Graph Databases
Huolala Tech
Huolala Tech
May 28, 2020 · Big Data

How Flink Powers Real‑Time Risk Control at HuoLaLa: Architecture and Insights

This article explains Flink's role in HuoLaLa's risk‑control system, covering its background, the Lambda‑style architecture that combines batch and streaming, the real‑time data pipeline, machine‑learning models, and operational safeguards that together enable proactive fraud detection.

Big Data ArchitectureFlinkLambda architecture
0 likes · 16 min read
How Flink Powers Real‑Time Risk Control at HuoLaLa: Architecture and Insights
dbaplus Community
dbaplus Community
Jan 6, 2020 · Big Data

How 58.com Built a Scalable Flink‑Based Real‑Time Data Platform (Wstream)

The article details how 58.com designed and evolved its one‑stop real‑time computation platform Wstream, migrating from Storm and Spark Streaming to Apache Flink, and describes the architecture, task isolation, stream‑SQL features, monitoring, and ongoing optimizations that enable processing of over 600 billion records daily.

Big DataFlinkReal-time Streaming
0 likes · 12 min read
How 58.com Built a Scalable Flink‑Based Real‑Time Data Platform (Wstream)
21CTO
21CTO
Nov 27, 2019 · Big Data

How Xiaohongshu Scales Real‑Time Personalized Recommendations with Flink

The article summarizes Guo Yi’s 2019 Alibaba Cloud conference talk, outlining Xiaohongshu’s personalized recommendation architecture, detailing the data stack from ingestion to warehouse, and showcasing a Flink‑based real‑time multi‑dimensional user behavior aggregation use case, followed by a vision for the next year’s data architecture evolution.

Data ArchitectureFlinkReal-time Streaming
0 likes · 3 min read
How Xiaohongshu Scales Real‑Time Personalized Recommendations with Flink
DataFunTalk
DataFunTalk
Nov 21, 2019 · Big Data

Evolution of 58.com Real-Time Computing Platform and the One-Stop Streaming Data Processing System Wstream

The article details the technical evolution of 58.com’s real-time computing platform—from Storm and Spark Streaming to a Flink‑based one‑stop solution called Wstream—covering use cases, architecture, stability measures, migration from Storm, operational diagnostics, and future development plans.

Big DataFlinkReal-time Streaming
0 likes · 11 min read
Evolution of 58.com Real-Time Computing Platform and the One-Stop Streaming Data Processing System Wstream
58 Tech
58 Tech
Nov 20, 2019 · Big Data

Evolution of 58.com Real-Time Computing Platform and the One‑Stop Streaming Platform Wstream Built on Flink

This article details the technical evolution of 58.com’s real‑time computing platform, describing the shift from Storm and Spark Streaming to Apache Flink, the design of the one‑stop Wstream platform, its large‑scale deployment, stability measures, SQL streaming capabilities, task migration, diagnostics, optimizations, and future plans.

FlinkReal-time StreamingTask Migration
0 likes · 11 min read
Evolution of 58.com Real-Time Computing Platform and the One‑Stop Streaming Platform Wstream Built on Flink
dbaplus Community
dbaplus Community
Oct 22, 2019 · Big Data

How Weibo Built a Billion‑Log Real‑Time Data Platform with Flink

This article details how Weibo’s advertising team designed and implemented a real‑time data platform capable of processing over a hundred billion daily logs, covering technology selection, Flink advantages, architecture evolution, data processing pipelines, component libraries, fault‑tolerance strategies, and the construction of a multi‑layer real‑time data warehouse.

Big DataCheckpointData Architecture
0 likes · 25 min read
How Weibo Built a Billion‑Log Real‑Time Data Platform with Flink
HomeTech
HomeTech
Oct 9, 2019 · Big Data

Design and Implementation of a Flink‑Based Real‑Time Data Platform at Autohome

This article describes how Autohome migrated its real‑time analytics from Storm to a Flink‑SQL platform, detailing the architectural design, development and operational advantages, practical use cases such as recommendation metrics, and future plans for ecosystem expansion and open‑source release.

Data WarehouseFlinkReal-time Streaming
0 likes · 12 min read
Design and Implementation of a Flink‑Based Real‑Time Data Platform at Autohome
Xueersi Online School Tech Team
Xueersi Online School Tech Team
Sep 6, 2019 · Big Data

Real-Time Data Architecture, Evolution, and Applications at an Online School

The article details the six‑layer big‑data architecture of an online school, chronicles its migration from Storm to Spark Streaming and finally to Flink, and showcases concrete real‑time applications such as gateway monitoring, user‑profile tagging, renewal reporting, and advertising analysis, while outlining future development directions.

AnalyticsBig Data ArchitectureFlink
0 likes · 14 min read
Real-Time Data Architecture, Evolution, and Applications at an Online School
Big Data Technology & Architecture
Big Data Technology & Architecture
Aug 25, 2019 · Big Data

Tencent Oceanus: Evolution, Productization, and Optimizations of Real‑Time Stream Computing with Flink

This article recounts Tencent's journey from adopting Flink to building the Oceanus platform, detailing its architecture, product features, and a series of deep extensions—including UI redesign, JobManager failover, checkpoint handling, enhanced windows, LocalKeyBy, watermark idle detection, and log isolation—aimed at supporting trillion‑scale real‑time data processing.

Big DataFlinkOceanus
0 likes · 18 min read
Tencent Oceanus: Evolution, Productization, and Optimizations of Real‑Time Stream Computing with Flink
Xianyu Technology
Xianyu Technology
Jun 20, 2019 · Big Data

Design of a High-Performance Real-Time Data Processing System for Service Diagnosis

The paper presents a high‑performance real‑time data processing pipeline that collects, transports, preprocesses, and computes service logs and metrics using Alibaba Logtail, LogHub, and an enhanced Flink (Blink) engine, persisting root‑cause graphs in Lindorm, achieving sub‑3‑second latency for tens of millions of events per second and cutting diagnosis time to about five seconds.

FlinkReal-time Streamingarchitecture
0 likes · 10 min read
Design of a High-Performance Real-Time Data Processing System for Service Diagnosis
Big Data Technology & Architecture
Big Data Technology & Architecture
Jun 2, 2019 · Big Data

Tencent's Oceanus Real-Time Stream Computing Platform and Flink Optimizations

The article presents Tencent's evolution of real‑time stream processing using Flink, the design of the Oceanus one‑stop visual platform, and a series of deep extensions and optimizations—including UI redesign, JobManager failover, checkpoint handling, enhanced windows, LocalKeyBy, idle detection, and log isolation—aimed at supporting petabyte‑scale data workloads.

Big DataFlinkOceanus
0 likes · 16 min read
Tencent's Oceanus Real-Time Stream Computing Platform and Flink Optimizations
Alibaba Cloud Developer
Alibaba Cloud Developer
May 23, 2019 · Big Data

How Blink Powers Alibaba’s Real‑Time Supply‑Chain Data Warehouse

This article explains how Alibaba's Blink engine tackles the complex challenges of building a real‑time supply‑chain data warehouse—covering retroduction, dimension‑table joins, data skew, timeout statistics, zero‑point optimizations, and future directions—through SQL‑based stream processing and intelligent resource tuning.

Data SkewDimension joinFlink
0 likes · 14 min read
How Blink Powers Alibaba’s Real‑Time Supply‑Chain Data Warehouse
dbaplus Community
dbaplus Community
Mar 21, 2019 · Big Data

How Real-Time Data Platforms Evolve: From Storm to Flink and Kubernetes

This article summarizes Wang Xinchun's 2018 DAMS China Data Asset Management Summit talk, detailing the current state, core services, responsibilities, evolution, architecture, challenges, and future directions of a large‑scale real‑time data platform built on Storm, Spark, Flink, and Kubernetes, including a unified data management approach.

Data PlatformFlinkKubernetes
0 likes · 22 min read
How Real-Time Data Platforms Evolve: From Storm to Flink and Kubernetes
Youzan Coder
Youzan Coder
Jan 16, 2019 · Big Data

How Youzan Scaled Real‑Time Analytics with Flink: Architecture, Pitfalls, and Lessons

This article walks through Youzan's real‑time platform architecture, explains why Flink was chosen over Spark Structured Streaming, details practical challenges such as container over‑provisioning and monitoring overhead, shares solutions for Spring integration and async caching, and outlines future directions for SQL‑based streaming and scheduler improvements.

Big DataFlinkReal-time Streaming
0 likes · 19 min read
How Youzan Scaled Real‑Time Analytics with Flink: Architecture, Pitfalls, and Lessons
Xianyu Technology
Xianyu Technology
Nov 6, 2018 · Big Data

Technical Evolution of Xianyu Real-Time Selection System for Double Eleven

To meet Double‑Eleven’s sub‑second, billion‑item feed demands, Alibaba’s Xianyu selection system evolved from a Solr‑based search pipeline through offline batch and PostgreSQL attempts to a Blink‑powered real‑time stream platform using Niagara’s low‑latency LSM storage, delivering high‑throughput, personalized product feeds.

AlibabaBig DataFlink
0 likes · 23 min read
Technical Evolution of Xianyu Real-Time Selection System for Double Eleven
21CTO
21CTO
Oct 19, 2018 · Big Data

How Meituan Scales Real‑Time Computing with Flink: Architecture, Challenges & Solutions

This article summarizes Meituan’s real‑time computing platform, detailing its layered architecture built on Kafka, Flink on YARN, state management, resource isolation, fault tolerance, monitoring, and the Petra metric aggregation system, while highlighting the challenges faced and the solutions implemented to achieve high‑throughput, low‑latency stream processing at massive scale.

Big DataFlinkReal-time Streaming
0 likes · 18 min read
How Meituan Scales Real‑Time Computing with Flink: Architecture, Challenges & Solutions
21CTO
21CTO
Apr 28, 2018 · Big Data

Why Kafka Dominates Real-Time Data Streaming in the Big Data Era

This article explains why Kafka has become essential for real‑time data streaming in the big‑data era, detailing its performance advantages, core use cases, major adopters, multilingual support, and how its scalable storage and retention mechanisms empower modern data pipelines.

KafkaReal-time Streaming
0 likes · 10 min read
Why Kafka Dominates Real-Time Data Streaming in the Big Data Era
21CTO
21CTO
Feb 20, 2018 · Big Data

Why Real-Time Streaming Is the Next Big Data Revolution for Developers

This article explains how real-time streaming has evolved from batch Hadoop systems through Lambda architecture to modern Kappa-style pipelines, highlighting its growing importance for developers, enterprises, and the integration of streaming with microservices, AI, and cloud-native technologies.

AI integrationBig DataKappa architecture
0 likes · 8 min read
Why Real-Time Streaming Is the Next Big Data Revolution for Developers
58 Tech
58 Tech
Dec 15, 2017 · Big Data

Design and Architecture of WMDA: A Comprehensive User Behavior Analysis Platform

The article details WMDA, a no‑code and manual‑code data collection platform for PC, mobile and app that supports real‑time and offline user behavior analysis, describing its functional model, behavior taxonomy, five‑layer architecture, tracking techniques, circle‑selection, data services, streaming and batch processing pipelines, and related technologies such as Storm, Spark, Druid and Roaring Bitmap.

Big DataDruidReal-time Streaming
0 likes · 18 min read
Design and Architecture of WMDA: A Comprehensive User Behavior Analysis Platform
ITPUB
ITPUB
Nov 13, 2017 · Big Data

How Real-Time Big Data Streaming Powers Double 11 E‑Commerce Success

The article explains how continuous data generation and real‑time stream processing enable e‑commerce platforms like NetEase Kaola to handle massive Double 11 traffic, showcasing use cases, architectural shifts from batch to incremental computing, and the technical challenges of latency, accuracy, and fault tolerance.

Distributed SystemsReal-time StreamingSQL
0 likes · 15 min read
How Real-Time Big Data Streaming Powers Double 11 E‑Commerce Success
Alibaba Cloud Developer
Alibaba Cloud Developer
Aug 4, 2017 · Big Data

From PhD to Alibaba: How a Real‑Time Streaming Expert Built Blink on Flink

Alibaba algorithm engineer Shi Xiaogang shares his journey from a Peking University PhD researching real‑time iterative computation on data streams to developing Blink’s state management and recovery features in Flink, highlighting the challenges of transitioning from academia to industry and the impact of large‑scale real‑time systems.

FlinkReal-time Streamingalgorithm engineering
0 likes · 13 min read
From PhD to Alibaba: How a Real‑Time Streaming Expert Built Blink on Flink
Alibaba Cloud Developer
Alibaba Cloud Developer
Jan 4, 2017 · Big Data

How Alibaba Powered Double 11 with Real‑Time Big Data Processing

Alibaba’s Double 11 live‑data dashboards required ultra‑high‑precision, low‑latency real‑time processing of billions of events, and the article explains the end‑to‑end architecture—including DRC, TimeTunnel, Galaxy, OTS, XTool, and OneService—used to achieve million‑plus QPS, fault‑tolerance, and flexible data collection.

AlibabaBig Data ArchitectureReal-time Streaming
0 likes · 14 min read
How Alibaba Powered Double 11 with Real‑Time Big Data Processing
21CTO
21CTO
Dec 21, 2015 · Fundamentals

Why UDP Is Overtaking TCP: Speed, Stability, and Real‑World Wins

With broadband speeds soaring, UDP’s simplicity and low latency now outperform TCP in web browsing, streaming, gaming, and IoT, as real‑world tests from Akamai and Google show dramatically lower packet loss, faster connections, and reduced buffering, signaling a shift toward UDP‑centric networking.

QUICReal-time StreamingTCP
0 likes · 5 min read
Why UDP Is Overtaking TCP: Speed, Stability, and Real‑World Wins
Architect
Architect
Dec 17, 2015 · Backend Development

Technical Architecture Evolution for Real-Time Multi-Party Audio and Video Streaming

The article details the progressive architectural decisions and technical solutions behind a real-time multi‑party audio/video streaming platform, covering format research, backend service design, protocol choices, codec selection, node topology redesign, and video integration to achieve low‑latency, cross‑platform communication.

Opus codecReal-time StreamingWebRTC
0 likes · 15 min read
Technical Architecture Evolution for Real-Time Multi-Party Audio and Video Streaming
Baidu Tech Salon
Baidu Tech Salon
Oct 29, 2014 · Big Data

Inside Baidu’s Real-Time Big Data Platforms: Dstream and TM Explained

This article examines Baidu’s home‑grown real‑time big‑data platforms Dstream and TM, detailing their architectures, performance metrics, key features, and practical use cases such as log ETL and real‑time bidding, while highlighting how they meet millisecond‑level processing demands.

BaiduBig DataDstream
0 likes · 9 min read
Inside Baidu’s Real-Time Big Data Platforms: Dstream and TM Explained