Custom Flink Scheduler Enhancements: Resource Balancing, Task Migration, and TmRestart Strategy

The article details Dewu’s custom Flink scheduler, DwScheduler, which adds JSON‑based resource specifications, per‑TaskManager slot sharing for balanced CPU use, hot TaskManager migration callbacks, and a new TmRestart strategy for rapid pod‑process recovery, offering practical techniques to enhance real‑time stream processing stability and performance.

Apache FlinkResource ManagementScheduler

0 likes · 9 min read

Custom Flink Scheduler Enhancements: Resource Balancing, Task Migration, and TmRestart Strategy

OPPO Kernel Craftsman

Apr 3, 2020 · Fundamentals

Linux CPU Load Balancing: Framework and Implementation Analysis

Linux kernel load balancing, explained through the 5.4.28 framework, defines precise CPU load via PELT, balances tasks according to each core’s capacity using hierarchical scheduling domains and groups, details class‑specific strategies, illustrates migration in a big‑little 8‑core system, and outlines the three balancing scenarios and their trigger mechanisms.

CFS SchedulerCPU SchedulerCPU topology

0 likes · 15 min read

Linux CPU Load Balancing: Framework and Implementation Analysis

dbaplus Community

Jan 6, 2020 · Big Data

How 58.com Built a Scalable Flink‑Based Real‑Time Data Platform (Wstream)

The article details how 58.com designed and evolved its one‑stop real‑time computation platform Wstream, migrating from Storm and Spark Streaming to Apache Flink, and describes the architecture, task isolation, stream‑SQL features, monitoring, and ongoing optimizations that enable processing of over 600 billion records daily.

Big DataFlinkReal-time Streaming

0 likes · 12 min read

How 58.com Built a Scalable Flink‑Based Real‑Time Data Platform (Wstream)

58 Tech

Nov 20, 2019 · Big Data

Evolution of 58.com Real-Time Computing Platform and the One‑Stop Streaming Platform Wstream Built on Flink

This article details the technical evolution of 58.com’s real‑time computing platform, describing the shift from Storm and Spark Streaming to Apache Flink, the design of the one‑stop Wstream platform, its large‑scale deployment, stability measures, SQL streaming capabilities, task migration, diagnostics, optimizations, and future plans.

FlinkReal-time StreamingTask Migration

0 likes · 11 min read

Evolution of 58.com Real-Time Computing Platform and the One‑Stop Streaming Platform Wstream Built on Flink