Tagged articles

Data Shuffle

2 articles · Page 1 of 1

Nov 22, 2021 · Big Data

How Flink’s Sort‑Shuffle Boosts Large‑Scale Batch Processing Performance

This article explains how Flink’s new Sort‑Shuffle mechanism improves stability and performance for massive batch jobs by reducing file counts, optimizing I/O, minimizing memory usage, and providing detailed implementation, test results, tuning tips, and future enhancements.

Batch ProcessingData ShuffleFlink

0 likes · 17 min read

How Flink’s Sort‑Shuffle Boosts Large‑Scale Batch Processing Performance

Big Data Technology Architecture

Nov 15, 2021 · Big Data

Flink Sort‑Shuffle: Design, Implementation, and Performance Evaluation

This article explains how Flink's new sort‑shuffle mechanism improves large‑scale batch processing by reducing file counts, optimizing I/O, lowering memory usage, and delivering up to tenfold speedups, while also detailing configuration tips and future enhancements.

Batch ProcessingData ShuffleFlink

0 likes · 16 min read

Flink Sort‑Shuffle: Design, Implementation, and Performance Evaluation