BestHub
Sign in
Tagged articles
224 articles
Page 2 of 3
Big Data Technology & Architecture
Big Data Technology & Architecture
Aug 21, 2023 · Big Data

Key Features and Benefits of Lakehouse Frameworks Hudi, Iceberg, and Paimon

This note outlines how Hudi, Iceberg, and Paimon provide unified batch‑stream storage, UPSERT support, time‑travel capabilities, and lower development costs, enabling a streaming‑warehouse architecture that offers near‑real‑time latency, consistent semantics, persisted intermediate results, and easier historical data repair.

Batch ProcessingHudiIceberg
0 likes · 5 min read
Key Features and Benefits of Lakehouse Frameworks Hudi, Iceberg, and Paimon
Architect
Architect
Jul 30, 2023 · Backend Development

Spring Boot Integration with Spring Batch: A Comprehensive Tutorial

This article provides a step‑by‑step guide on integrating Spring Batch with Spring Boot, covering configuration, job and step setup, ItemReader/Processor/Writer implementations, database interactions, handling CSV and database sources, and troubleshooting common issues such as connection‑pool compatibility.

BackendBatch ProcessingSpring Batch
0 likes · 22 min read
Spring Boot Integration with Spring Batch: A Comprehensive Tutorial
Senior Tony
Senior Tony
Jul 29, 2023 · Fundamentals

Essential Coding Habits Every Engineer Should Master

The article outlines practical coding habits—thorough input validation, comprehensive logging, careful RPC handling, batch processing, cautious SQL execution, safe extensions, disciplined refactoring, minimal dependencies, data consistency, and avoiding over‑engineering—to help engineers write reliable, maintainable code.

Batch ProcessingData ConsistencyRPC
0 likes · 9 min read
Essential Coding Habits Every Engineer Should Master
Architect
Architect
Jul 10, 2023 · Big Data

Understanding Lambda Architecture for Real‑Time Billion‑Scale Data Analysis

This article explains the Lambda Architecture—a three‑layer big‑data processing model combining batch and speed layers to deliver accurate, low‑latency analytics, and illustrates its use with Twitter hashtag tracking and a smart‑parking recommendation system.

Batch ProcessingBig DataLambda architecture
0 likes · 10 min read
Understanding Lambda Architecture for Real‑Time Billion‑Scale Data Analysis
DevOps
DevOps
Jul 4, 2023 · Operations

Best Practices for Batch Processing in Microservice Architecture

This article examines batch processing in microservice environments, defining its role versus data warehouses, exploring optimal placement across architectural layers, discussing access methods, and providing criteria such as complexity, consistency, security, performance, reliability, and scalability to guide the selection of an appropriate batch processing architecture.

Batch ProcessingMicroservicesarchitecture
0 likes · 11 min read
Best Practices for Batch Processing in Microservice Architecture
Efficient Ops
Efficient Ops
Jun 13, 2023 · Operations

How to Automate Batch Job Retries and Eliminate Midnight Outages

This article explores a real‑world scenario where a support manager faces nightly batch job interruptions, analyzes common database and environment failures, and presents a systematic redesign of the batch framework and executor to enable automatic retry, reducing manual intervention and improving operational reliability.

Batch ProcessingException Handlingmysql
0 likes · 7 min read
How to Automate Batch Job Retries and Eliminate Midnight Outages
Architecture Breakthrough
Architecture Breakthrough
May 8, 2023 · Backend Development

Designing a Robust Batch Processing Module: Key Architecture Insights

This article outlines the essential architectural considerations for building a production‑ready batch processing module, covering design principles, task scheduling, parallelism, error handling, resource management, data‑layer concerns, deployment strategies, and monitoring practices.

Backend ArchitectureBatch ProcessingScalability
0 likes · 10 min read
Designing a Robust Batch Processing Module: Key Architecture Insights
DataFunSummit
DataFunSummit
Apr 28, 2023 · Big Data

Building a Unified Streaming‑Batch Storage Architecture at Xiaohongshu

This article presents Xiaohongshu's design and implementation of a unified streaming‑batch storage system that integrates Lambda architecture, Kafka, Flink, Iceberg, and modern OLAP engines to solve real‑time data warehouse pain points and enable consistent, exactly‑once analytics across streaming and batch workloads.

Batch ProcessingFlinkIceberg
0 likes · 16 min read
Building a Unified Streaming‑Batch Storage Architecture at Xiaohongshu
Java Architect Essentials
Java Architect Essentials
Apr 28, 2023 · Backend Development

12 Proven Strategies to Supercharge API Performance in Java

This article outlines twelve practical techniques—including batch processing, asynchronous execution, caching, pre‑processing, pooling, parallelization, indexing, transaction sizing, code refactoring, pagination, SQL tuning, and lock granularity—to dramatically reduce API latency and improve overall backend efficiency.

API performanceAsynchronousBatch Processing
0 likes · 9 min read
12 Proven Strategies to Supercharge API Performance in Java
IT Architects Alliance
IT Architects Alliance
Apr 23, 2023 · Backend Development

Common Interface Performance Optimization Strategies

This article outlines a comprehensive set of backend interface optimization techniques—including batch processing, asynchronous execution, caching, pre‑processing, pooling, parallelization, indexing, transaction sizing, code restructuring, deep pagination, SQL tuning, and lock granularity—to reduce latency and improve overall system efficiency.

API optimizationAsynchronous ExecutionBatch Processing
0 likes · 8 min read
Common Interface Performance Optimization Strategies
Top Architect
Top Architect
Apr 21, 2023 · Backend Development

Common Interface Performance Optimization Strategies

This article presents a comprehensive set of practical techniques—including batch processing, asynchronous execution, caching, pre‑processing, pooling, parallelization, indexing, transaction size reduction, code refactoring, and pagination optimization—to systematically reduce API latency and improve overall backend service efficiency.

Batch Processingcachingdatabase
0 likes · 12 min read
Common Interface Performance Optimization Strategies
ByteDance Cloud Native
ByteDance Cloud Native
Apr 20, 2023 · Cloud Native

How Dragonfly Accelerates Image Distribution and Scales Kubernetes Batch Processing

At KubeCon+CloudNativeCon 2023 in Amsterdam, Volcano Engine and ByteDance presented two technical sessions covering Dragonfly's P2P image distribution best practices and large‑scale Kubernetes batch processing strategies, offering deep insights and real‑world implementations for cloud‑native developers.

Batch ProcessingDragonflyImage Distribution
0 likes · 4 min read
How Dragonfly Accelerates Image Distribution and Scales Kubernetes Batch Processing
ITPUB
ITPUB
Mar 24, 2023 · Big Data

What’s New in Apache Flink 1.17? Key Features, Performance Gains, and Streaming Warehouse Advances

Apache Flink 1.17 introduces a suite of batch and streaming enhancements—including a new Streaming Warehouse API, significant TPC‑DS performance boosts, adaptive batch scheduling, improved checkpointing, expanded SQL capabilities, Hive connector upgrades, and broader filesystem support—while also delivering upgrades to FRocksDB, Calcite, and the token framework to strengthen its position as a leading unified data‑processing engine.

Apache FlinkBatch ProcessingCheckpoint
0 likes · 23 min read
What’s New in Apache Flink 1.17? Key Features, Performance Gains, and Streaming Warehouse Advances
DataFunTalk
DataFunTalk
Mar 12, 2023 · Big Data

Apache Kyuubi 1.6.0 Feature Overview and Enhancements

The article provides a comprehensive walkthrough of Apache Kyuubi 1.6.0, detailing server‑side enhancements such as batch (JAR) task submission, metadata store and unified API/authentication, client‑side improvements to the built‑in JDBC driver and Beeline, as well as engine plugins for Spark, Flink, Trino and Hive, and concludes with the community’s roadmap and statistics.

Apache KyuubiBatch ProcessingBig Data
0 likes · 12 min read
Apache Kyuubi 1.6.0 Feature Overview and Enhancements
DataFunTalk
DataFunTalk
Feb 21, 2023 · Databases

Building a Stream‑Batch Integrated Data Architecture with Apache Doris at SelectDB

This article details how SelectDB’s data technology architect designed and implemented a new stream‑batch unified data platform using Apache Doris, covering the shortcomings of the early CDH‑based architecture, the selection process, data modeling, ingestion pipelines, performance testing, operational optimizations, and future plans.

Apache DorisBatch ProcessingBig Data
0 likes · 17 min read
Building a Stream‑Batch Integrated Data Architecture with Apache Doris at SelectDB
DataFunTalk
DataFunTalk
Feb 2, 2023 · Big Data

SeaTunnel: Design Goals, Current Status, Architecture, and Future Roadmap

This article provides a comprehensive overview of Apache SeaTunnel, covering its design objectives, current capabilities such as multi‑engine support and extensive connector ecosystem, detailed architecture including engine‑independent APIs and execution flows, and outlines the upcoming roadmap to expand connectors, launch a visual web UI, and introduce a dedicated SeaTunnel Engine.

ApacheBatch ProcessingBig Data
0 likes · 12 min read
SeaTunnel: Design Goals, Current Status, Architecture, and Future Roadmap
DataFunSummit
DataFunSummit
Jan 8, 2023 · Big Data

Streaming‑Batch Integrated Real‑time Multi‑dimensional Analysis

This article presents a comprehensive overview of evolving big‑data architectures—from classic offline warehouses to Lambda and Kappa models—and details a streaming‑batch integrated solution that addresses latency, data freshness, and multi‑table join challenges to achieve minute‑level real‑time multi‑dimensional analytics.

Batch ProcessingKappa architectureLambda architecture
0 likes · 18 min read
Streaming‑Batch Integrated Real‑time Multi‑dimensional Analysis
Big Data Technology & Architecture
Big Data Technology & Architecture
Jan 3, 2023 · Big Data

Migrating Hive SQL Jobs to Flink Using the SQL Gateway

This article explains how to use Apache Flink 1.16's SQL Gateway to migrate Hive SQL tasks to Flink, covering the underlying Hive‑on‑Flink architecture, dialect compatibility, streaming and batch demos, configuration details, and practical tips for developers and platform engineers.

Batch ProcessingBig DataFlink
0 likes · 19 min read
Migrating Hive SQL Jobs to Flink Using the SQL Gateway
Architects' Tech Alliance
Architects' Tech Alliance
Dec 25, 2022 · Operations

Mastering Time‑Space Trade‑offs: Indexing, Caching, Compression & More

This article consolidates three performance‑optimization posts, detailing six universal time‑space trade‑off techniques—indexing, caching, compression, prefetching, peak‑valley smoothing, and batch processing—plus four advanced parallelism strategies, illustrated with real‑world analogies and practical guidelines for developers.

Batch Processingcachingcompression
0 likes · 22 min read
Mastering Time‑Space Trade‑offs: Indexing, Caching, Compression & More
Data Thinking Notes
Data Thinking Notes
Dec 21, 2022 · Big Data

Why Your Spark Batch Job Fails: Memory Limits, Data Skew, and Practical Fixes

This article examines a recurring Spark batch task failure caused by OutOfMemory errors and data skew, details the investigation steps—including increasing executor memory, raising parallelism, and analyzing shuffle metrics—and proposes solutions such as data validation, filtering oversized keys, and memory adjustments.

Batch ProcessingData SkewOutOfMemory
0 likes · 4 min read
Why Your Spark Batch Job Fails: Memory Limits, Data Skew, and Practical Fixes
Big Data Technology & Architecture
Big Data Technology & Architecture
Dec 15, 2022 · Big Data

Migrating Hive SQL to Flink SQL: Motivation, Challenges, Practice, Demo, and Future Plans

This technical article presents a comprehensive overview of migrating Hive SQL to Flink SQL, covering the motivations behind the migration, key challenges such as compatibility, stability and performance, practical implementation steps, a detailed demo, future development directions, and a Q&A session addressing common concerns.

Batch ProcessingBig DataData Lake
0 likes · 13 min read
Migrating Hive SQL to Flink SQL: Motivation, Challenges, Practice, Demo, and Future Plans
Top Architect
Top Architect
Dec 5, 2022 · Backend Development

Integrating Spring Batch with Spring Boot: Full Tutorial and Example

This article provides a comprehensive step‑by‑step guide on integrating Spring Batch with Spring Boot, covering configuration, entity and mapper definitions, job, step, reader, processor, writer implementations, database setup, error handling, and execution via a REST controller, complete with code examples.

Batch ProcessingMyBatisSpring Batch
0 likes · 22 min read
Integrating Spring Batch with Spring Boot: Full Tutorial and Example
Java High-Performance Architecture
Java High-Performance Architecture
Dec 4, 2022 · Backend Development

How to Efficiently Import and Export Millions of Records with EasyExcel and POI

This article explains the challenges of large‑scale Excel import/export in Java, compares POI workbook implementations, introduces EasyExcel for high‑performance handling of hundreds of thousands to millions of rows, and provides detailed code examples for batch querying, sheet management, and JDBC batch insertion with transaction control.

Batch ProcessingData ExportJDBC
0 likes · 23 min read
How to Efficiently Import and Export Millions of Records with EasyExcel and POI
Java Architect Essentials
Java Architect Essentials
Nov 24, 2022 · Backend Development

How to Efficiently Import and Export Millions of Rows with POI and EasyExcel

This article explains how to overcome the limitations of traditional Apache POI when handling massive Excel import/export tasks by comparing POI workbook types, selecting the appropriate one, and using EasyExcel together with batch database queries and JDBC transactions to process up to three million rows efficiently.

Apache POIBatch ProcessingJDBC
0 likes · 23 min read
How to Efficiently Import and Export Millions of Rows with POI and EasyExcel
Top Architect
Top Architect
Nov 22, 2022 · Big Data

Efficient Massive Excel Import/Export with POI and EasyExcel in Java

This article explains how to efficiently import and export massive datasets (up to millions of rows) between Excel and databases using Apache POI, SXSSF, and Alibaba's EasyExcel, comparing workbook types, outlining performance considerations, and providing Java code examples for batch processing, paging, and transaction management.

Batch ProcessingBig DataExcel
0 likes · 23 min read
Efficient Massive Excel Import/Export with POI and EasyExcel in Java
ITPUB
ITPUB
Nov 18, 2022 · Big Data

How Xiaomi Uses Iceberg for Real‑Time Streaming and Batch Data Lakes

This article introduces Iceberg’s table‑format fundamentals, details Xiaomi’s large‑scale deployment of Iceberg for CDC and log ingestion, explores their streaming‑batch integration experiments, outlines future roadmap items, and provides a comprehensive Q&A covering practical challenges and solutions.

Batch ProcessingBig DataData Lake
0 likes · 23 min read
How Xiaomi Uses Iceberg for Real‑Time Streaming and Batch Data Lakes
Architecture Digest
Architecture Digest
Nov 7, 2022 · Backend Development

Comprehensive Guide to Software Performance Optimization: Indexing, Compression, Caching, Prefetching, Throttling, and Batch Processing

This article presents a thorough, multi‑part exploration of software performance optimization techniques—including indexing, compression, caching, prefetching, peak‑shaving, and batch processing—explaining their principles, trade‑offs, practical applications, and how they relate to hardware constraints and system design.

Batch ProcessingScalabilitycaching
0 likes · 36 min read
Comprehensive Guide to Software Performance Optimization: Indexing, Compression, Caching, Prefetching, Throttling, and Batch Processing
Top Architect
Top Architect
Oct 20, 2022 · Databases

Performance Evaluation of Inserting Billion‑Scale Data into MySQL Using Java: MyBatis vs JDBC vs Batch Processing

This article presents a detailed performance test of inserting massive data into MySQL with Java, comparing three strategies—MyBatis lightweight insertion without transactions, direct JDBC handling with and without transactions, and JDBC batch processing—showing timing results for millions of rows and offering practical recommendations for high‑throughput data loading.

Batch ProcessingDatabase OptimizationJDBC
0 likes · 14 min read
Performance Evaluation of Inserting Billion‑Scale Data into MySQL Using Java: MyBatis vs JDBC vs Batch Processing
DeWu Technology
DeWu Technology
Oct 10, 2022 · Big Data

Offline and Real-Time User Profile Fusion Architecture

The architecture combines a nightly batch job that generates offline user profiles stored in HBase with a Flink‑based stream layer that lazily loads those profiles on app start and creates real‑time updates, then fuses both streams into a unified, timestamp‑ordered profile in Redis, forming a Lambda‑style pipeline.

Batch ProcessingFlinkHBase
0 likes · 10 min read
Offline and Real-Time User Profile Fusion Architecture
Architect's Tech Stack
Architect's Tech Stack
Oct 9, 2022 · Backend Development

Spring Batch Overview: Architecture, Core Concepts, and Practical Usage

This article provides a comprehensive introduction to Spring Batch, covering its purpose for large‑scale data processing, overall architecture, key concepts such as Job, Step, ItemReader/Writer/Processor, chunk processing, skip policies, and practical configuration examples with Java code.

Batch ProcessingSpring BatchSpring Framework
0 likes · 17 min read
Spring Batch Overview: Architecture, Core Concepts, and Practical Usage
dbaplus Community
dbaplus Community
Sep 6, 2022 · Backend Development

How to Scale High‑Concurrency Write Operations with Asynchronous Patterns

This article examines a high‑traffic video‑watching scenario, identifies database write bottlenecks, and presents four asynchronous solutions—thread‑pool, in‑memory queue with scheduled tasks, message‑queue, and Agent‑plus‑MQ—detailing their implementation steps, trade‑offs, and best‑practice guidelines.

AsynchronousBatch Processingbackend optimization
0 likes · 8 min read
How to Scale High‑Concurrency Write Operations with Asynchronous Patterns
DataFunTalk
DataFunTalk
Aug 25, 2022 · Big Data

Applying OpenMLDB for Efficient AI Toolchain and Data‑Driven Architecture at Akulaku

This article presents Akulaku’s practical experience with OpenMLDB, describing the company’s data‑driven requirements, the design of a unified stream‑batch architecture, implementation details across offline, online and RocksDB modes, and future recommendations for high‑performance, scenario‑agnostic big‑data processing.

AIBatch ProcessingOpenMLDB
0 likes · 17 min read
Applying OpenMLDB for Efficient AI Toolchain and Data‑Driven Architecture at Akulaku
DaTaobao Tech
DaTaobao Tech
Aug 11, 2022 · Big Data

Unify SQL Engine: Integrating Stream, Batch, and Online Computing for Data Warehousing

The article describes how fragmented real‑time, batch, and online data‑warehouse pipelines suffer from low productivity and inconsistent data quality, and introduces a unified SQL engine built on Apache Calcite that parses, optimizes, and compiles a single SQL statement into executable plans for ODPS, Flink, or Java, leveraging Janino code generation, multi‑backend state storage, and snapshot‑join semantics to boost performance and simplify development.

Batch ProcessingCalciteFlink
0 likes · 16 min read
Unify SQL Engine: Integrating Stream, Batch, and Online Computing for Data Warehousing
Java High-Performance Architecture
Java High-Performance Architecture
Aug 1, 2022 · Backend Development

Master Spring Batch: From Basics to Advanced Architecture and Best Practices

This article introduces Spring Batch, a lightweight Java batch‑processing framework, explains its architecture and core concepts such as Job, Step, ItemReader/Writer/Processor, chunk handling, skip strategies, and provides practical guidelines for building reliable, high‑throughput batch jobs while avoiding common pitfalls like memory exhaustion.

Batch ProcessingChunkJob
0 likes · 18 min read
Master Spring Batch: From Basics to Advanced Architecture and Best Practices
Programmer DD
Programmer DD
Jul 22, 2022 · Backend Development

Master Spring Batch: From Core Concepts to Advanced Chunk Processing

This article provides a comprehensive introduction to Spring Batch, covering its purpose, core architecture, key concepts such as Job, Step, ItemReader/Processor/Writer, chunk processing, skip handling, best practices, and common pitfalls like memory exhaustion, all illustrated with code examples and diagrams.

Batch ProcessingChunkSkip Strategy
0 likes · 20 min read
Master Spring Batch: From Core Concepts to Advanced Chunk Processing
Java Architect Essentials
Java Architect Essentials
Jul 17, 2022 · Backend Development

Master Spring Batch: Core Concepts, Architecture, and Practical Tips

This article provides a comprehensive guide to Spring Batch, covering its purpose, architecture, core components such as Job, Step, ItemReader/Writer/Processor, chunk processing, skip strategies, common pitfalls, and practical configuration examples for robust batch applications.

Batch ProcessingChunk ProcessingJob Scheduling
0 likes · 18 min read
Master Spring Batch: Core Concepts, Architecture, and Practical Tips
Code Ape Tech Column
Code Ape Tech Column
Jul 16, 2022 · Backend Development

Spring Batch Architecture Overview and Core Concepts

This article introduces Spring Batch as a lightweight, comprehensive batch‑processing framework for enterprise applications, explains its overall architecture, and details core concepts such as Job, JobInstance, JobParameters, JobExecution, Step, StepExecution, ExecutionContext, JobRepository, JobLauncher, ItemReader, ItemWriter, ItemProcessor, chunk processing, skip/failure handling, best‑practice guidelines, and common troubleshooting tips.

Batch ProcessingChunkItemReader
0 likes · 20 min read
Spring Batch Architecture Overview and Core Concepts
Java Backend Technology
Java Backend Technology
Jul 12, 2022 · Backend Development

Boost Java Batch Updates with Multithreading: A Practical Guide

This article explains how to efficiently handle large‑scale batch updates in Java by splitting the data set, using a thread pool for concurrent processing, and providing reusable utility code, all illustrated with a clear workflow diagram and step‑by‑step examples.

BackendBatch Processingjava
0 likes · 7 min read
Boost Java Batch Updates with Multithreading: A Practical Guide
DataFunTalk
DataFunTalk
May 23, 2022 · Big Data

Real-Time Data Lake Practices at ByteDance: Architecture, Challenges, and Solutions

ByteDance shares its real‑time data lake implementation, covering the evolving definition of data lakes, six core capabilities, challenges such as data management, weak concurrent updates, performance, and log ingestion, and detailed solutions including Hudi Metastore Server, bucket indexing, multi‑stage use cases, and future roadmap.

Batch ProcessingHudiReal-time Data Lake
0 likes · 32 min read
Real-Time Data Lake Practices at ByteDance: Architecture, Challenges, and Solutions
Top Architect
Top Architect
May 15, 2022 · Backend Development

Understanding MyBatis Batch Processing: Common Pitfalls and Optimized Solutions

This article walks through the challenges of inserting massive data sets with MyBatis, explains the inner workings of batch execution, highlights common mistakes with commit, clearCache and flushStatements, and presents progressively refined code examples—including Oracle-specific optimizations—to dramatically improve performance.

Batch ProcessingMyBatisjava
0 likes · 12 min read
Understanding MyBatis Batch Processing: Common Pitfalls and Optimized Solutions
IT Architects Alliance
IT Architects Alliance
May 14, 2022 · Backend Development

Request Collapsing Techniques: Hystrix Collapser, Custom BatchCollapser, and ConcurrentHashMultiset

This article compares three request‑collapsing techniques—Hystrix Collapser, a custom BatchCollapser, and Guava’s ConcurrentHashMultiset—detailing their designs, implementations, configurations, and suitable scenarios for reducing downstream load and improving system throughput, including code examples, timer‑based batching, and thread‑safe container usage.

Batch ProcessingGuavaHystrix
0 likes · 14 min read
Request Collapsing Techniques: Hystrix Collapser, Custom BatchCollapser, and ConcurrentHashMultiset
Java High-Performance Architecture
Java High-Performance Architecture
May 12, 2022 · Backend Development

Master MyBatis Batch Processing: Common Pitfalls and Optimized Solutions

This article walks readers through MyBatis batch processing from basics to advanced, highlighting common mistakes, explaining the roles of commit, clearCache, and flushStatements, and presenting three progressively refined code versions plus Oracle-specific optimizations to dramatically improve bulk insert performance.

Batch ProcessingMyBatisbackend-development
0 likes · 12 min read
Master MyBatis Batch Processing: Common Pitfalls and Optimized Solutions
Shopee Tech Team
Shopee Tech Team
Apr 28, 2022 · Big Data

Building Real-Time Data Warehouse with Flink + Hudi at Shopee

Shopee replaced its hourly Hive pipeline with a hybrid Flink‑Hudi real‑time data warehouse that groups Kafka topics, applies lightweight stream ETL, uses partial‑update MOR tables for multi‑stream joins and COW tables for versioned batches, cutting latency from about 90 minutes to 2–30 minutes and halving resource usage.

Apache FlinkApache HudiBatch Processing
0 likes · 20 min read
Building Real-Time Data Warehouse with Flink + Hudi at Shopee
dbaplus Community
dbaplus Community
Apr 11, 2022 · Databases

Why JDBC executeBatch Sends Single Queries and How to Enable True Batch Execution in MySQL

This article explains why MySQL 8.0’s JDBC driver processes executeBatch() calls as separate statements, demonstrates the root cause in the driver’s rewriteBatchedStatements logic, and shows how configuring rewriteBatchedStatements=true (and optionally allowMultiQueries=true) enables genuine batch execution.

Batch ProcessingJDBCallowMultiQueries
0 likes · 6 min read
Why JDBC executeBatch Sends Single Queries and How to Enable True Batch Execution in MySQL
Alibaba Cloud Native
Alibaba Cloud Native
Jan 25, 2022 · Operations

How SchedulerX Enables Cloud‑Native Distributed Task Scheduling and Visual Management

This article explains the core concepts of SchedulerX, a cloud‑native distributed task scheduling platform, covering its resource model, visual control features, distributed batch processing capabilities, practical use‑case scenarios, and step‑by‑step integration with a SpringBoot application.

Batch ProcessingDistributed SystemsSchedulerX
0 likes · 10 min read
How SchedulerX Enables Cloud‑Native Distributed Task Scheduling and Visual Management
Architecture Digest
Architecture Digest
Jan 21, 2022 · Big Data

Building a Real-Time Data Warehouse with Flink: Architecture, Core Concepts, and Practical Implementation

This article explains how to build a unified stream‑batch real‑time data warehouse using FlinkSQL, covering prerequisite knowledge, five core concepts, two implementation approaches, a comparison of traditional versus real‑time architectures, and a comprehensive hands‑on example, illustrated with diagrams.

Batch ProcessingData ArchitectureFlink
0 likes · 6 min read
Building a Real-Time Data Warehouse with Flink: Architecture, Core Concepts, and Practical Implementation
Architecture Digest
Architecture Digest
Jan 20, 2022 · Backend Development

Implementing Scheduled Device Upgrade with Spring Batch and Quartz in Spring Boot

This article explains how to handle a PC‑triggered device upgrade record by using Quartz for timed execution and Spring Batch for bulk processing, detailing Maven dependencies, YAML configuration, service and batch classes, custom reader/writer logic, a processor that calls an upgrade‑dispatch API, and the overall challenges encountered.

Batch ProcessingQuartzScheduler
0 likes · 13 min read
Implementing Scheduled Device Upgrade with Spring Batch and Quartz in Spring Boot
Top Architect
Top Architect
Jan 17, 2022 · Backend Development

Implementing Device Upgrade Batch Processing with Spring Batch and Quartz

This article explains how to use Spring Batch together with Quartz to periodically process device upgrade records, covering Maven dependencies, configuration files, service implementation, batch job and step definitions, a custom processor for HTTP dispatch, entity mapping, and job retry scheduling for failed updates.

Batch ProcessingJob SchedulingQuartz
0 likes · 10 min read
Implementing Device Upgrade Batch Processing with Spring Batch and Quartz
Big Data Technology & Architecture
Big Data Technology & Architecture
Oct 9, 2021 · Big Data

Apache Flink 1.7–1.14 Release Highlights and Feature Evolution

This article provides a comprehensive overview of Apache Flink's major releases from version 1.7 to 1.14, detailing new APIs, state management improvements, Kubernetes integration, SQL and Table API enhancements, checkpointing advances, and performance optimizations that together illustrate the platform's evolution for both streaming and batch processing workloads.

Apache FlinkBatch ProcessingCheckpoint
0 likes · 78 min read
Apache Flink 1.7–1.14 Release Highlights and Feature Evolution
MaGe Linux Operations
MaGe Linux Operations
Sep 30, 2021 · Operations

Boost Your Productivity: Automate Excel with Python and xlwings

This guide demonstrates how to use Python's xlwings library to programmatically create, rename, list, print, and split Excel workbooks and worksheets, enabling batch processing and automation that significantly improves efficiency for repetitive spreadsheet tasks.

Batch ProcessingExcel AutomationScripting
0 likes · 5 min read
Boost Your Productivity: Automate Excel with Python and xlwings
Big Data Technology Architecture
Big Data Technology Architecture
Aug 10, 2021 · Big Data

Building a Real‑Time Data Warehouse with Apache Flink and Apache Iceberg: Architecture, Challenges, and Best Practices

This article presents Tencent's practical experience of constructing a real‑time data warehouse by integrating Apache Flink with Apache Iceberg, covering background pain points of traditional Lambda architectures, Iceberg's table format and capabilities, Flink‑Iceberg sink design, small‑file handling, and future roadmap for a unified streaming‑batch data lake.

Apache FlinkApache IcebergBatch Processing
0 likes · 20 min read
Building a Real‑Time Data Warehouse with Apache Flink and Apache Iceberg: Architecture, Challenges, and Best Practices
Big Data Technology & Architecture
Big Data Technology & Architecture
Jul 30, 2021 · Big Data

Enterprise Big Data Platform Architecture: Insights from Taobao, Meituan, and Didi

This article examines the architecture of enterprise-level big data platforms at leading Chinese tech firms—Taobao, Meituan, and Didi—detailing their data sources, synchronization components, batch and streaming processing layers, scheduling systems, and common design patterns, while highlighting shared principles across these implementations.

Batch ProcessingEnterpriseStreaming
0 likes · 9 min read
Enterprise Big Data Platform Architecture: Insights from Taobao, Meituan, and Didi
macrozheng
macrozheng
Jul 28, 2021 · Backend Development

Master Spring Batch: Core Concepts, Architecture, and Best Practices

This article provides a comprehensive overview of Spring Batch, covering its purpose, architecture, core components such as Job, Step, ItemReader/Writer/Processor, execution contexts, chunk processing, skip strategies, and practical tips for configuration and memory management.

Batch ProcessingChunk ProcessingJob Scheduling
0 likes · 20 min read
Master Spring Batch: Core Concepts, Architecture, and Best Practices
Liangxu Linux
Liangxu Linux
Jul 18, 2021 · Backend Development

Master Bulk File Renaming with F2: Install, Commands, and Tips

F2 is a fast, safe Go‑based command‑line tool for batch renaming files and folders on Linux, macOS, and Windows, offering multiple installation methods, preview mode, undo, conflict resolution, built‑in variables, and powerful regex‑driven rename examples.

Batch ProcessingCLIFile Renaming
0 likes · 7 min read
Master Bulk File Renaming with F2: Install, Commands, and Tips
Architecture Digest
Architecture Digest
Jul 17, 2021 · Backend Development

Introduction to Spring Batch and Its Core Concepts

Spring Batch is a lightweight, comprehensive Java batch processing framework that provides reusable features such as job/step architecture, ItemReader/Writer/Processor, chunk processing, transaction management, and restart capabilities, with detailed explanations of core concepts, configuration examples, and best practices for building robust enterprise batch jobs.

Batch ProcessingChunkJob
0 likes · 19 min read
Introduction to Spring Batch and Its Core Concepts
Top Architect
Top Architect
Jul 13, 2021 · Backend Development

Introduction to Spring Batch and Its Core Concepts

This article provides a comprehensive overview of Spring Batch, covering its purpose, architecture, core components such as Job, Step, ItemReader/Writer/Processor, chunk processing, skip strategies, and practical guidelines for building robust batch processing solutions in Java.

Batch ProcessingSpring BatchSpring Framework
0 likes · 19 min read
Introduction to Spring Batch and Its Core Concepts
Python Programming Learning Circle
Python Programming Learning Circle
Jul 8, 2021 · Operations

Python Office Automation Techniques: Document Conversion, Geocoding, Distance Calculation, Batch File Processing, and Outlook Email Automation

This article presents a collection of Python scripts for office automation, covering Word doc‑to‑docx conversion, batch address geocoding using Baidu API, distance computation with geopy, coordinate conversion between Baidu and Gaode, Excel merging, Word‑to‑PDF conversion, extracting tables from Word, and bulk Outlook email sending.

Batch ProcessingGeocodingOffice Automation
0 likes · 11 min read
Python Office Automation Techniques: Document Conversion, Geocoding, Distance Calculation, Batch File Processing, and Outlook Email Automation
Baidu Geek Talk
Baidu Geek Talk
Jul 7, 2021 · Backend Development

Design and Implementation of Baidu's Commodity Promotion System

The article details Baidu’s 2020‑built commodity promotion system for Baijiahao and live‑stream e‑commerce, linking merchants with authors/streamers through CPS billing, three user interfaces and five core services, and highlights technical choices such as dynamic‑library tracking, asynchronous batch writes, and a high‑cohesion, low‑coupling architecture requiring cross‑team collaboration.

Baidu ecosystemBatch ProcessingCPS
0 likes · 15 min read
Design and Implementation of Baidu's Commodity Promotion System
Programmer DD
Programmer DD
Jul 5, 2021 · Backend Development

Master Spring Batch: Core Concepts, Architecture, and Best Practices

This comprehensive guide explains Spring Batch's purpose, core components such as Job, Step, ItemReader/Writer/Processor, execution flow, chunk processing, skip strategies, and practical tips for configuration, performance tuning, and troubleshooting in enterprise Java batch applications.

Batch ProcessingChunkJob
0 likes · 19 min read
Master Spring Batch: Core Concepts, Architecture, and Best Practices
Code Ape Tech Column
Code Ape Tech Column
Jul 1, 2021 · Backend Development

Master Spring Batch: Core Concepts, Architecture, and Practical Tips

This article provides a comprehensive guide to Spring Batch, covering its purpose, architecture, core components such as Job, Step, ItemReader/Writer/Processor, chunk processing, skip strategies, configuration tips, and common memory issues, all illustrated with code examples and diagrams.

Batch ProcessingChunk ProcessingData Integration
0 likes · 19 min read
Master Spring Batch: Core Concepts, Architecture, and Practical Tips
Python Programming Learning Circle
Python Programming Learning Circle
Jun 25, 2021 · Artificial Intelligence

Batch Image Translation Demo Using Youdao OCR API with Python

This article presents a step‑by‑step Python demo that uses Youdao's OCR translation API to batch‑process cosmetic product images, covering API key setup, request parameters, signature generation, GUI implementation with Tkinter, and code snippets for file selection, result storage, and API invocation.

AIBatch ProcessingOCR
0 likes · 10 min read
Batch Image Translation Demo Using Youdao OCR API with Python
DataFunTalk
DataFunTalk
Apr 23, 2021 · Big Data

Building and Evolving Zhihu’s Flink‑Based Data Integration Platform

This article details Zhihu’s transition from a Sqoop‑driven data integration system to a Flink‑centric platform, covering business scenarios, historical architecture, design goals, technology choices, performance optimizations, and future plans for unified streaming‑batch processing across diverse storage systems.

Batch ProcessingBig DataData Integration
0 likes · 14 min read
Building and Evolving Zhihu’s Flink‑Based Data Integration Platform
Kuaishou Tech
Kuaishou Tech
Apr 9, 2021 · Backend Development

Design and Implementation of Red Packet Calculation and Distribution System for Spring Festival Activity

This article details the design of a red packet calculation and distribution system for a Spring Festival promotion, addressing mixed-type packet computation, seamless transition to awarding, distributed task processing, stability guarantees, and successful implementation results.

Batch ProcessingSpring Festivaldistributed computing
0 likes · 12 min read
Design and Implementation of Red Packet Calculation and Distribution System for Spring Festival Activity
DataFunTalk
DataFunTalk
Mar 28, 2021 · Big Data

Flink Stream‑Batch Integration: Layered Architecture, Unified SDK, DAG Scheduler, Shuffle, and Fault‑Tolerance

This article explains how Apache Flink has evolved into a unified stream‑batch engine by introducing a three‑layer architecture, a unified DataStream SDK, a pipeline‑region‑based DAG scheduler, a common shuffle framework, and enhanced fault‑tolerance mechanisms to address efficiency, consistency, and resource‑utilisation challenges in real‑time big‑data processing.

Apache FlinkBatch ProcessingDAG scheduler
0 likes · 25 min read
Flink Stream‑Batch Integration: Layered Architecture, Unified SDK, DAG Scheduler, Shuffle, and Fault‑Tolerance
DataFunSummit
DataFunSummit
Nov 17, 2020 · Big Data

Sohu Intelligent Media Data Warehouse Architecture and Technical Practices

This article presents Sohu Intelligent Media's data warehouse construction practice, covering fundamental concepts, batch and real‑time processing, OLAP theory, multidimensional modeling, workflow management, data quality, metadata lineage, and security, with a focus on Apache Doris and a Lambda‑style architecture.

Apache DorisBatch ProcessingData Quality
0 likes · 18 min read
Sohu Intelligent Media Data Warehouse Architecture and Technical Practices
DataFunTalk
DataFunTalk
Nov 17, 2020 · Artificial Intelligence

Alink: A Flink‑Based Machine Learning Platform – Overview, Features, and Quick‑Start Guide

This article introduces Alink, Alibaba's open‑source machine‑learning platform built on Flink, explains its core algorithms, performance comparison with Spark ML, version‑wise feature evolution, and provides practical quick‑start instructions for both Java (Maven) and Python (PyAlink) users, including data source handling, type conversion components, unified file‑system operations, and an overview of its FM algorithm implementation.

AlinkBatch ProcessingData Integration
0 likes · 13 min read
Alink: A Flink‑Based Machine Learning Platform – Overview, Features, and Quick‑Start Guide
Architect
Architect
May 30, 2020 · Big Data

Understanding Flink’s Unified Programming API for Batch and Streaming Jobs

This article examines Apache Flink’s programming model, comparing its batch DataSet API with the streaming DataStream API, detailing class hierarchies, key code examples such as groupBy and job submission, and explaining how both paradigms are unified into a common JobGraph representation.

Batch ProcessingBig DataFlink
0 likes · 9 min read
Understanding Flink’s Unified Programming API for Batch and Streaming Jobs
21CTO
21CTO
May 20, 2020 · Artificial Intelligence

Batch Remove Image Backgrounds with Python and PaddlePaddle

This tutorial shows how to install PaddlePaddle and PaddleHub, then use a concise Python script to batch‑remove backgrounds from multiple photos, automatically saving the segmented results, and includes sample code and before‑after images to illustrate the process.

Batch ProcessingPaddlePaddlePython
0 likes · 5 min read
Batch Remove Image Backgrounds with Python and PaddlePaddle
ITPUB
ITPUB
May 15, 2020 · Artificial Intelligence

Batch Background Removal with PaddlePaddle: One-Click Python Script

This guide shows how to install PaddlePaddle and PaddleHub, then use a short Python script to automatically segment and remove backgrounds from multiple photos, turning each into a transparent PNG with just a few lines of code.

Batch ProcessingPaddlePaddleimage segmentation
0 likes · 5 min read
Batch Background Removal with PaddlePaddle: One-Click Python Script
Python Programming Learning Circle
Python Programming Learning Circle
May 12, 2020 · Artificial Intelligence

Batch Image Segmentation with Python and PaddlePaddle

This tutorial demonstrates how to use Python and the PaddlePaddle deep‑learning platform to automatically remove backgrounds from multiple photos in one step, covering installation, verification, and a concise five‑line code example for batch human segmentation.

Batch ProcessingComputer VisionDeep Learning
0 likes · 6 min read
Batch Image Segmentation with Python and PaddlePaddle
Programmer DD
Programmer DD
Mar 28, 2020 · Backend Development

Why Is Kafka So Fast? Uncover the 11 Performance Secrets

Kafka achieves its remarkable speed by combining sequential I/O, batch processing, compression, zero‑copy, careful client‑side work, and a design that avoids costly fsync and garbage collection, while maintaining durability, ordering, and at‑least‑once delivery, making it a high‑throughput, low‑latency event streaming platform.

Batch ProcessingDistributed SystemsKafka
0 likes · 15 min read
Why Is Kafka So Fast? Uncover the 11 Performance Secrets
Liangxu Linux
Liangxu Linux
Mar 28, 2020 · Fundamentals

Master Linux File Renaming with mv and rename: Simple and Complex Techniques

This guide explains how to rename files on Linux using the mv and rename commands, covering basic single‑file renaming, batch renaming with patterns, handling different OS syntaxes, and advanced strategies for generating name mappings from custom files or filename patterns.

Batch ProcessingShell scriptingawk
0 likes · 11 min read
Master Linux File Renaming with mv and rename: Simple and Complex Techniques
Alibaba Cloud Developer
Alibaba Cloud Developer
Feb 24, 2020 · Big Data

What’s New in Apache Flink 1.10? Deep Dive into Major Features and Enhancements

Apache Flink 1.10 introduces a major upgrade that merges the Blink engine, boosts performance and stability, adds native Kubernetes support, enhances SQL DDL, delivers production‑ready Hive batch compatibility, optimizes memory management, and expands Python UDF capabilities, with detailed feature breakdowns and code examples.

Apache FlinkBatch ProcessingKubernetes
0 likes · 8 min read
What’s New in Apache Flink 1.10? Deep Dive into Major Features and Enhancements
Programmer DD
Programmer DD
Nov 4, 2019 · Backend Development

Master Efficient Excel Imports in Java with EasyExcel and Spring Boot

This article explains how Java backend developers can replace memory‑heavy Apache POI with EasyExcel for fast, low‑memory Excel imports, covering basic reading, custom converters for gender and dates, and multiple Spring Boot integration patterns including controllers, listeners, constructor injection and lambda‑based approaches.

Batch ProcessingExcelSpring Boot
0 likes · 12 min read
Master Efficient Excel Imports in Java with EasyExcel and Spring Boot