BestHub
Sign in
Tagged articles
224 articles
Page 3 of 3
Tencent Cloud Developer
Tencent Cloud Developer
Aug 1, 2019 · Databases

FeatureKV: A High-Performance Key-Value Storage System for WeChat's Billion-Scale Challenges

FeatureKV, WeChat’s high‑performance key‑value store, handles one‑billion queries per second and ingests a billion keys per hour by separating write‑only DataSvr from read‑only KVSvr, supporting in‑memory, indexed, and block‑indexed tables, scaling horizontally, guaranteeing eventual consistency with versioned reads, and delivering up to 11 billion reads per second with sub‑15 ms latency.

Batch ProcessingFeatureKVKey-Value
0 likes · 22 min read
FeatureKV: A High-Performance Key-Value Storage System for WeChat's Billion-Scale Challenges
Programmer DD
Programmer DD
Jul 14, 2019 · Backend Development

Step‑by‑Step Spring Batch Hello World Job with Spring Boot

This tutorial walks you through building a simple Spring Batch Hello World job using Spring Boot and Maven, covering the framework basics, project setup, Maven configuration, entity modeling, job and step configuration, data processing, testing, and how to run the example to generate greeting output.

Batch ProcessingSpring Batch
0 likes · 19 min read
Step‑by‑Step Spring Batch Hello World Job with Spring Boot
Qunar Tech Salon
Qunar Tech Salon
Jul 5, 2019 · Big Data

Understanding Big Data Processing Architectures: Lambda, Kappa, and Lambda Plus

This article explains the technical challenges of large‑scale data processing, compares the classic Lambda and Kappa architectures, and introduces the cloud‑native Lambda Plus solution built on TableStore and Blink that simplifies batch‑stream integration for TB‑scale workloads.

Batch ProcessingCloud ServicesKappa architecture
0 likes · 13 min read
Understanding Big Data Processing Architectures: Lambda, Kappa, and Lambda Plus
Programmer DD
Programmer DD
Jun 16, 2019 · Backend Development

From Two‑Month Crawl to Four‑Hour Sprint: Optimizing a 20M‑Record Data Migration

This article chronicles a step‑by‑step performance overhaul of a 20‑million‑record migration project, detailing four architectural revisions—from a single‑threaded procedural script to a fully decoupled, multithreaded, interface‑driven solution—that reduced processing time from two months to just four hours while ensuring data consistency, recoverability, and scalability.

Batch ProcessingData MigrationSpring Boot
0 likes · 15 min read
From Two‑Month Crawl to Four‑Hour Sprint: Optimizing a 20M‑Record Data Migration
Big Data Technology & Architecture
Big Data Technology & Architecture
Mar 12, 2019 · Big Data

Understanding Apache Flink’s Core Design: “Batch Is a Special Case of Stream” and Its Architecture

This article explains Apache Flink’s fundamental design principle that treats batch as a special case of stream, compares native streaming with micro‑batching, describes its deployment modes, fault‑tolerance mechanisms, unified data and scheduling layers, and outlines Alibaba’s architectural optimizations for the platform.

Apache FlinkBatch Processingnative streaming
0 likes · 15 min read
Understanding Apache Flink’s Core Design: “Batch Is a Special Case of Stream” and Its Architecture
Zhuanzhuan Tech
Zhuanzhuan Tech
Jan 11, 2019 · Databases

Differences Between TiDB and MySQL: Transactions, Queries, Server‑Side Prepared Statements, and Batch Processing

This article examines TiDB, a world‑class open‑source distributed NewSQL database, comparing its transaction and query behavior with MySQL, discussing underlying Percolator model, server‑side prepared statements, batch processing techniques, and practical optimization strategies for developers.

Batch ProcessingTiDBdistributed database
0 likes · 10 min read
Differences Between TiDB and MySQL: Transactions, Queries, Server‑Side Prepared Statements, and Batch Processing
Liangxu Linux
Liangxu Linux
Nov 29, 2018 · Fundamentals

Three Simple Linux Commands to Batch Rename Files Efficiently

Learn how to quickly rename multiple files in Linux using the rename utility, a bash for‑loop with mv, and a sed‑based loop, each illustrated with clear command examples and explanations of the underlying string manipulation techniques.

BashBatch ProcessingFile Renaming
0 likes · 5 min read
Three Simple Linux Commands to Batch Rename Files Efficiently
Java Backend Technology
Java Backend Technology
Jul 29, 2018 · Backend Development

Turning a 2‑Month Data Migration into 4 Hours: Backend Performance Secrets

After struggling with a two‑month data migration that processed 20 million users, the author iteratively redesigned the system—from a single‑threaded, procedural approach to a fully decoupled, multithreaded architecture using queues and batch operations—ultimately reducing runtime to just four hours while ensuring data consistency and recoverability.

Batch ProcessingData Migrationmultithreading
0 likes · 13 min read
Turning a 2‑Month Data Migration into 4 Hours: Backend Performance Secrets
ITPUB
ITPUB
Jun 19, 2018 · Big Data

Is Hadoop Still Relevant? Comparing Hadoop, PostgreSQL, and Storm

The article examines Hadoop's relevance by contrasting it with PostgreSQL and Storm, discussing when each technology fits big‑data challenges such as volume, velocity, and variety, and highlighting cost, complexity, and use‑case considerations for enterprises.

Batch ProcessingHadoopStorm
0 likes · 8 min read
Is Hadoop Still Relevant? Comparing Hadoop, PostgreSQL, and Storm
JD Tech
JD Tech
May 31, 2018 · Backend Development

Design and Architecture of a Unified MySQL Data Synchronization Platform

This article details the design of a unified MySQL data synchronization platform that consolidates offline sync, real‑time subscription, and real‑time sync into BatchJob, StreamJob, and PieJob abstractions, describing task implementations, cluster architecture, high‑availability mechanisms, and evolution challenges such as file loss and metadata handling.

Backend ArchitectureBatch Processingdata synchronization
0 likes · 10 min read
Design and Architecture of a Unified MySQL Data Synchronization Platform
Tencent Cloud Developer
Tencent Cloud Developer
Nov 17, 2017 · Industry Insights

How Cloud Computing Is Bringing High‑Performance Computing to Everyday Enterprises

This article examines the evolution of high‑performance computing from exclusive national supercomputers to accessible cloud‑based services, outlines the three adoption stages for enterprises, and details Tencent Cloud's Batch platform features that simplify large‑scale HPC workloads.

Batch ProcessingHPC as a serviceHigh‑performance computing
0 likes · 11 min read
How Cloud Computing Is Bringing High‑Performance Computing to Everyday Enterprises
Ctrip Technology
Ctrip Technology
Jan 25, 2017 · Backend Development

Handling Duplicate Messages, Ordering, Concurrency, and Batch Processing in Message‑Driven Systems

This article shares practical patterns and built‑in mechanisms for dealing with duplicate messages, message ordering, concurrent updates, asynchronous acknowledgments, and batch processing in a large‑scale, message‑driven architecture, illustrated with QMQ examples from Qunar's platform.

Batch ProcessingConcurrency ControlDuplicate Message Handling
0 likes · 16 min read
Handling Duplicate Messages, Ordering, Concurrency, and Batch Processing in Message‑Driven Systems
Qunar Tech Salon
Qunar Tech Salon
Jan 21, 2017 · Backend Development

Message Consumption Patterns and Best Practices in Qunar's QMQ

This article shares Qunar's practical experiences with message-driven architecture, detailing consumer handling of duplicate messages, ordering, concurrency control, asynchronous processing, and batch strategies, and presents concrete solutions such as idempotent checks, deduplication tables, versioning, and QMQ's built‑in executors.

Batch Processingasynchronous ackconcurrency
0 likes · 18 min read
Message Consumption Patterns and Best Practices in Qunar's QMQ
StarRing Big Data Open Lab
StarRing Big Data Open Lab
Oct 10, 2016 · Big Data

Mastering Lambda Architecture: Real‑Time & Batch Processing for Smart Traffic

This article explains the principles of Lambda Architecture, its three‑layer design for combining batch and real‑time analytics, and demonstrates a detailed smart‑traffic case study with component selection, capacity planning, and implementation guidance for building scalable big‑data systems.

Batch ProcessingLambda architectureSmart Traffic
0 likes · 15 min read
Mastering Lambda Architecture: Real‑Time & Batch Processing for Smart Traffic
ITPUB
ITPUB
Sep 7, 2016 · Databases

Optimizing and Migrating a 20 Million‑Row MySQL Table

This guide explains how to clean redundant fields, restructure a massive MySQL table, batch‑fetch data with LIMIT, rewrite slow queries, and compare three insertion methods—including prepared statements and bulk inserts—to cut migration time from hours to under 20 minutes.

Batch ProcessingData MigrationSQL Optimization
0 likes · 7 min read
Optimizing and Migrating a 20 Million‑Row MySQL Table
dbaplus Community
dbaplus Community
Aug 10, 2016 · Databases

How Oracle 12c Solves Batch‑Load Statistics Problems with Online Gathering

This article explains the classic Oracle batch‑processing issue caused by inaccurate statistics on tables with sharply changing row counts, reviews traditional mitigation methods and their drawbacks, and introduces Oracle 12c's online statistics gathering for bulk‑load as an automated solution while noting its current limitations.

12cBatch ProcessingDatabase Optimization
0 likes · 9 min read
How Oracle 12c Solves Batch‑Load Statistics Problems with Online Gathering
Meituan Technology Team
Meituan Technology Team
Apr 29, 2016 · Big Data

Introduction to Spark in Big Data

Apache Spark, a versatile big‑data platform supporting batch processing, SQL queries, real‑time streaming, and machine‑learning workloads, dramatically accelerates data‑intensive jobs, as demonstrated by Meituan‑Dianping, where its high‑performance engine reduces execution times and enhances scalability across diverse analytical and operational pipelines.

Batch ProcessingBig DataSpark
0 likes · 1 min read
Introduction to Spark in Big Data
Qunar Tech Salon
Qunar Tech Salon
Jul 28, 2015 · Fundamentals

Introduction to xargs with Basic and Advanced Usage Examples

This article explains the purpose of the Unix xargs command, demonstrates basic usage for concatenating log files, shows an advanced example for renaming text files to log files using the -I placeholder, and provides a step‑by‑step breakdown of how the command pipeline works.

Batch ProcessingScriptingShell
0 likes · 3 min read
Introduction to xargs with Basic and Advanced Usage Examples