This guide explains how to install the Gearman job server and its PHP extension on Linux, start the daemon, and build a simple PHP worker and client that perform a string‑reversal task, illustrating the basics of distributed task processing with Gearman.
This article provides practical Python code examples for various messaging and serialization tools—including Celery, RQ, Huey, ZeroMQ, kafka‑python, Pika, stomp.py, nats‑py, gRPC, Thrift, Protobuf, Avro, msgpack, and Flatbuffers—demonstrating how to set up producers, consumers, and services for asynchronous processing and data exchange.
This article analyzes how to import one billion 1 KB log records stored in HDFS or S3 into MySQL efficiently, covering constraints, single‑table limits, batch insertion, storage‑engine choices, sharding, file‑reading techniques, task reliability, and concurrency control to achieve optimal throughput.
This guide explains how to efficiently load one billion 1‑KB log entries from HDFS or S3 into MySQL by analyzing B‑tree limits, using batch inserts, choosing the right storage engine, sharding tables, optimizing file reading, and coordinating tasks with Redis, Redisson, and Zookeeper.
This article introduces Python's multiprocessing module and the Celery task queue, explains their core concepts, and provides practical code examples for multi‑process parallel computation, inter‑process communication, asynchronous execution, scheduled jobs, result callbacks, retries, and distributed task orchestration.
This article walks through the full engineering process of importing a billion 1 KB log records into MySQL, covering data size calculations, B‑tree index limits, batch‑insert strategies, storage‑engine trade‑offs, fast file‑reading techniques, task coordination with Redis and Redisson, and distributed‑lock design using Zookeeper.
This article explains how to import 1 billion 1 KB log records stored in HDFS or S3 into MySQL by analyzing table capacity limits, choosing storage engines, designing batch inserts, coordinating file reading and writing, and handling task reliability with Redis, Redisson, and Zookeeper.
This article introduces Python's multiprocessing module and the Celery distributed task queue, explains their core features, and provides ten practical code examples demonstrating multi‑process parallel computation, inter‑process communication, asynchronous tasks, scheduling, retries, and distributed processing for real‑world applications.
This article details how a large‑scale project environment platform was refactored using domain‑driven design, distributed sharding, thread‑pool parallelism, second‑level scheduling, and distributed locks to achieve over 99% creation success, reduce creation time below 100 seconds, and keep exception rates under 1% despite massive task volume.
This article reviews ten widely used scheduling methods—from Linux crontab and Java Thread/Timer to Spring Task, Spring Quartz, and distributed frameworks like XXL‑Job and Elastic‑Job—explaining their configuration, code examples, and advantages and disadvantages for various backend scenarios.
This article provides a detailed overview of Celery, covering its architecture, common use cases, installation steps, project structure, task definition, periodic tasks, worker management, distributed deployment, advanced monitoring, custom task handling, task routing, and monitoring with Flower, all illustrated with practical code examples.
This article explains how front‑end teams can leverage the Node.js Bull queue to implement lightweight asynchronous calls, rate‑limiting traffic spikes, and distributed scheduled jobs, detailing the selection rationale, architectural changes, core Redis‑based mechanisms, and practical deployment tips.
This article chronicles the rapid growth of ZhaiZhai's second‑hand trading platform and explains how its transaction core was refactored from a single‑service monolith into a unified, configurable, FSM‑driven architecture that isolates business lines, abstracts common logic, and improves scalability and maintainability.
This article introduces Celery, a Python‑based distributed task queue, explains its three‑part architecture, shows how to install RabbitMQ and Celery on an AWS EC2 Linux instance, and provides step‑by‑step code examples for defining tasks, running workers, and retrieving asynchronous results.
