distributed tasks

Test Development Learning Exchange

Oct 20, 2024 · Backend Development

Python Examples of Distributed Task Queues, Message Brokers, RPC, and Serialization Libraries

This article provides practical Python code examples for various messaging and serialization tools—including Celery, RQ, Huey, ZeroMQ, kafka‑python, Pika, stomp.py, nats‑py, gRPC, Thrift, Protobuf, Avro, msgpack, and Flatbuffers—demonstrating how to set up producers, consumers, and services for asynchronous processing and data exchange.

Message QueuePythonRPC

0 likes · 19 min read

Python Examples of Distributed Task Queues, Message Brokers, RPC, and Serialization Libraries

Selected Java Interview Questions

Apr 8, 2024 · Databases

Designing High‑Performance Import of 1 Billion Records into MySQL

This article analyzes how to import one billion 1 KB log records stored in HDFS or S3 into MySQL efficiently, covering constraints, single‑table limits, batch insertion, storage‑engine choices, sharding, file‑reading techniques, task reliability, and concurrency control to achieve optimal throughput.

MySQLbatch insertdistributed tasks

0 likes · 19 min read

Designing High‑Performance Import of 1 Billion Records into MySQL

Test Development Learning Exchange

Dec 6, 2023 · Backend Development

Using Python multiprocessing and Celery for Parallel and Distributed Task Processing

This article introduces Python's multiprocessing module and the Celery task queue, explains their core concepts, and provides practical code examples for multi‑process parallel computation, inter‑process communication, asynchronous execution, scheduled jobs, result callbacks, retries, and distributed task orchestration.

CeleryParallel ComputingPython

0 likes · 7 min read

Using Python multiprocessing and Celery for Parallel and Distributed Task Processing

Selected Java Interview Questions

Oct 31, 2023 · Databases

Designing High‑Performance Import of 1 Billion Records into MySQL

To import one billion 1 KB unstructured log entries stored in HDFS or S3 into MySQL efficiently, the article discusses constraints, B+‑tree limits, batch insertion, storage engine choices, sharding, task coordination, concurrency control, progress tracking with Redis, and reliable distributed execution strategies.

MySQLbatch insertdata-import

0 likes · 19 min read

Code Ape Tech Column

Oct 18, 2023 · Databases

Efficient Strategies for Importing One Billion Records into MySQL

This article explains how to import 1 billion 1 KB log records stored in HDFS or S3 into MySQL by analyzing table capacity limits, choosing storage engines, designing batch inserts, coordinating file reading and writing, and handling task reliability with Redis, Redisson, and Zookeeper.

MySQLRedisRedisson

0 likes · 18 min read

Efficient Strategies for Importing One Billion Records into MySQL

Test Development Learning Exchange

Aug 2, 2023 · Backend Development

Practical Examples of Python multiprocessing and Celery for Parallel and Distributed Task Processing

This article introduces Python's multiprocessing module and the Celery distributed task queue, explains their core features, and provides ten practical code examples demonstrating multi‑process parallel computation, inter‑process communication, asynchronous tasks, scheduling, retries, and distributed processing for real‑world applications.

CeleryParallel ComputingPython

0 likes · 7 min read

Practical Examples of Python multiprocessing and Celery for Parallel and Distributed Task Processing

Code Ape Tech Column

Nov 27, 2021 · Backend Development

A Comprehensive Guide to 10 Practical Scheduling Solutions for Java and Linux

This article reviews ten widely used scheduling methods—from Linux crontab and Java Thread/Timer to Spring Task, Spring Quartz, and distributed frameworks like XXL‑Job and Elastic‑Job—explaining their configuration, code examples, and advantages and disadvantages for various backend scenarios.

CronJavaScheduling

0 likes · 22 min read

A Comprehensive Guide to 10 Practical Scheduling Solutions for Java and Linux

Python Programming Learning Circle

Oct 23, 2021 · Backend Development

Comprehensive Guide to Using Celery for Distributed Task Processing in Python

This article provides a detailed overview of Celery, covering its architecture, common use cases, installation steps, project structure, task definition, periodic tasks, worker management, distributed deployment, advanced monitoring, custom task handling, task routing, and monitoring with Flower, all illustrated with practical code examples.

CeleryPythonRedis

0 likes · 12 min read

Comprehensive Guide to Using Celery for Distributed Task Processing in Python

58 Tech

Nov 13, 2019 · Backend Development

Using Bull Queue in Node.js to Handle Asynchronous Calls, Traffic Shaping, and Distributed Scheduled Tasks for Frontend Applications

This article explains how front‑end teams can leverage the Node.js Bull queue to implement lightweight asynchronous calls, rate‑limiting traffic spikes, and distributed scheduled jobs, detailing the selection rationale, architectural changes, core Redis‑based mechanisms, and practical deployment tips.

BullFrontend InfrastructureNode.js

0 likes · 9 min read

Using Bull Queue in Node.js to Handle Asynchronous Calls, Traffic Shaping, and Distributed Scheduled Tasks for Frontend Applications

Zhuanzhuan Tech

Dec 28, 2018 · Backend Development

Evolution of ZhaiZhai Transaction System: From Monolithic Design to FSM‑Based Modular Architecture

This article chronicles the rapid growth of ZhaiZhai's second‑hand trading platform and explains how its transaction core was refactored from a single‑service monolith into a unified, configurable, FSM‑driven architecture that isolates business lines, abstracts common logic, and improves scalability and maintainability.

State MachineTransaction Systembackend design

0 likes · 12 min read

Evolution of ZhaiZhai Transaction System: From Monolithic Design to FSM‑Based Modular Architecture

Qunar Tech Salon

Oct 15, 2015 · Backend Development

Getting Started with Celery: Architecture, Installation, and Basic Usage

This article introduces Celery, a Python‑based distributed task queue, explains its three‑part architecture, shows how to install RabbitMQ and Celery on an AWS EC2 Linux instance, and provides step‑by‑step code examples for defining tasks, running workers, and retrieving asynchronous results.

Backend DevelopmentCeleryMessage Broker

0 likes · 6 min read

Getting Started with Celery: Architecture, Installation, and Basic Usage