Implementing Distributed Task Scheduling and Dispatch in PHP Microservices with Redis and RabbitMQ

With the rapid growth of the Internet, system scale and complexity increase, raising the demand for efficient task scheduling and dispatch; distributed task scheduling offers a solution by allocating tasks across multiple nodes to improve efficiency and fault tolerance.

Using Redis to Implement Distributed Task Scheduling and Dispatch

Redis, a high‑performance key‑value store, can act as a task queue. By using Redis lists, tasks are stored as JSON strings and multiple nodes subscribe to the list to retrieve and process tasks.

1.1. Create a Task Queue

Push tasks onto a Redis list using the lpush command:

$taskData = json_encode(['task_id' => 1, 'task_data' => 'task content']);
$redis->lpush('task_queue', $taskData);

1.2. Multiple Nodes Subscribe to the Task Queue

Nodes subscribe to a Redis channel; when a new task is added, Redis pushes a message to subscribers:

$redis->subscribe(['task_channel'], function ($redis, $channel, $message) {
    // Retrieve task and process
    $taskData = json_decode($message, true);
    $taskId = $taskData['task_id'];
    $taskContent = $taskData['task_data'];
    // Execute task handling logic...
});

Using RabbitMQ to Implement Distributed Task Scheduling and Dispatch

RabbitMQ, an open‑source message broker, enables efficient task scheduling via queues and a publish/subscribe model.

2.1. Create a Task Queue

Publish task messages to a RabbitMQ queue using the AMQP library:

$taskData = json_encode(['task_id' => 1, 'task_data' => 'task content']);

$connection = new AMQPConnection('localhost', 5672, 'guest', 'guest');
$channel = $connection->channel();

$channel->queue_declare('task_queue', false, true, false, false);

$msg = new AMQPMessage($taskData, ['delivery_mode' => AMQPMessage::DELIVERY_MODE_PERSISTENT]);
$channel->basic_publish($msg, '', 'task_queue');

$channel->close();
$connection->close();

2.2. Multiple Nodes Consume the Task Queue

Consumers retrieve tasks from the queue; RabbitMQ automatically distributes tasks to available consumers:

$connection = new AMQPConnection('localhost', 5672, 'guest', 'guest');
$channel = $connection->channel();

$callback = function ($msg) {
    // Retrieve task and process
    $taskData = json_decode($msg->body, true);
    $taskId = $taskData['task_id'];
    $taskContent = $taskData['task_data'];
    // Execute task handling logic...
    $msg->delivery_info['channel']->basic_ack($msg->delivery_info['delivery_tag']);
};

$channel->basic_qos(null, 1, null);
$channel->basic_consume('task_queue', '', false, false, false, false, $callback);

while (count($channel->callbacks)) {
    $channel->wait();
}

$channel->close();
$connection->close();

The examples demonstrate how to use Redis and RabbitMQ in PHP microservices to achieve distributed task distribution and processing, enhancing system flexibility and scalability.