This article walks Python developers through designing high‑concurrency back‑end systems, covering architectural planning, modular project layout, async I/O with asyncio and FastAPI, load balancing with Gunicorn, database scaling, Celery task queues, caching, rate limiting, monitoring, and graceful shutdown techniques.
This article walks through practical techniques for designing a highly concurrent Python backend that stays stable under load, covering architecture planning, async programming, load balancing, database scaling, distributed tasks, caching, rate limiting, monitoring, and graceful shutdown.
This article compares Ray Serve and Celery, explaining their design philosophies, scaling models, GPU‑aware scheduling, operational trade‑offs, and real‑world case studies to help engineers choose the right tool for high‑throughput online inference or large‑scale batch processing.
This guide walks you through setting up a Python 3.8/Django 2.2 vulnerability scanner that uses Celery for asynchronous port and background scanning, integrates the AWVS API for vulnerability detection, and requires MySQL and Redis configuration, with detailed deployment steps and troubleshooting tips.
FastAPI’s async model excels with I/O‑bound tasks but can be crippled by CPU‑heavy or blocking code; this guide explains why, compares built‑in BackgroundTasks and run_in_executor, and shows when to offload work to dedicated queues like Celery or Dramatiq for reliable, scalable processing.
This guide walks through setting up RabbitMQ via Docker, configuring Celery with FastAPI, defining asynchronous tasks, launching workers, testing endpoints, and monitoring execution with Flower, providing a complete, step‑by‑step solution for background processing in Python web applications.
This article provides a comprehensive guide to five methods for running Python scripts in the background on Linux, covering nohup with &, screen sessions, systemd services, Celery task queues, and supervisord, including command examples and configuration steps.
This article introduces Python's multiprocessing module and the Celery task queue, explains their core concepts, and provides practical code examples for multi‑process parallel computation, inter‑process communication, asynchronous execution, scheduled jobs, result callbacks, retries, and distributed task orchestration.
This article explains how to use Celery together with Redis to manage and execute periodic and asynchronous tasks in Python, covering basic concepts, architecture, configuration steps, single‑worker and multi‑worker setups, distributed processing strategies, and practical considerations for reliable task execution.
This article reviews multiple ways to implement periodic tasks in Python, covering simple loops with sleep, libraries such as Timeloop, threading.Timer, sched, schedule, the APScheduler framework, as well as distributed solutions like Celery and Apache Airflow, and provides code examples for each method.
This article introduces Python's multiprocessing module and the Celery distributed task queue, explains their core features, and provides ten practical code examples demonstrating multi‑process parallel computation, inter‑process communication, asynchronous tasks, scheduling, retries, and distributed processing for real‑world applications.
The paper presents a scalable sFlow‑based traffic analysis system that combines high‑performance agents, collectors, and analyzers—extending Elastiflow with sFlowtool, Logstash, Kafka, and Elasticsearch/Kibana, while adding CMDB integration, Druid storage, and Celery stream processing to achieve sub‑30‑second latency for data‑center monitoring, anomaly detection, and IP‑level analytics, and discusses future needs for broader protocol support and adaptive collection.
This article reviews several Python approaches for creating scheduled or periodic jobs—including a simple while‑True loop with sleep, the Timeloop library, the built‑in sched module, the schedule package, APScheduler, Celery, and Apache Airflow—explaining their usage, advantages, limitations, and providing ready‑to‑run code samples.
This guide explains how to set up Celery, a distributed asynchronous task framework, to use Redis both as the message broker and result backend, covering project structure, configuration code, task definition, execution, state monitoring, and common control operations.
This article provides a detailed overview of Celery, covering its architecture, common use cases, installation steps, project structure, task definition, periodic tasks, worker management, distributed deployment, advanced monitoring, custom task handling, task routing, and monitoring with Flower, all illustrated with practical code examples.
This comprehensive guide explores ten Python techniques for implementing periodic tasks—from simple while‑loop sleeps and the Timeloop library to advanced frameworks like APScheduler, Celery, and Apache Airflow—providing code samples, advantages, limitations, and architectural insights for reliable scheduling.
This article provides a comprehensive overview of various Python approaches for implementing periodic tasks, including simple loops with sleep, third‑party libraries such as Timeloop, schedule, APScheduler, as well as distributed solutions like Celery and Apache Airflow, complete with code examples and architectural explanations.
This article presents a comprehensive guide to implementing periodic tasks in Python, covering simple while‑loop with sleep, Timeloop, threading.Timer, the built‑in sched module, the schedule library, APScheduler, Celery, and Apache Airflow, with code examples and practical tips.
This guide walks through installing the Python Elasticsearch client, building a reusable class with CRUD methods, importing data from MongoDB, writing a simple Baidu Baike crawler, and scaling the workflow with Celery and Flask for a complete search‑engine pipeline.
The article presents DuLab, a distributed UI‑automation platform for mobile apps that leverages Airtest, Python/Tornado, Celery, and Redis to schedule and run bulk test cases on remote devices, detailing its layered case management, package database, mock services, and the resulting scalable, efficient regression testing across diverse devices.
This article describes Qunar's DBA department automated migration platform, detailing its motivation, overall architecture, task pool and templates, customizable web‑based workflow, scheduling, execution queue, tooling, and the significant efficiency and reliability gains achieved across numerous database migration scenarios.
This guide explains how to configure Celery to use Redis as both broker and result backend, covering project layout, code setup, task definition, execution flow, task states, and common control operations for asynchronous Python workloads.
This guide explains how to install the Elasticsearch Python client, build a wrapper class for index management and CRUD operations, import data from MongoDB, use a Celery‑based crawler to harvest Baidu Baike content, and expose search functionality through Flask or other Python web frameworks.
This guide explains how to set up DjCelery—combining Django and Celery—to create, schedule, and manage automated testing tasks such as single‑interface, API, App, and Web UI scripts, replacing Jenkins‑based cron jobs with a fully integrated backend solution.
After costly data‑deletion mishaps, 37 Interactive Entertainment engineered a robust, multi‑region backup platform that evolved from simple cron scripts to streaming xtrabackup with Celery‑driven task queues, encrypted HDFS/S3 storage, automated rotation and restore verification, ensuring reliable protection against high‑impact data loss.
This article deep‑dives into Celery task fundamentals, covering how to instantiate tasks with @app.task, customize task names, use binding for self‑reference, implement retries, access request context, and extend tasks via inheritance, all illustrated with clear Python code examples.
Explore how to configure Celery’s periodic (Beat) tasks, perform synchronous task calls, persist results using Redis, monitor workers with Flower, and debug remotely via telnet, with practical code examples and step‑by‑step instructions for robust backend task management.
This article introduces Celery, a flexible Python distributed task queue, explains its architecture—including broker, beat, worker, and result backend—then provides a complete, step‑by‑step setup using RabbitMQ and Redis, covering installation, configuration, task definition, worker launch, custom exchanges, and queue routing.
This article explains how attackers can combine SSRF vulnerabilities with unauthorized Redis access and unsafe serialization in Celery to infiltrate internal networks, illustrating the attack flow, exploitation techniques, and mitigation considerations for operations and security teams.
The article explains how insecure serialization in message‑queue middleware, especially Python's pickle used by Celery, can be abused to inject malicious payloads that trigger remote code execution on distributed workers, and it demonstrates detection and exploitation techniques against vulnerable Redis and MongoDB brokers.
This article introduces Celery's architecture, its integration with the OWL monitoring system, explains workers, brokers, result handling, and presents three practical alerting strategies—fixed thresholds, dynamic floating limits, and period‑over‑period comparisons—to build an efficient, automated operations platform.
This article introduces Celery, a Python‑based distributed task queue, explains its three‑part architecture, shows how to install RabbitMQ and Celery on an AWS EC2 Linux instance, and provides step‑by‑step code examples for defining tasks, running workers, and retrieving asynchronous results.
Learn how to install Celery, configure it with Redis as a broker, write a simple task to send emails asynchronously, and run workers, all demonstrated with clear code snippets and step‑by‑step instructions for Python developers.
