Master Python Scheduling: 10 Practical Ways to Run Periodic Tasks

In daily work we often need periodic tasks; besides Linux crond we can use Python. This article lists common Python scheduling methods.

Table of Contents

Using while True + sleep() to implement a timer

Using Timeloop library to run scheduled jobs

Using threading.Timer to implement a timer

Using built‑in sched module to implement a timer

Using schedule module to implement a timer

Using APScheduler task framework

Using distributed message system Celery

Using data‑flow tool Apache Airflow

Using while True + sleep() to implement a timer

The sleep(secs) function in the time module pauses the current thread for secs seconds, after which the thread becomes ready for scheduling. By combining an infinite while loop with sleep() we can create a simple periodic task.

import datetime
import time

def time_printer():
    now = datetime.datetime.now()
    ts = now.strftime('%Y-%m-%d %H:%M:%S')
    print('do func time :', ts)

def loop_monitor():
    while True:
        time_printer()
        time.sleep(5)  # pause 5 seconds

if __name__ == "__main__":
    loop_monitor()

Can only set intervals, not specific times (e.g., 08:00 every day). sleep blocks the thread, preventing any other work during the pause.

Using Timeloop library to run scheduled jobs

Timeloop is a lightweight library that runs periodic jobs using decorators in a separate thread.

import time
from timeloop import Timeloop
from datetime import timedelta

tl = Timeloop()

@tl.job(interval=timedelta(seconds=2))
def sample_job_every_2s():
    print("2s job current time : {}".format(time.ctime()))

@tl.job(interval=timedelta(seconds=5))
def sample_job_every_5s():
    print("5s job current time : {}".format(time.ctime()))

@tl.job(interval=timedelta(seconds=10))
def sample_job_every_10s():
    print("10s job current time : {}".format(time.ctime()))

Using threading.Timer to implement a timer

The Timer class in the threading module creates a non‑blocking timer. Multiple timers can run concurrently, but each timer fires only once, so a loop is needed for repeated execution. interval: the delay time. function: the callable to execute. args/kwargs: arguments for the callable.

Note: Timer executes only once; you must restart it inside the callback for repeated runs.

Using built‑in sched module to implement a timer

The sched.scheduler class provides a generic event scheduler that works with any time and delay functions (commonly time.time and time.sleep). It supports multithreaded applications and can schedule multiple events.

import datetime
import time
import sched

def time_printer():
    now = datetime.datetime.now()
    ts = now.strftime('%Y-%m-%d %H:%M:%S')
    print('do func time :', ts)
    loop_monitor()

def loop_monitor():
    s = sched.scheduler(time.time, time.sleep)  # create scheduler
    s.enter(5, 1, time_printer, ())
    s.run()

if __name__ == "__main__":
    loop_monitor()

Key methods of the scheduler object: enter(delay, priority, action, argument): schedule an event after delay seconds. cancel(event): remove a scheduled event. run(): execute all pending events, waiting as needed.

Using schedule module to implement a timer

schedule

is a third‑party lightweight scheduler that allows you to run functions at seconds, minutes, hourly, daily, or custom intervals using a clear, human‑readable syntax.

import schedule
import time

def job():
    print("I'm working...")

schedule.every(10).seconds.do(job)
schedule.every(10).minutes.do(job)
schedule.every().hour.do(job)
schedule.every().day.at("10:30").do(job)
schedule.every(5).to(10).minutes.do(job)
schedule.every().monday.do(job)
schedule.every().wednesday.at("13:15").do(job)
schedule.every().minute.at(":17").do(job)

while True:
    schedule.run_pending()
    time.sleep(1)

Decorators and parameter passing are also supported:

import schedule
from schedule import every, repeat, run_pending

@repeat(every().second)
def job():
    print('working...')

while True:
    run_pending()
    time.sleep(1)

Jobs can be cancelled, run once, or filtered by tags:

import schedule

def some_task():
    global i
    i += 1
    print(i)
    if i == 10:
        schedule.cancel_job(job)
        print('cancel job')
        exit(0)

job = schedule.every().second.do(some_task)
while True:
    schedule.run_pending()

Using APScheduler to implement a timer

APScheduler (Advanced Python Scheduler) is a Quartz‑based framework that supports date‑based, interval‑based, and cron‑style triggers, with persistence, multiple executors, and job stores.

Key components:

Job : the smallest execution unit; defines the callable, arguments, and execution settings.

Trigger : determines the next run time (DateTrigger, IntervalTrigger, CronTrigger).

Executor : runs the job (thread pool, process pool, asyncio, etc.).

Jobstore : stores jobs (memory, MongoDB, Redis, SQLAlchemy, etc.).

Event : lifecycle events such as job added, executed, error, missed.

Scheduler : orchestrates triggers, executors, and jobstores.

from apscheduler.schedulers.blocking import BlockingScheduler
from datetime import datetime

def job():
    print(datetime.now().strftime("%Y-%m-%d %H:%M:%S"))

sched = BlockingScheduler()
sched.add_job(job, 'interval', seconds=5, id='my_job_id')
sched.start()

Important APScheduler concepts (Job, Trigger, Executor, Jobstore, Event, Scheduler) are illustrated in the diagram above.

Using distributed message system Celery to implement a timer

Celery is a robust distributed task queue that can also handle periodic tasks via celery beat. It requires a broker (RabbitMQ, Redis, etc.) and a result backend. While powerful, Celery is heavyweight for simple scheduling.

Celery architecture consists of Beat (scheduler), Producer, Broker, Workers, and Result Backend.

Using data‑flow tool Apache Airflow to implement a timer

Apache Airflow, originally from Airbnb, expresses workflows as Directed Acyclic Graphs (DAGs). Each node is a task (Operator) and edges define dependencies. Airflow provides a rich UI, many built‑in operators, and supports branching, retries, and scheduling.

Core Airflow concepts:

DAGs : define the overall workflow and execution order.

Operators : reusable task classes (BashOperator, PythonOperator, EmailOperator, etc.).

Tasks : instances of operators within a DAG.

Task Instances : a single run of a task with its state.

Task Relationships : dependencies expressed with >> or set_upstream / set_downstream.

Airflow architecture includes a metadata database, Scheduler, Executor (Sequential, Local, Celery, Dask, Kubernetes), and Workers. Diagrams illustrate the flow for CeleryExecutor and KubernetesExecutor.

Original source: https://www.biaodianfu.com/python-schedule.html