How to Run Celery Tasks with Redis Without a Web Framework

Celery is a distributed asynchronous task queue that can be used independently of a web framework, and it can reuse an existing Redis installation as both broker and result backend.

Project structure:

$ tree your_project
your_project
├── __init__.py
├── main.py
├── celery.py
└── tasks.py

0 directories, 4 files

main.py

contains the business code that triggers tasks, celery.py defines the Celery app, and tasks.py holds task definitions.

Configure Celery

Install Celery with Redis support and add the following to celery.py:

from celery import Celery
from .settings import REDIS_URL

APP = Celery(
    main=__package__,
    broker=REDIS_URL,
    backend=REDIS_URL,
    include=[f'{__package__}.tasks'],
)

APP.conf.update(task_track_started=True)

The REDIS_URL (e.g., redis://localhost:6379/0) is read from settings.py and serves as both broker and backend, storing results for up to one day.

Write and Call Tasks

In tasks.py define a task:

from .celery import APP

@APP.task
def do_sth():
    pass

Trigger it from business code with .apply_async():

from celery.result import AsyncResult
from .tasks import do_sth

result = do_sth.apply_async()
assert isinstance(result, AsyncResult)

Start a worker:

celery -A your_project worker

Execution Flow

The diagram below shows the sequence from task submission to completion.

The main process (e.g., a Django or Flask app) sends a message to Redis, receives an AsyncResult containing a task_id, and can later query the task state.

Task States

The state diagram illustrates possible statuses:

Besides SUCCESS, tasks may end in FAILURE or REVOKED. RETRY indicates a temporary waiting state when retries are configured. If the result expires in Redis (default one day), the state becomes PENDING, which requires proper error handling.

Common Control Operations

result = AsyncResult(task_id)
# block until result is ready
result.wait()
# cancel the task
result.revoke()
# delete task record
result.forget()

Use wait when the calling process needs the result, revoke to cancel pending or running tasks, and forget to remove the task’s metadata.