Master Python Plugin Architecture with Pluggy and Stevedore – A Hands‑On Guide

In computing, a plug‑in is a software component that adds a specific feature to an existing program, allowing flexible configuration and easy loading.

Python’s dynamic nature makes plugin implementation especially flexible; dynamic plugins typically rely on Python’s namespace and dynamic import mechanisms to discover and load external dependencies. See the pytest documentation for creating and discovering plugins.

Plugin Framework

pluggy

pluggy is a plugin tool that evolved from pytest. It provides external plug‑in support for pytest, allowing developers to extend its functionality by defining a specification and implementing it.

The specification is marked with hookspec = pluggy.HookspecMarker("eggsample") and the implementation with hookimpl = pluggy.HookspecMarker("eggsample").

import pluggy

hookspec = pluggy.HookspecMarker("eggsample")

@hookspec
def eggsample_add_ingredients(ingredients: tuple):
    """Have a look at the ingredients and offer your own.

    :param ingredients: the ingredients, don't touch them!
    :return: a list of ingredients
    """
    ...

@hookspec
def eggsample_prep_condiments(condiments: dict):
    """Reorganize the condiments tray to your heart's content.

    :param condiments: some sauces and stuff
    :return: a witty comment about your activity
    """
    ...

Implementation example:

import pluggy

hookimpl = pluggy.HookimplMarker("eggsample")

class ExamplePluggy:
    @hookimpl
    def eggsample_add_ingredients(self):
        spices = ["salt", "pepper"]
        eggs = ["egg", "egg"]
        return spices + eggs

    @hookimpl
    def eggsample_prep_condiments(self, condiments):
        condiments["mint sauce"] = 1
        return "Now this is what I call a condiments tray!"

Plugins are loaded into a PluginManager and discovered via load_setuptools_entrypoints:

import pluggy
import itertools
import random

def get_plugin_manager():
    pm = pluggy.PluginManager("eggsample")
    pm.add_hookspecs(hookspecs)
    pm.load_setuptools_entrypoints("eggsample")
    pm.register(ExamplePluggy)
    return pm

pm = get_plugin_manager()
pm.hook.eggsample_add_ingredients()

stevedore

stevedore is an OpenStack‑maintained plugin library. It provides plug‑in functionality for projects such as Ceilometer and recommends defining a base class to specify the plug‑in interface.

import abc

class FormatterBase(metaclass=abc.ABCMeta):
    def __init__(self, max_width=60):
        self.max_width = max_width

    @abc.abstractmethod
    def format(self, data):
        """Format the data and return unicode text."""

Simple plug‑in implementation:

class Simple(FormatterBase):
    """A very basic formatter."""

    def format(self, data):
        for name, value in sorted(data.items()):
            line = f"{name} = {value}
"
            yield line

Packaging with setup.py and entry points:

from setuptools import setup, find_packages

setup(
    name='stevedore-examples',
    version='1.0',
    description='Demonstration package for stevedore',
    packages=find_packages(),
    include_package_data=True,
    entry_points={
        'stevedore.example.formatter': [
            'simple = stevedore.example.simple:Simple',
            'plain = stevedore.example.simple:Simple',
        ],
    },
    zip_safe=False,
)

Using the driver interface:

from stevedore import driver

mgr = driver.DriverManager(
    namespace='stevedore.example.formatter',
    name='simple',
    invoke_on_load=True,
    invoke_args=(60,),
)

for chunk in mgr.driver.format(data):
    print(chunk, end='')

Using the extension interface:

from stevedore import extension

mgr = extension.ExtensionManager(
    namespace='stevedore.example.formatter',
    invoke_on_load=True,
    invoke_args=(60,),
)

def format_data(ext, data):
    return (ext.name, ext.obj.format(data))

results = mgr.map(format_data, data)
for name, result in results:
    print(f"Formatter: {name}")
    for chunk in result:
        print(chunk, end='')
    print()

Practice

Based on hands‑on experience, the author recommends using stevedore for Python plug‑in development. Advantages include:

Defining an interface via a base class provides clear specifications.

Plug‑in invocation is more flexible.

Stevedore offers lower complexity compared to alternatives.