How to Build a Custom Python Import Hook Probe with sys.meta_path and sitecustomize

Previously I wrote a detailed article on Python's import mechanism; this piece focuses on implementing a simple probe to measure function execution time, covering two key concepts: sys.meta_path and sitecustomize.py .

sys.meta_path

sitecustomize.py

sys.meta_path

The sys.meta_path list enables custom import hooks. When an import statement is executed, Python iterates over objects in sys.meta_path and calls their find_module method. If find_module returns an object with a load_module method, that object is used to load the module.

import sys
class MetaPathFinder:
    def find_module(self, fullname, path=None):
        print('find_module {}'.format(fullname))
        return MetaPathLoader()

class MetaPathLoader:
    def load_module(self, fullname):
        print('load_module {}'.format(fullname))
        sys.modules[fullname] = sys
        return sys

sys.meta_path.insert(0, MetaPathFinder())
if __name__ == '__main__':
    import http
    print(http)
    print(http.version_info)

The load_module method returns a module object, which becomes the imported module. For example, the above code replaces the http module with the sys module.

$ python meta_path1.py
find_module http
load_module http
<module 'sys' (built-in)>
sys.version_info(major=3, minor=5, micro=1, releaselevel='final', serial=0)

Using sys.meta_path, we can swap a module's objects ("狸猫换太子") to inject timing logic. For functions, a decorator can wrap the original function:

import functools
import time

def func_wrapper(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        print('start func')
        start = time.time()
        result = func(*args, **kwargs)
        end = time.time()
        print('spent {}s'.format(end - start))
        return result
    return wrapper

import time

def sleep(n):
    time.sleep(n)
    return n

if __name__ == '__main__':
    func = func_wrapper(sleep)
    print(func(3))

$ python func_wrapper.py
start func
spent 3.004966974258423s
3

Next, we create a probe that measures the execution time of a specific function in a target module. Assume hello.py contains:

import time

def sleep(n):
    time.sleep(n)
    return n

The import hook ( hook.py) is:

import functools
import importlib
import sys
import time

_hook_modules = {'hello'}

class MetaPathFinder:
    def find_module(self, fullname, path=None):
        print('find_module {}'.format(fullname))
        if fullname in _hook_modules:
            return MetaPathLoader()

class MetaPathLoader:
    def load_module(self, fullname):
        print('load_module {}'.format(fullname))
        if fullname in sys.modules:
            return sys.modules[fullname]
        finder = sys.meta_path.pop(0)
        module = importlib.import_module(fullname)
        module_hook(fullname, module)
        sys.meta_path.insert(0, finder)
        return module

sys.meta_path.insert(0, MetaPathFinder())

def module_hook(fullname, module):
    if fullname == 'hello':
        module.sleep = func_wrapper(module.sleep)

def func_wrapper(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        print('start func')
        start = time.time()
        result = func(*args, **kwargs)
        end = time.time()
        print('spent {}s'.format(end - start))
        return result
    return wrapper

Test code:

>> import hook
>>> import hello
find_module hello
load_module hello
>>> hello.sleep(3)
start func
spent 3.0029919147491455s
3

To avoid manually importing the hook, we can use sitecustomize.py, which Python automatically imports on startup if it is on PYTHONPATH. A simple sitecustomize.py can contain:

import hook

With the directory structure:

.
├── hello.py
├── hook.py
└── sitecustomize.py

Running Python with PYTHONPATH=. automatically registers the hook.

agent

To package this into a command‑line tool, we reorganize the project:

mkdir bootstrap
mv hook.py bootstrap/_hook.py
touch bootstrap/__init__.py
touch agent.py

Bootstrap's sitecustomize.py imports the hidden hook:

import _hook

The agent.py script sets PYTHONPATH to the bootstrap directory and then execs the target Python program:

import os
import sys
current_dir = os.path.dirname(os.path.realpath(__file__))
boot_dir = os.path.join(current_dir, 'bootstrap')

def main():
    args = sys.argv[1:]
    os.environ['PYTHONPATH'] = boot_dir
    os.execl(sys.executable, sys.executable, *args)

if __name__ == '__main__':
    main()

Usage example:

$ python agent.py test.py arg1 arg2
find_module usercustomize
find_module hello
load_module hello
['test.py', 'arg1', 'arg2']
start func
spent 3.005035161972046s
3

This demonstrates a minimal Python probe; real‑world agents are far more sophisticated. The full source code is available at https://github.com/mozillazg/apm-python-agent-principle .