Implementing a Python Import‑Hook Probe with sys.meta_path and sitecustomize

This article explains the principle behind a Python probe and demonstrates a minimal program that measures the execution time of a specified function. The implementation relies on two key concepts: sys.meta_path and sitecustomize.py.

sys.meta_path is a list of finder objects that the import machinery consults when an import statement is executed. By inserting a custom finder that implements find_module (returning a loader with a load_module method), we can intercept imports and replace the loaded module with our own implementation. The following example defines a MetaPathFinder and MetaPathLoader that simply prints messages when modules are found and loaded:

import sys
class MetaPathFinder:
    def find_module(self, fullname, path=None):
        print('find_module {}'.format(fullname))
        return MetaPathLoader()
class MetaPathLoader:
    def load_module(self, fullname):
        print('load_module {}'.format(fullname))
        sys.modules[fullname] = sys
        return sys
sys.meta_path.insert(0, MetaPathFinder())
if __name__ == '__main__':
    import http
    print(http)
    print(http.version_info)

Using this hook we can replace an imported module (e.g., http) with another object ( sys) and observe the hook being triggered:

$ python meta_path1.py
find_module http
load_module http<module 'sys' (built-in)>
sys.version_info(major=3, minor=5, micro=1, releaselevel='final', serial=0)

To measure function execution time we wrap the target function with a decorator. The decorator prints a start message, records the time before and after the call, and prints the elapsed time:

import functools
import time

def func_wrapper(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        print('start func')
        start = time.time()
        result = func(*args, **kwargs)
        end = time.time()
        print('spent {}s'.format(end - start))
        return result
    return wrapper

def sleep(n):
    time.sleep(n)
    return n

if __name__ == '__main__':
    func = func_wrapper(sleep)
    print(func(3))

Running the script yields:

$ python func_wrapper.py
start func
spent 3.004966974258423s
3

Building on this, we create a hook that automatically decorates a specific function (e.g., hello.sleep) when the hello module is imported. The hook is placed in hook.py and the decorator logic is reused:

import functools
import importlib
import sys
import time

_hook_modules = {'hello'}

class MetaPathFinder:
    def find_module(self, fullname, path=None):
        print('find_module {}'.format(fullname))
        if fullname in _hook_modules:
            return MetaPathLoader()

class MetaPathLoader:
    def load_module(self, fullname):
        print('load_module {}'.format(fullname))
        if fullname in sys.modules:
            return sys.modules[fullname]
        finder = sys.meta_path.pop(0)
        module = importlib.import_module(fullname)
        module_hook(fullname, module)
        sys.meta_path.insert(0, finder)
        return module

sys.meta_path.insert(0, MetaPathFinder())

def module_hook(fullname, module):
    if fullname == 'hello':
        module.sleep = func_wrapper(module.sleep)

def func_wrapper(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        print('start func')
        start = time.time()
        result = func(*args, **kwargs)
        end = time.time()
        print('spent {}s'.format(end - start))
        return result
    return wrapper

Testing the hook:

>> import hook
>>> import hello
find_module hello
load_module hello
>>> hello.sleep(3)
start func
spent 3.0029919147491455s
3

To avoid manually importing the hook each time, we can use sitecustomize.py. Python automatically imports any sitecustomize module found on PYTHONPATH during interpreter startup. Placing import hook inside sitecustomize.py ensures the hook is registered automatically.

$ cat sitecustomize.py
import hook

Running Python with the current directory added to PYTHONPATH now prints the messages from sitecustomize and the hook without any extra import statements.

$ export PYTHONPATH=.$ python
this is sitecustomize
this is usercustomize
>>> import hello
find_module hello
load_module hello
>>> hello.sleep(3)
start func
spent 3.005002021789551s
3

For a more convenient command‑line experience similar to tools like newrelic-admin run-program, we create an agent.py script that temporarily prepends a dedicated bootstrap directory to PYTHONPATH and then execs the target Python program. The directory layout becomes:

├── bootstrap/
│   ├── __init__.py
│   ├── _hook.py
│   └── sitecustomize.py
├── hello.py
├── test.py
└── agent.py

The bootstrap/sitecustomize.py simply imports the internal hook:

import _hook

The agent.py implementation:

import os
import sys
current_dir = os.path.dirname(os.path.realpath(__file__))
boot_dir = os.path.join(current_dir, 'bootstrap')

def main():
    args = sys.argv[1:]
    os.environ['PYTHONPATH'] = boot_dir
    # Replace the current process with the target Python command
    os.execl(sys.executable, sys.executable, *args)

if __name__ == '__main__':
    main()

Usage example:

$ python agent.py test.py arg1 arg2
find_module usercustomize
find_module hello
load_module hello
['test.py', 'arg1', 'arg2']
start func
spent 3.005035161972046s
3

In summary, the article walks through building a basic Python probe using import hooks, automatic loading via sitecustomize, and a lightweight command‑line wrapper, illustrating core backend‑development techniques for runtime instrumentation.