Reverse Engineering and Patching Python .pyd Modules with IDA

Recently I encountered a foreign app whose protocol software is written in Python 3.8, compiled into .pyd modules and packaged with PyInstaller. The program checks a machine code and requires an authorization key, prompting me to attempt a crack by modifying the assembly inside the .pyd.

Prerequisite Knowledge

1. py, pyc, pyo, pyd

py: Python source script.

pyc: Compiled bytecode file, improves load speed.

pyo: Optimized bytecode generated with -O flag.

pyd: Windows DLL that serves as a Python extension module.

2. Compiling a .pyd

Script to compile:

util1.py

def fun_hello(s):
    if s == 1:
        return 'hello world'
    elif s == 2:
        return '222222222'

Setup script:

setup.py

from setuptools import setup
from Cython.Build import cythonize

setup(
    name='test',
    ext_modules=cythonize('util1.py')
)

Run the following command in the directory containing setup.py: python setup.py build_ext --inplace This creates a .pyd file in the same directory (32‑bit Python produces a 32‑bit .pyd, 64‑bit produces a 64‑bit .pyd).

Using the .pyd

Example script test.py:

import util1

if __name__ == '__main__':
    print(util1.fun_hello(2))

Packaging with PyInstaller

pip install pyinstaller

pyinstaller test.py

Unpacking a PyInstaller‑packed EXE

Use pyinstxtractor.py to extract the contents. The packed files are compressed with zlib.compress and then assembled into the EXE, so directly editing the binary is difficult. Instead, unpack, modify the .pyd, gather required libraries, and re‑package with PyInstaller.

GitHub repository: https://github.com/countercept/python-exe-unpacker

Decompiling .pyc

Use uncompyle6 to decompile Python 3.8 bytecode. If the extracted .pyc lacks the magic header, add the correct header from a matching Python installation before decompiling.

Magic header for Python 3.8 32‑bit can be found in any .pyc file of that version.

GitHub repository: https://github.com/rocky/python-uncompyle6

Analyzing the .pyd Assembly and Mapping to Python Code

Load the .pyd into IDA (32‑bit or 64‑bit as appropriate). The module typically has a single exported initialization function. Inside, locate the structure __pyx_moduledef_slots and the function __pyx_pymod_exec_util1, which initializes all Python objects.

Find the constant table by searching for calls to PyUnicode_InternFromString. For example, the offset dword_10006DFC corresponds to the string "222222222". Cross‑reference this to locate related code.

Locate the function table that maps Python function names to assembly implementations. The entry aFunHello points to __pyx_pf_5util1_fun_hello, the compiled version of fun_hello.

In the assembly, the condition s == 1 is implemented with __Pyx_PyInt_EqObjC followed by PyObject_IsTrue. The jump instruction jz short loc_10004753 determines the branch.

Patch this jump to jnz short loc_10004753 to flip the condition, effectively bypassing the original check.

In IDA: Edit → Patch program → Assemble, then Apply patches to the input file to produce a modified .pyd.

After patching, running the script outputs "hello world" instead of the expected "222222222", demonstrating a successful bypass.

This example shows how to locate and modify simple conditional logic; similar techniques can be applied to alter other execution paths.

Note: The original foreign app was not fully cracked; this post only shares the analysis steps performed on a self‑created example for reference.