Design and Implementation of a Full-Platform Automated Test Execution Engine

With the gradual deepening of interface automation at the company, the shortcomings of the legacy automation solution (offline scripts + Jenkins + platform) have become increasingly evident, such as large differences in technology stacks, poor user compatibility, low case authoring efficiency, complex platform integration, poor adaptability, uncontrollable case scripts, high maintenance cost, and long execution time. To address these issues, the automation platform was transformed from a “semi‑platform” to a “full‑platform”, achieving a lightweight, efficient, feature‑complete, easy‑to‑use, and highly standardized solution that supports online visual editing, component‑based (reusable), full‑code, low‑code, and zero‑code case creation.

The Automation Executor is a self‑developed component of the automation platform responsible for executing the cases and scripts written on the platform. It supports both individual debugging and batch execution according to test plans, providing capabilities such as serial/parallel case execution, placeholders, system methods, read‑only environment variables, variable spaces (read/write), interpreted API steps, and native script execution.

The executor draws inspiration from popular API testing tools (JMeter, Postman, eolink, MeterSphere) and mainstream unit‑test frameworks (TestNG, PyTest, unittest) and adds platform‑specific features.

class EnvVarsProxy(object):
    """Environment variable proxy – read‑only"""
    def __init__(self, env_vars_space: EnvVarsSpace):
        self.__env_vars_space = env_vars_space

    def __setattr__(self, name, value):
        # Intercept illegal calls to protect critical data
        stacks = inspect.stack()
        source_stack: FrameInfo = stacks[1]
        if source_stack.filename != __file__:
            raise Exception("EnvVarsProxy object attribute setting is prohibited")
        super().__setattr__(name, value)

    def __getattribute__(self, name):
        if name in ('_EnvVarsProxy__env_vars_space', '__dict__'):
            stacks = inspect.stack()
            source_stack: FrameInfo = stacks[1]
            if source_stack.filename != __file__:
                raise Exception("EnvVarsProxy protected attribute access is prohibited")
        return super().__getattribute__(name)

Deep protection is also applied to reference‑type data: the get method returns a deep‑copied value, ensuring that modifications in user space do not affect the original metadata.

def get(self, key):
    value = self.__env_vars_dict.get(str(key))
    if value is not None:
        return copy.deepcopy(value)
    return value

The ScriptComponentWorker manages the lifecycle of script components, dynamically binds main‑step environment variables, initializes local variable spaces, and injects key system variables. It ensures that scripts run in a controlled environment and that placeholders are resolved before execution.

def _init(self):
    """Initialize script component worker"""
    self.component = ScriptComponent(self.task_md, self.raw_data, self)
    l_vars_space: VarsSpace = VarsSpaceManager.get_l_vars_space()
    l_vars_space.set(VarsSpaceManager.SCRIPT_COMPO_ID_KEY, self.component.component_md.component_id)
    l_vars_space.set(VarsSpaceManager.SCRIPT_COMPO_NAME_KEY, self.component.component_md.name)

The CaseWorker controls case execution, handling retries, pre‑ and post‑scripts, variable space initialization, and result aggregation. It isolates logs per retry to avoid cross‑contamination.

def run(self):
    for i in range(self._retry_count):
        self._set_case_status(False)
        if i > 0:
            time.sleep(self.data_case.data_case_md.retry_sleep)
            self.logger_group.logger_proxy.info(f"Retry {i} for case")
        try:
            self._run()
            if self.logger_group.logger_space.fail_msg_records:
                self._set_case_status(True)
                continue
        except AssertFail as e:
            self.logger_group.logger_space.add_fail_msg(LogUnit(title=self.data_case.case_desc_info, content=f"Assert failed: {e}"))
            self._set_case_status(True)
            self._run_after_script()
            continue

Placeholders are a core feature of the platform, enabling high‑frequency parameter substitution from variable spaces, environment variables, or system methods. They support both simple (${var}) and enhanced (${_l->list[2:10:2]}) forms, with full Python‑like slicing, attribute access, and method invocation.

@SysFuncsLib.load_sys_func(name='switch_timestamp', param_rules=[ParamRule('length', int, required=True), ParamRule('date_string', str, required=True), ParamRule('format', str, required=True)])
def switch_timestamp(length: int, date_string: str, format: str) -> int:
    date_obj = datetime.datetime.strptime(date_string, format)
    timestamp = str(date_obj.timestamp()).replace('.', '')
    if len(timestamp) < length:
        timestamp += '0' * (length - len(timestamp))
    return int(timestamp[:length])

System methods are loaded into a central library and accessed via SysFuncsProxy , allowing scripts and placeholders to call utilities such as random data generators, timestamp converters, and more.

class SysFuncsProxy(object):
    """System functions proxy"""
    def __getattr__(self, name):
        func_info = SysFuncsLib.SYS_FUNCS_DICT.get(name)
        if not func_info:
            raise SysFuncsNonExistentError(f'System method "{name}" does not exist')
        return func_info['func']

The platform also provides a rich set of built‑in libraries (Python standard libraries, third‑party packages, and custom utilities) that can be imported in any script, further extending the capabilities of test cases.

In summary, the automation executor integrates the best ideas from existing testing frameworks, adds extensive placeholder and system‑method support, and offers a flexible, extensible backend solution for large‑scale, high‑efficiency automated testing.