How Suning Built a Python‑Powered Automated SAP Testing Framework to Cut Manual Effort by 10×

Background

Increasingly complex systems and strict user‑experience requirements make manual testing labor‑intensive and inconsistent, prompting the need to convert human‑driven test actions into automated processes.

Suning Automation Tester (SAT)

Suning previously built SAT, a Java‑based keyword‑driven automation tool supporting web pages, HTTP, mobile apps, databases, and Linux operations.

New Python‑Based Solution

The current approach abandons SAT and uses Python to implement a web‑client interaction framework that combines data‑driven, structured, and keyword‑driven scripting techniques.

1. Goal and Current Situation Analysis

The objective is to enable automated interaction between a B‑side web application and a C‑side SAP client, eliminating cumbersome manual testing of non‑UI backend systems.

1.1 Current Systems

Testingland, a self‑developed B/S Python platform, serves as the automation test framework, bridging B‑side web apps and C‑side SAP R/3 (FI/CO modules).

1.2 Research Findings

Two common methods for invoking a local SAP client from a web page were evaluated:

ActiveX via IE (limited to IE).

Custom URL protocol (supports Chrome, IE, Firefox) but requires registry configuration.

Both only launch the client without automating subsequent actions.

1.3 Analysis

Existing methods lack sustainability, flexibility, and remote execution capabilities.

2. Proposed Solution

A custom automation framework was designed to automate SAP operations, support both local and remote agents, and provide configurable, version‑controlled scripts.

2.1 Implementation Steps

User selects an execution terminal on the web and submits input data.

The web server determines whether the terminal is a shared remote agent or a local PC and locks an idle terminal.

Commands are fetched from a database command library and transmitted via a socket protocol.

The terminal checks the environment and script versions, updating if necessary.

Data commands are sent to the SAP client, triggering the SAP automation script.

Execution results are sent back through the socket service to the web server for display.

2.2 Solving Specific Problems

Python Selenium handles browser actions; win32 and SAP‑API automate SAP GUI; pywinauto manipulates generic clients. Remote agents run a Python socket service that downloads, updates, and executes scripts, returning results to the web UI.

3. Core Technologies

3.1 Socket Service

Implemented with Python's socket module and packaged via pyinstaller. The server listens for requests, manages script versions, executes automation scripts, and returns results. The client sends commands and receives execution outcomes.

3.2 Process Invocation

Subprocess is used to run commands independently, with PIPE for parameter passing and result capture.

3.3 Windows CMD/DOS Commands

CMD commands retrieve SAP installation paths and perform login operations.

# 获取 sap 安装路径
os.system("start saplogon.exe")  # 打开进程
cmd = 'wmic process where name="saplogon.exe" get executablepath'  # cmd 命令
sb = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)  # 执行命令

3.4 SAP Script Development

Using win32 to connect SAP and the SAP GUI Scripting API; component locations are obtained via recorded scripts or the Scripting Tracker tool.

4. Practical Application and Analysis

4.1 Manual Testing Effort

Opening SAP, logging in, and creating a voucher takes ~2 minutes; verification adds ~1 minute; documenting results adds ~2 minutes per voucher.

4.2 Automated Execution

Automation logs show that generating five vouchers took 97 seconds, covering data checks, voucher creation, result verification, UI updates, screenshots, and log display.

4.3 Result Analysis

System automatically checks and validates results, reducing human effort by nearly tenfold.

Automation is over five times faster than manual execution.

Testers benefit from simpler operations, direct result visibility, precise issue localization, and lower resource costs.

5. Conclusion

Although the initial investment exceeded 60 person‑days over two months, the framework saved approximately 5,000 man‑hours in a year. The key challenge was not the Python‑based SAP automation itself, but establishing reliable, automated interaction between B‑side web applications and C‑side SAP systems, which was solved via custom socket communication, modular command libraries, and automated script deployment.