Software Security Testing: Objectives, Common Vulnerabilities, and Static/Dynamic Approaches

Security testing is the process of inspecting a software product during its lifecycle, especially from near completion to the release stage, to verify that the product conforms to defined security requirements and quality standards.

Goal Confirm that the software has no code bugs or security vulnerabilities caused by developers' limited knowledge.

Common Security Issues

Shell usage vulnerabilities

Thread handling vulnerabilities

Relative‑path DLL loading problems

DLL search‑path loading issues

Various types of DLL hijacking

ASLR/DEP exploitation

Privilege‑escalation attacks

Heap overflow vulnerabilities

Improper validation of sensitive files

Network security flaws

Cache buffer vulnerabilities

Testing Approach

1. Static Scanning Similar to white‑box testing but with a different focus, static scanning builds rule‑based engines for known dangerous vulnerabilities. The engine runs on each code commit, alerts on findings, and continuously expands a knowledge base of security rules. For example, DLL hijacking occurs when a program loads a DLL via LoadLibrary (or similar) without an absolute path, causing the system to search directories in order and potentially load a malicious DLL placed earlier in the search path.

Search order for a relative‑path DLL: Directory of the executable loading the DLL Current working directory of the loading process \WINNT\SYSTEM32 directory \WINNT directory Directories listed in the PATH environment variable

Because system DLLs are usually not found in the first two locations, placing a forged DLL there causes it to be loaded, so a static rule can alert on calls to LoadLibrary (or similar) that use non‑absolute paths.

Static scanning quickly addresses popular vulnerabilities and serves as a pre‑release gate for code quality, but it is less effective for ambiguous or indirect security issues such as privilege‑escalation bugs, which may require dynamic testing as a complement.

2. Dynamic Testing Dynamic testing runs the software in a controlled environment to trigger security flaws and observe how the program handles them. It includes two main methods:

Setting up environments that contain known vulnerabilities (e.g., placing a forged DLL in a directory to provoke DLL hijacking) and verifying that the software improperly loads the malicious DLL.

Testing less common but serious issues, such as functions that launch other processes, by exercising those functions and using monitoring tools to ensure low‑privilege processes are not escalated to high‑privilege ones.

Both static and dynamic techniques together provide a comprehensive security assessment before software release.

Long press the QR code to follow