Exploring AI-Assisted Penetration Testing and Vulnerability Discovery

Introduction

General large language models (LLMs) have shown strong abilities in code generation, semantic understanding, and multi‑turn reasoning, prompting the question of how far these capabilities can be mapped to penetration‑testing scenarios.

Many penetration‑testing steps rely on "understanding" and "reasoning": interpreting an HTTP interface, inferring parameter semantics, and comparing responses to identify vulnerability evidence. From a capability‑matching perspective, LLMs appear naturally suited for these tasks.

At the same time, AI‑driven coding shortens development cycles, shrinking the time window for security testing and turning efficiency gains into a practical necessity.

AI Burp Copilot Design

1.1 Task Decomposition: Understanding and Execution

The workflow is broken into three layers:

Understanding Layer: What does the interface do? What is the business meaning of each parameter? What risks might exist?</code>
<code>↓</code>
<code>Execution Layer: Which parameter, which payload, how to judge the response?</code>
<code>↓</code>
<code>Review Layer: Is the response difference a true vulnerability evidence or normal behavior?

The nature of each layer determines the suitable executor:

Understanding Layer : semantic analysis, fuzzy classification, open reasoning – best handled by LLMs.

Execution Layer : precise construction, bulk replay, rule matching – best handled by a rule engine.

Review Layer : diff interpretation, rational explanation, false‑positive filtering – handled by a combination of LLM and rule engine.

The core architectural decision is to embed LLMs only at two workflow entry points (front‑end analysis and back‑end review) while delegating exact execution to a deterministic rule engine.

1.2 Why Not Drive Execution Directly with LLMs

Three fundamental issues prevent using LLMs for full request generation and sending:

Determinism loss : Penetration testing requires reproducible results. LLM outputs are probabilistic; even with temperature = 0, the model may still produce different recommendations for the same input, especially when judging subtle response differences.

Execution efficiency mismatch : Parameter‑level validation often involves hundreds of requests. Each LLM call adds hundreds of milliseconds to seconds plus token cost, whereas a rule engine can compare responses in microseconds, a difference of three orders of magnitude.

Audit‑trail breakage : Direct LLM‑driven requests hide the decision rationale inside the model, making it impossible to trace why a particular request was issued. Rule‑engine payloads and oracle logic are explicitly defined in YAML files, providing full auditability.

1.3 Architecture Overview

The system consists of a pipeline orchestration layer, multiple processing stages, a verification engine, and a UI presentation layer. Key stages include traffic collection, deduplication, status‑code filtering, endpoint classification (LLM), static scanning, AI analysis (LLM), risk assessment, and workflow verification (rule engine). The verification engine performs parameter influence assessment, rule matching, probe execution, local oracle judgment, optional LLM re‑review, and finding aggregation with confidence merging (confidence = 1 − ∏(1 − ci), default threshold 0.55).

1.4 Code Organization

pipeline/ – pipeline orchestration implementing IPipelineStage for independent toggling and hot‑loading.

verification/ – core verification engine handling influence analysis, payload loading, probe execution, oracle judgment, finding aggregation, and LLM review.

ai/ – abstraction layer for AI providers (OpenAI, DeepSeek, Qwen, compatible APIs) using a factory pattern for runtime switching.

config/ – external configuration (application.yml) for LLM endpoints, keys, and rate limits.

prompts/ – prompt templates for endpoint analysis, diff judgment, and endpoint classification.

1.5 Design Principles

HTTP‑first : Replay, diff calculation, and request construction are generic capabilities independent of specific vulnerability types.

Rules over hard‑coding : Detection logic resides in YAML files; Java code only provides the execution framework, enabling new vulnerability types without code changes.

LLM at the boundaries : LLMs appear only at workflow entry (classification, analysis) and exit (review), allowing AI provider replacement without affecting core verification logic.

Key Implementation Details

2.1 YAML Rule Definition

Rules are expressed in YAML files that describe what to test, how to test, and how to judge. A typical SQL‑injection rule includes fields such as id, technique, payloads, oracle, and confidence thresholds. Adding a new vulnerability type only requires placing a new YAML file under rules/payloads/, after which the plugin hot‑loads it.

2.2 Oracle Evidence Judgment

The rule engine supports nine oracle types, each with a specific judgment logic, e.g., ERROR_KEYWORD_OR_RECOVERY (detects error keywords or recovery from baseline errors for SQL‑error detection), TIME_DELAY (response time exceeds a threshold for time‑based blind injection), HTML_REFLECTION (payload appears in HTML for XSS), and others such as PAIR_DIFF, REDIRECT_LOCATION, BASELINE_DIFF, BASELINE_SIMILAR, EXPRESSION_EVALUATION, and KEYWORD. Each oracle outputs a confidence score (0‑1) that is merged across probes.

2.3 Parameter Influence Assessment

Before rule validation, a pre‑filter stage minimally mutates candidate parameters (e.g., increment numeric values) and re‑issues the request. Only parameters that cause a measurable difference from the baseline proceed to full verification, reducing noise from irrelevant parameters.

2.4 Evidence Chain Design

Each probe retains the full context: baseline request/response, mutated request/response, diff summary, local oracle result with confidence, optional LLM review, and final aggregated confidence. All data flow through a WorkflowContext object, ensuring each finding can be traced back to the original Burp traffic.

Real‑World Challenges

3.1 LLM Inconsistency

LLMs naturally produce nondeterministic outputs; the same interface and parameters may yield different attack‑surface recommendations at different times, violating the reproducibility required in security testing. Mitigations include caching analysis results, fixing temperature and prompt templates, enforcing strict JSON schema validation, and cross‑model evaluation (e.g., lowering confidence when GPT and DeepSeek disagree). Nonetheless, the inconsistency must be managed rather than eliminated, with the rule engine providing the final deterministic verdict.

3.2 Coverage of Business‑Logic Vulnerabilities

Pattern‑based vulnerabilities (SQLi, XSS, etc.) are well covered by YAML rules, but business‑logic flaws (e.g., amount manipulation, workflow skipping, coupon race conditions) depend heavily on understanding the application flow, which rules cannot capture. The current approach is to let LLMs issue "risk hints" when sensitive fields (amount, quantity, status) are detected, leaving the decision to manually verify these hints.

Conclusion

AI‑Burp‑Copilot is not a disruptive breakthrough but an exploratory auxiliary tool that demonstrates how LLMs can assist penetration testing without replacing deterministic rule‑based execution. The project is open‑source on GitHub (https://github.com/zxcvbn001/AI-Burp-Copilot) and invites collaboration from others attempting similar integrations.