Auditing AI Coding Agents: From Execution Logs to Risk Investigation

Behavior Recording with LoongSuite-Pilot

When a coding agent fixes an issue, it may read documentation, access files, run commands, and execute tests. Developers often only see the final answer and test results, missing the intermediate data accesses and tool operations. LoongSuite-Pilot addresses this by ingesting session logs or hook records from various coding agents (Cursor, Claude Code, etc.), converting them into a unified event stream. It handles continuous log writing, file rotation, truncation, and process restarts, ensuring a complete audit trail.

Event Phases and Call Correlation

LoongSuite-Pilot distinguishes four event phases using event.name: llm.request – model receives input (user request, context) llm.response – model returns output (answer, tool call requests) tool.call – agent invokes a tool (tool name, parameters, target) tool.result – tool returns result (content, errors, status)

A model proposing a tool call does not mean the agent executed it; the tool result must be examined. To correlate calls with results, LoongSuite-Pilot preserves Session, Turn, Step, and Tool Call ID from the original logs. If IDs are missing, it cross-references parameters, turn numbers, and log content rather than relying on temporal proximity alone.

Data Masking for Sensitive Content

Model messages, tool parameters, and results may contain API keys, cloud credentials, private keys, database passwords, or personal information. LoongSuite-Pilot can mask these before output. For example, an API key becomes [APIKEY_MASKED], indicating a secret was present while retaining event name, timestamp, and call relationships. Masked values are normalized (different keys become the same token), so investigators cannot assume two masked occurrences represent the same key. Masking only affects LoongSuite-Pilot output; original agent logs remain unchanged and require separate access control.

Risk Analysis with AgentLoop

AgentLoop consumes the unified events and applies rules to identify risks: sensitive data access, credential usage, destructive commands, and suspected prompt injection. Deterministic checks (e.g., command patterns) produce immediate findings; contextual analyses (task meaning, behavioral sequences) are delegated to a model that compares user intent, tool parameters, and returns against the recorded activity.

Case Study: Indirect Prompt Injection (Cursor CVE-2026-31854)

Cursor's March 2026 advisory (CVE-2026-31854, affecting versions ≤1.4.5, fixed in 2.0) described a risk where a malicious web page could inject instructions that the model treats as task requirements, potentially bypassing the Use AllowList mode (which only auto-executes allow-listed commands). AgentLoop investigates by comparing the web page's demanded actions and targets with the actual tool calls, then contrasting those with the user's original task. The key question: did the agent adopt external instructions and execute operations beyond user authorization? If the agent merely explained a command example from the page, that is not a successful injection. Only when records show the agent acted on external instructions and initiated unauthorized operations is risk confirmed. Tool results must also be examined: a rejected call, an error, or a successful execution carry different implications.

Case Study: Environment Variable Misuse Leading to Data Loss (Claude Code Issue #75859)

In July 2026, a user reported that Claude Code deleted personal files while cleaning a test directory. The agent first ran export HOME=$(mktemp -d) in one Bash call, setting HOME to a temporary directory. In a subsequent Bash call, it executed rm -rf "$HOME" intending to remove that temporary directory. However, the environment variable assignment did not persist across calls; HOME reverted to the user's real home directory, causing the deletion to target personal files (documents, downloads, pictures, configs). The command timed out after ~2 minutes, but the timeout only terminated the process — it did not undo prior deletions. AgentLoop's built-in rule flags rm -rf as a destructive command. It then correlates the user's task (clean test artifacts), the tool parameters, and the call sequence to determine whether the deletion target matched the authorized scope. Investigators must verify the actual value of $HOME in each call via the execution environment or other logs; the variable name alone is insufficient.

Custom Enterprise Rules

Beyond built-in rules, organizations can define custom detection rules targeting specific sensitive directories, business data, or operational constraints. A rule specifies what to inspect (model output content or tool parameters), hit conditions, exclusions for normal behavior, and a risk type (existing or new). Rules should be validated with both positive and negative samples before deployment. Enabled rules appear alongside built-in findings in a unified risk list, linked to their sessions for seamless investigation.

From Risk Finding to Investigation

AgentLoop's risk browser allows filtering by risk type, severity, application, and user. Each finding retains type, severity, reasoning, and links to the underlying events, session, and related entities (user, host, tool). Investigators open the session timeline to inspect model inputs, tool calls, and returns step by step. Sessions without findings can still be audited via fact queries for spot checks; absence of alerts does not guarantee safety.

Entity-Centric Investigation

After resolving an incident, investigators can pivot from a risky tool, sensitive file, or user to discover other sessions involving the same entity, assessing whether the issue is isolated or systemic. For example, if a tool accesses a sensitive path in multiple calls, each session can be reviewed for proper authorization and configuration.

Verification and Ongoing Monitoring

After adjusting permissions or tool configurations, teams can re-run tasks in a controlled environment and verify the new session records match expectations. Continuous monitoring of data source health and detection task status is essential to avoid silent gaps in coverage.

References: LoongSuite-Pilot event schema (GitHub), data masking docs (GitHub), Cursor CVE-2026-31854 advisory (GitHub), Claude Code issue #75859 (GitHub), AgentLoop audit overview (Alibaba Cloud), AgentLoop demo environment (Alibaba Cloud).