Open‑Source Breach & Attack Simulation Tools: CALDERA, Infection Monkey, Atomic Red Team

Introduction

Before internal security validation, a collection of seventeen open‑source security validation tools (covering BAS, cloud, endpoint, WAF, traffic detection, and adversary simulation) was gathered and tested. The evaluation focused on three aspects: test case library, execution/control, and data collection/analysis.

Breach and Attack Simulation (BAS)

BAS validates the capabilities of existing defense mechanisms by simulating real attacks against network systems.

CALDERA (Highly Mature)

CALDERA™ is a MITRE‑ATT&CK‑based platform that automates adversary emulation, assists manual red‑team activities, and supports automated incident response.

Basic Information

Frequent updates, high completeness, API integration.

Agent‑Server architecture with plugin extensibility.

Core framework includes a REST API and web‑based C2 server.

Key Components

CAMPAIGNS – Agents : Sandcat (Go), Manx (Go reverse shell), Ragdoll (Python).

Abilities : ATT&CK techniques implemented as commands, customizable.

Adversaries : Collections of abilities representing threat actors.

Operations : Control execution of adversaries on selected agents.

Plugins

Access, Atomic, Compass, Debrief, Fieldmanual, Manx, Sandcat, Stockpile, Training.

Configuration

Fact Sources, Objectives, Planners, Contacts, Obfuscators, Exfilled Files.

Typical Use Cases

Autonomous Red‑Team Participation : Deploy agents, select an adversary, and run an operation to test defenses.

Autonomous Incident Response (Blue Team) : Deploy blue agents, select a defender profile, and execute response operations.

Manual Red‑Team Participation : Use the Manx agent to run manual commands during assessments.

Infection Monkey 1.13 (Ransomware Simulation)

Infection Monkey simulates lateral movement and ransomware behavior across data‑center environments. It consists of a Monkey agent (Go/Python) and an Island server providing a GUI.

Features include easy installation, support for containers, public and private clouds, and detailed security reports (Infection Map, Security Report, Zero Trust Report, ATT&CK Report, Ransomware Report).

Atomic Red Team

Atomic Red Team™ is a community‑maintained library of ATT&CK‑mapped test cases that can be executed directly from the command line or via frameworks such as Invoke‑AtomicRedTeam.

Invoke‑AtomicRedTeam

A PowerShell module that imports and runs Atomic Red Team test cases, requiring PowerShell Core and providing commands to list, execute, and clean up tests.

ARTi‑c2

An ATOMIC‑based C2 framework; installation issues are noted, but it can be experimented with.

Purple Team ATT&CK Automation (MSF)

A set of Metasploit modules implementing ATT&CK techniques; last updated in 2020.

Metta (Legacy)

Uber’s Python‑based adversary simulation framework using Redis/Celery and Vagrant; no longer maintained.

RTA (Legacy)

Python‑2 scripts that emulate ATT&CK techniques for testing detection capabilities.

Cloud Security Validation

Stratus Red Team (AWS)

Simulates ATT&CK techniques in AWS environments using API calls; requires AWS credentials.

AWS‑Attack (Pacu Fork)

Modified Pacu framework that adds ATT&CK context to AWS exploitation scenarios.

Endpoint Detection and Response

Chain Reactor

Generates Linux executables from JSON‑defined reaction scripts to test detection coverage.

Web Application Firewall (WAF) Testing

Gotestwaf

Go‑based tool that tests API and OWASP attacks against WAFs, providing reports.

waf‑bypass

Python tool that checks WAF false‑positive/negative rates using predefined payloads.

go‑ftw

Go implementation of the FTW framework for testing WAF rule sets against the OWASP CRS.

Wafbench

Performance testing tool (similar to ab) with components for normal testing, WAF performance, and FTW‑compatible testing.

Traffic Detection

flightsim

Generates malicious network traffic (DNS tunneling, DGA, C2) to evaluate IDS/IPS visibility.

Egress‑Assess

Tests data exfiltration detection across multiple protocols (FTP, SMTP, HTTP, SMB, DNS, etc.).

Adversary Simulation Libraries

Community‑threats and the Center for Threat‑Informed Defense’s adversary_emulation_library provide extensive, continuously updated threat‑actor playbooks in various formats (Markdown, JSON, MITRE ATT&CK Navigator).