Incalmo: An Autonomous LLM-assisted System for Red Teaming Multi-Host Networks
Brian Singer, Keane Lucas, Lakshmi Adiga, Meghna Jain, Lujo Bauer, and Vyas Sekar. In IEEE Symposium on Security and Privacy (S&P) 2026. [BibTeX]
In realistic enterprise settings, red teaming involves executing multi-host network attacks that span many “stepping stone” hosts, but red teams are expensive and entail significant expertise and effort. To date, the extent to which LLMs can autonomously execute such attacks is not well understood. We find that state-of-the-art LLM-assisted offense systems (e.g., PentestGPT, CyberSecEval3) with leading LLMs cannot autonomously execute multi-host network attacks. To enable them to, we built Incalmo, a high-level attack-abstraction layer: instead of having LLMs interact with low-level tools and commands, Incalmo lets LLMs plan red team exercises in terms of high-level declarative tasks (e.g., infect a host, scan a network) that are executed by domain-specific task agents, with auxiliary services to manage context and acquired assets. To evaluate it, we built MHBench, a multi-host attack benchmark of realistic emulated networks (from 22 to 50 hosts). Incalmo successfully acquires critical assets in 37 out of 40 MHBench environments, whereas state-of-the-art LLM-assisted systems succeed in only 3 out of 40.