Content area
As network attackers become increasingly autonomous, security operators will need machine-timescale defenses that can detect, analyze, and respond with minimal human intervention. Today, however, designing and evaluating autonomous network attack and defense strategies requires significant amounts of manual effort, limiting our ability to develop robust and autonomous network defenses.
This thesis accelerates the design and evaluation of autonomous systems, enables the creation of novel autonomous network attackers and defenders, and sheds light on network attack vs. defense capabilities. To realize these goals, we introduce: (A) HAL, a high-level abstraction layer that accelerates the design of autonomous network attackers and defenders; (B) MHBench, a system for generating multi-host environments to accelerate their evaluation; (C) Perry, a library of deception defenses to shed light on deception capabilities; (D) Incalmo, a novel LLM-assisted autonomous network attacker; and (E) a systems-level leaderboard to evaluate the interplay of attack vs. defense capabilities.
We verify that these systems lower the effort of designing and evaluating autonomous network attackers and defenders, and provide insights into their capabilities. For instance, we measure the reduction in effort from HAL through a lines-of-code (LoC) analysis and show how HAL reduced the LoC by 6—22.2x of autonomous attackers and defenders compared to prior state-of-the-art tools. As one example of a capability finding, we found that Incalmo, an autonomous LLM-assisted system for network attacks, can successfully attack 37 out of 40 environments in MHBench’s benchmark while prior systems only succeeded in 3.