Content area
System provenance analysis has become the predominant approach for defending against sophisticated attackers. System provenance analysis captures causal and informational flow dependencies by correlating telemetry data across key system resources such as processes, files, and network sockets. These dependencies are efficiently represented as system provenance graphs, which are directed, heterogeneous, and multi-attributed. These system provenance graphs can be used by Provenance-based Intrusion Detection Systems (PIDSs) to train adaptive behavioral Machine Learning (ML) models for intrusion detection tasks. PIDSs can effectively thwart Advanced Persistent Threat (APT) actors and Fileless Malware writers since they can measure the program behavioral deviations. Graph Neural Networks (GNNs) are the de-facto standard for learning from graphs. Consequently, GNN-based PIDS can detect zero-day and mimicry attacks by measuring deviations in program behavior.
Despite their undeniable advantages, modern PIDSs still face several open problems: (1) current system provenance analysis techniques are designed primarily for resource-rich environments, leaving IoT ecosystems vulnerable; (2) the resilience of PIDS against dedicated adversaries have not been fully examined; (3) GNN-based PIDS operate as black-box models, lacking transparency in their detection decisions.
This dissertation addresses these three key challenges in system provenance analysis: extending provenance analysis to IoT environments, improving robustness against adversarial attacks, and enhancing the explainability of GNN-based PIDS.
First, we introduce ProvIoT, a federated edge-cloud security framework that brings PIDSs to resource-constrained IoT devices. ProvIoT leverages federated learning to minimize network and computational overhead while maintaining high accuracy in detecting stealthy attacks, even in diverse real-world environments.
Next, we present ProvNinja, an adversarial testing framework designed to evaluate the robustness of PIDSs against realistic evasive attacks. ProvNinja generates adversarial attack variants that closely mimic benign system behaviors, allowing it to effectively test the resilience of State-of-The-Art (SOTA) PIDSs. Our experiments reveal vulnerabilities in current security models, leading to reduced detection rates in realistic attack scenarios.
Finally, we develop ProvExplainer, an explainability framework for GNN-based PIDSs to provide interpretable, security-focused explanations. ProvExplainer projects the GNN’s decision boundaries onto the interpretable surrogate model’s feature space (e.g., discriminative subgraph patterns). By integrating with SOTA GNN explainers, ProvExplainer improves both precision and recall in explaining stealthy attacks (i.e., APTs campaigns and Fileless malware) detections, offering a transparent and verifiable tool for security operations.
Together, these contributions offer scalable, robust, and explainable security solutions for increasingly interconnected and vulnerable digital infrastructure.