Machine Learning for Malicious URL Classification with Expanded Feature Selection and Natural Language Processing: A Temporal Analysis

Abstract

This praxis further investigates the research performed by Evan Wehr (2024), who argued that URLs change over time, and that when Machine Learning (ML) is applied to malicious URL classification, performance should decay over time. This means that ML performance should decay over time when applied to malicious URL classification. Wehr’s (2024) research does not include the use of natural language processing for malicious URL classification, to which this praxis extends.

Traditional approaches to ML model training and testing assume static datasets, neglecting the temporal dynamics inherent in URLs. By addressing this gap, the aim is to determine the effectiveness of incorporating natural language-based features in enhancing model performance and resilience to concept drift over time. The research performed demonstrates the potential improvements or shortcomings of including natural language processing in a temporal analysis over existing selections. To test the hypotheses, a dataset comprising of 2,292,882 URLs, one of the largest in this domain, was used. The temporal analysis revealed the presence of concept drift and indicated potential performance decay. Models resistant to such decay, such as XGB, LR, and NB, with normalization and standardization, exhibited the strong lasting power. This study underscores the importance of considering temporal dynamics and feature selection in designing robust ML solutions for malicious URL classification, providing valuable insights for security engineers to make informed decisions in safeguarding against evolving threats.