Rage Against the Machine: Using Machine Learning Models to Predict Library Search Query Failure and Success Based on Search Filter Application

Abstract

The problem addressed by this quantitative study was the high rate of search failure when using online library discovery search due to flaws in design of the user interface or user error. Research has shown that failed library searches may lead to feelings of frustration as well as a lack of trust in the library and, more alarmingly, student attrition and withdrawal. This study used a quantitative non-experimental design underpinned by theoretical frameworks centered on analyzing behaviors, patterns, and trends associated with information search and retrieval systems and specific user groups including Ellis’ behavioral theory, Fitzgerald’s faculty information seeking theory, Roger’s diffusion of innovations theory, and Zipf’s principle of least effort. The study employed machine learning models and natural language processing text classification using a massive dataset of 3 million search query logs from National University library’s discovery search system EBSCO Discovery Service. The study used machine learning in tandem with transaction log analysis to determine the extent to which there is a relationship between the application of search filters and search success or failure. The purpose was to test four different machine learning models in their ability to predict library search query success or failure and identify how search filters influence success or failure of user searches when using online library discovery search systems. The four predictive machine learning models tested were naïve Bayes, gradient boosting, random forest, and decision tree. The findings revealed that the decision tree classifier demonstrated the highest predictive success of library search query success and failure, indicating it may be worth studying further as a foundational approach. However, due to the decision tree model’s inherent limitations in text classification accuracy and feature dependency, additional strategies could be used to refine and extend this approach. By employing predictive machine learning models and examining the role of search filters in query success and failure, this study highlighted the importance of data science and machine learning within the library science field. The findings aimed to contribute actionable insights for improving library resources, interfaces, and instructional support in university libraries, especially for online adult graduate students.