Data science for imbalanced data: Methods and applications

Abstract

Data science is a broad, interdisciplinary field concerned with the extraction of knowledge or insights from data, with the classification of data as a core, fundamental task. One of the most persistent challenges faced when performing classification is the class imbalance problem. Class imbalance refers to when the frequency with which each class appears in data is not roughly equivalent, and the problem is that introduction of class imbalance into the task of classification poses serious challenges that must be addressed in order to provide knowledge and insight. Yet, the challenges that class imbalance gives rise to are in part due to its ubiquitous prevalence, and it stands as a problem that pervades almost every area of investigation under which data science has provenance. In this dissertation, we investigate and counter class imbalance in several domains, touching upon areas as diverse as ecological informatics, scientific impact prediction, healthcare analytics, and education. By the end of this dissertation, we will develop and apply a variety of methods and techniques used to combat the class imbalance that is endemic to these and other domains. We will also compare and contrast the approaches we use to combat class imbalance with more traditional approaches typically employed within each domain. Before beginning, however, we present a preliminary overview of the class imbalance problem and the algorithms, methods, and evaluation metrics generally employed to address it.