1. Introduction
Electronic commerce dramatically boosted the quantity of information accessible to customers when it first began. In the digital age, consumers have become more informed about the products they buy. Armed with access to information about a wide range of related products or services, the behavior of consumers has shifted towards being less impulsive. Instead, getting a range of information on a wide range of products to make a more calculated purchasing decision has become the norm. This change in attitude makes attracting and retaining new customers one of the most critical difficulties confronting businesses in marketing today. While new businesses focus on obtaining new consumers, established businesses are more focused on retaining current customers to increase cross-selling opportunities. Customer satisfaction is one of the keystone concepts in strategic marketing [1]. Efforts have been made to enhance customer value, such as the 7Cs framework proposed by [2]. In several businesses, customer churn rates are a significant problem. Many studies have demonstrated that even a minor shift in churn rates may have a big effect on profits. Understanding client behavior in advance can provide businesses with a competitive edge.
Even though mobile phones account for over 75% of all potential phone calls worldwide, the mobile telephone market is one of the most rapidly growing segments of the telecom sector. For the same reasons that every other competitive market has witnessed a movement in the competitive landscape from customer acquisition to customer retention, retail has also seen a shift in the means of competition from customer acquisition to customer retention. In the telecom industry, churn refers to a company losing customers to other service providers [3]. Like many businesses that work with long-term clientele, telecom firms utilize customer churn research as one of their primary business indicators [4]. According to a survey presented by [5], 30–35% of clients leave their telecom company annually post-COVID. This churn rate may continue to rise with market growth or the emergence of new, big telecom players in the future. In addition, for cellular corporations, client acquisition costs can be comparable to 5–10 times the amount spent on customer retention or satisfaction costs [6].
Therefore, it is essential to learn the reasons why customers leave to reduce the harm churn has on a business’s bottom line. Predictive marketing analytics and churn analysis can help identify factors influencing consumers’ voluntary churn by using advanced machine learning algorithms [7]. By investigating the existing studies, it was discovered that ensemble learning, including classical machine learning algorithms like Random Forest, Decision Trees, and Naïve Bayes [8,9,10], and deep learning methods, such as particle-classification-optimization-based BP networks [11] and Deep-BP-ANN [12], have been employed to improve the accuracy of churn prediction.
However, the models should not only focus on accuracy in predicting churning [9] but also be comprehensible, which means it should provide reasons for churning so that experts can validate its results and check that it predicts intuitively and correctly. If the company had understandable and transparent models to work with, it would increase the understanding of what was causing the churn and how to enhance customer happiness to boost retention.
As a result, this research aims to construct an accurate, efficient, responsible, and explainable prediction model for customer attrition in the telecom industry using ensemble learning methods. Various data mining methods like Decision trees, Random Forests, and Logistic Regression were utilized by experts to build the predictive model. The performance of the models was evaluated using the accuracy measures, area under the curve, and sensitivity and specificity measures. In addition, this study also aims to improve the interpretability of customer churn predictive models through the use of LIME and SHAP to provides decision-makers with an overall explanation of the factors affecting the customer’s decision to churn, as well as a specific analysis for every single customer. Following a brief introduction to customer churn amplification in the telecom business, the existing literature is reviewed and summarized in detail.
The remainder of this paper is structured as follows: Section 2 focuses on reviewing the contribution of predictive marketing to Customer Relationship Management, different machine learning-based churn analysis models for the telecom industry, as well as the explainable AI. In Section 3, the most acceptable attributes are determined, and the techniques used in this study are introduced. Section 4 is concerned with the research analysis and findings. Finally, the conclusion and several alternate interpretations are presented in Section 5.
2. Literature Review
The current section is divided into three distinct components. The first section will introduce Customer Relationship Management (CRM) with its core ideas. The next part discusses the most well-known and significant ensemble learning models in CRM, which were also employed in the model design phase of this examination. Finally, the third segment discusses current research on the importance of explainability and transparency of the methods used in this regard.
2.1. Customer Relationship Management and Predictive Marketing
Customer Relationship Management (CRM) started gaining popularity in the late 1990s through work conducted by American tech companies such as IBM and Gartner. There are four main types of CRM—strategic, operational, analytical, and collaborative—which all aim to manage the company’s relationship with both potential and current customers:
Strategic CRM, which is the use of customer data through systematic analysis as a means of marketing management [13];
Operational CRM, which supports company operations whereby information on employees, customers, and leads is stored;
Analytical CRM, which is a strategy whereby customer and market information is analyzed to aid the decision-making process for business management [14];
Collaborative CRM, which allows for multi-way communications between companies and their customers, with the goal of facilitating more profitable retention of customers [15].
The primary goal of CRM is to maximize customer retention while attracting a steady stream of new customers to maximize the company’s revenue streams [16]. By adopting certain CRM methodologies, companies can improve their understanding of how to engage and interact with their customers to form mutually beneficial relationships through collaboration [17].
Utilizing information technology and information systems in CRM is becoming increasingly important. More specifically, modern data analysis techniques combined with the increased amount of available data have made this approach particularly attractive to businesses. It is critical, therefore, for companies to adopt such practices to achieve innovative CRM capabilities. Machine learning has emerged as one of the most influential and popular tools to increase the effectiveness of any CRM strategy. The main four dimensions [18] across which ML is commonly applied in the context of CRM are customer identification (target customer analysis and customer segmentation), customer attraction (marketing), customer retention (loyalty program analysis, one-to-one marketing, complaint, and conflict management), and customer development (customer lifetime cycle analysis, upselling and cross-selling, the market basket analysis).
Big data analytics is transforming businesses by transferring the focus from products and channels to the customer, with the aim of maintaining personal relationships and moving from mass marketing to highly personalized marketing. Predictive analytics, used to transform raw data into useful information, has great relevance for marketing purposes, allowing the prediction of customer behavior and allocation to specific groups of customers. By using innovative instruments, companies can adopt big-data-driven, micro-targeting marketing practices, which permit the improved precision of segmentation and targeting.
Predictive marketing is a powerful tool that can greatly enhance a company’s CRM capabilities. One of the predictive marketing methods is cluster analysis. Cluster analysis algorithms can manage vast amounts of data to identify elements that characterize certain consumers and find correlations that are difficult to identify manually. Conjoint analysis is also a method adopted in predictive marketing for CRM. It is a research technique used to identify customer preferences by identifying relevant attributes for consumers in the selection and purchase processes. The outputs of conjoint analysis often guide business decisions on new products and promotions. Moreover, online reviews have a strong impact on consumer choices, and sentiment analysis or opinion mining can potentially predict future sales and assist marketing strategy. Sentiment analysis is important for a better understanding of consumer preferences, which leads to the identification of precise targets and better advertising strategies. Eachempati et al. [19] conducted sentiment analysis to analyze the effect of the emotions expressed on social media platforms by Indian customers regarding automobile companies’ stock prices. They found that customer sentiment is a strong factor contributing to the stock price as well as the corporate value. In addition to the classification algorithms for cluster analysis, Lamrhari et al. [20] also employed cluster analysis and k-means algorithms to group customers into different clusters for the purpose of extracting insights from social media data. Random Forest outperformed the other models used with 98.46% accuracy and K-means helped to realize the eWoM communication balance.
While gaining new clients is necessary for a company to develop, customer retention should not be disregarded. Companies employing CRM strategies should take particular care to analyze the factors contributing most to improving customer retention rates. According to [21], such key factors involve the interplay between relationship management practices, such as customer trust, employee commitment, and conflict handling. However, it is also noted that more research needs to be conducted in these areas. Predictive analytics not only enhances segmentation techniques but also enables the implementation of churn analysis, which predicts the likelihood of customers leaving the company for competitors. This information helps the company take initiative-taking measures to prevent customer churn. Predictive analytics algorithms can estimate the likelihood of customers switching to competitors and identify the factors that contribute to that probability.
2.2. Customer Churn Prediction Review
Customer churn prediction (CCP) plays a crucial role in this research, since it can contribute to the impact of customer retention on business profitability and growth. Well-known papers have been studied and summarized as follows.
Researchers [16] have investigated the understanding of nuanced behaviors leading to customer churn in the telecommunication industry. Their combined approaches, using temporal centrality metrics and data analytics, have provided insights into early churn indicators and the role of customer lifecycle stages in influencing churn tendencies. Their emphasis on leveraging advanced predictive analytics for proactive churn management underscores the importance of timely interventions and personalized customer outreach.
De Caigny et al. [22] focused on developing their convolution neural network (CNN) techniques in CCP and compared their approach with current practices of text data analysis via text mining. They have found that CNNs outperform the current practices and unstructured data in text data prevent them from achieving a high accuracy. Their approach can extract the most valuable features from the unstructured textual data, thus achieving more reliable prediction with higher accuracy. Additionally, they [23] have focused on business-to-business (B2B) customer retention and analyzed 6432 customers. They developed the uplift Logit Leaf model (LLM), which can achieve better performance in CCP than competing models. By integrating behavioral economics principles with their LLM model, they have paved the way for more human-centric churn prediction strategies.
Seymen et al. [24] proposes a deep learning model for predicting whether retail customers will churn in the future. The results showed that the deep learning model achieved better classification and prediction results than the Logistic Regression model and an artificial neural network model. Deep learning models have been proved to be effective in a variety of domains, including agriculture [25] and stock price forecasting [26]. Researchers [27] blended deep learning and natural language processing (NLP) to analyze 25,943 customer survey data. Their blended model can identify patterns, which can lead to improved decision-making and improve accuracy in their CCP analysis. Therefore, the use of ML algorithms can be effective in prediction robustness.
2.3. Customer Churn Prediction Model for Telecom Industry
Ensemble learning contains the use of multiple ML models to achieve improved performance and accuracy [28,29] and also provides more independent views, so that decision-making can be better and more accurate [30]. Ensemble-based ML classifiers have recently emerged as a new way of building ML models. Since models have different strengths and weaknesses, researchers have started building hybrid models consisting of two or more models. Mishra and Reddy [31] took this approach and applied ensemble models, such as Random Forest, Decision Trees, bagging algorithms, and boosting algorithms, to predict customer churn in the telecom industry. They compared the results of these ensemble models with more classical models such as Naïve Bayes. From their results, they found ensemble classifiers performed favorably, with Random Forest returning the highest accuracy of 91.66% and all other ensemble classifiers returning results above 90%.
Researchers [19] used a particle classification optimization-based BP network for telecom customer churn prediction. The PBCCP algorithm is based on particle classification optimization and particle fitness calculation. In this case, particles refer to vectors containing the thresholds and weights within a BP neural network. The particles are classified into categories using their fitness values and are updated using distinct equations. They found that increasing the number of layers can improve the performance of the algorithm at the cost of training time. As a result of this, they opted to use one hidden layer in the neural network. They used a balanced dataset made up of 50% churn customers and 50% non-churn customers, resulting in the PBCCP network returning an overall accuracy of 73.3%. By comparison, the PSO-BP network returned an overall accuracy of 69.6% and the BP model had 63.6%.
Similarly, researchers [9] devised the Logit Leaf model, an ensemble model using aspects of Logistic Regression and Decision Trees, and compared their results with standard Decision Tree, Logistic Model Tree, Logistic Regression, and Random Forest models. The AUC performance criteria were one of the metrics used to evaluate performance—a metric commonly used for evaluating the performance of binary classification systems, such as customer churn prediction. In this regard, the Logit Leaf model performed the best, slightly better than Random Forest. The same was true when the models were evaluated using the TDL (10%) performance criteria. The Logit Leaf model was also more efficient at making predictions, taking less time than the other models used. When real-time predictions are required, this can be a crucial factor when deciding which model to use for a given problem.
Finally, scientists [10] applied gravitational search algorithms to perform effective feature selection, resulting in a dimensionality reduction in the dataset used for customer churn prediction. After this, they applied a selection of ML models for comparison: Logistic Regression, Naïve Bayes, Support Vector Machine, Decision Trees, Random Forest, and Extra Tree Classifiers and boosting algorithms such as Adaboost, XGBoost, and CatBoost. They found that, when comparing the models used, the boosting algorithms performed the best, with CatBoost achieving the highest accuracy, recall, and precision with scores of 81.8%, 82.2%, and 81.2%, respectively. When comparing their AUC scores, boosting algorithms performed the best again, with XGBoost and Adaboost scoring the highest at 84%.
One of the most important applications of ML with regard to CRM is to give companies the ability to predict customer churn. This is particularly important given that retaining existing customers is significantly more valuable than acquiring new customers, as more resources are needed for new customer acquisition. It follows, therefore, that customer retention is one of the key priorities of the CRM strategy [32]. The ability to harness ML to significantly improve customer churn rates is of particular importance.
In this study, we will make use of predictive machine learning models to identify clients who are likely to churn after categorizing current customers using clear machine learning.
2.4. Explainable Artificial Intelligence (AI) in Churn Analysis
While the application of machine learning algorithms has brought many benefits to business, it has also revealed several drawbacks. For example, in Customer Relationship Management, it is essential that every decision is meaningful. Advanced data analytics should enable decision-makers to understand the reason underlying the model’s prediction of customer behavior results so that they can tailor their decisions and adjust personalized marketing strategies accordingly [4]. Decision-makers are often wary of or even reject AI systems because the performance of the predictive model or system is overemphasized while interpretability or transparency is ignored [33].
Many popular ML models are considered as “black boxes” [7,34], meaning that the inner workings of the algorithms and associated decisions or classifications made are secretive. For example, individual predictions often lack interpretability when predicting a potential customer’s risk of discontinuing a telecommunications service. It is difficult to relate the predicted probability of customer churn to customer characteristics, which creates challenges in determining customer retention success and deciding on refinement strategies. Additionally, from a legislative perspective, the interpretability of AI algorithm conclusions is itself a right of consumers and regulators have a responsibility to ensure this right. Studies have discussed data transparency and ethical issues from the perspective of users or data subjects [35]. To address this problem, explainable AI (XAI) models are required to provide details or interpretations that make the operations of AI understandable.
According to [36,37], explainable AI enables interested parties to comprehend the primary influences on model-driven decisions as well as how AI algorithms carry out their operations, forecast outcomes, and make decisions. The European General Data Protection Regulation (GDPR) emphasizes that intelligent decisions should be accompanied by relevant information about the logic involved and the significance and possible effects of their processing for the data subject. Therefore, the GDPR grants data subjects the right to obtain pertinent information regarding the basis for smart decision-making [38].
A few XAI approaches have been used for customer churn predictive models in the existing literature, including global surrogate models, Partial Dependence Plots (PDPs), Accumulated Local Effects (ALEs) [4], LIME [39], and Shapley values [40,41].
In the work of [4], a number of XAI methods were employed to explain the customer churn predictive model by using a dataset of the telecom industry. The discussed global explanation methods included Partial Dependence Plots (PDPs), Accumulated Local Effects (ALEs), global surrogate models, and Shapley Valuea, as well local methods, including Individual Conditional Expectation (ICE) and the local surrogate model (LIME). The results show that a thorough understanding of the data, the inner workings of the model, and the issue can be addressed by the combination of several interpretations and approaches in a specific way. Scientists [42] proposed a novel XAI framework to identify the most significant factors that influence a customer’s decisions on purchasing or abandoning non-life insurance coverage. Their framework applied similarity clustering to the Shapley values from a predictive model created by XGBoost. The results showed that the integration of the clustering technique and Shapley values effectively grouped the customers, which was used to forecast the churn decisions of customers. Researchers [7] provided an XAI solution for predicting customer churn and outputting its explanations through visualization. This system integrates two major methods, the Random Forest algorithm used to train a churn classifier and Shapley values used to provide global and local explanations for the churn classifier. Using LIME and Shapley techniques, Ref. [39] interpreted customer churn predictive models constructed using Random Forest and light-gradient-boosting machines from local and global perspectives, respectively.
Personalized marketing initiatives, especially those related to managing customer churn, are essential for companies looking to establish and uphold strong customer relationships. The use of predictive analytics methods undoubtedly enhances the efficacy of these endeavors [43]. For better customer churn prediction, various emerging ML algorithms have been employed in the existing literature and have performed well in terms of accuracy. Nevertheless, decisions produced via ML-based customer churn predictive models are not easily understood by managers of customer relationships because of the black-box nature of artificial intelligence. Further research in this area is nascent. Therefore, the aim of this research is to contribute to the literature by improving the interpretability of customer churn predictive models through the use of XAI interpretations to optimize their local and global interpretability.
3. Research Methodology
The telecommunications sector has long struggled with churn. This research aims to create and deploy a cost-effective system for predicting client churn in the telecommunications sector. Addressing this issue is expected to yield a deeper comprehension of churning customers, enabling the identification of such customers and providing a foundation for future initiatives aimed at reducing the sector’s churn rate. Section 3 discusses both the research approach selected and the machine learning models adopted.
3.1. The Crisp Model
The CRoss-Industry Standard Process for Data Mining (CRISP-DM) model is referred to as a standardized way of obtaining a good process via data mining across businesses and industries where data and modeling are a priority [44]. Researchers advise that, after twenty years of developing CRISP-DM, the emphasis is on data science and the methodologies should accommodate the need for data release, data architecting, data simulation, and data acquisition [45]. Business processes and demands can be centered based on the data-driven approach. In other words, we can check work progress, evaluate our outputs, and make decisions in real-time. By doing so, our efficiency and accuracy in our tasks can be significantly improved.
Thus, CRISP-DM models are an apparent methodological way of directing the research’s procedure. The process diagram of the CRISP-DM model is depicted in Figure 1 below.
We first define a specific business problem, i.e., customer churn in the telecommunication industry, and set explicit goals to mitigate this problem through predictive modeling. Following the CRISP-DM framework, we collected and preprocessed customer data from the telecom industry, focusing on the characteristics that may indicate customer churn. Throughout the modeling phase, we applied various data mining techniques such as Decision Trees, Random Forests, and Logistic Regression. After modeling, we evaluated the performance of the models using rigorous metrics such as accuracy and area under the curve to ensure that they meet the operational requirements of the business environment. Finally, we translated the findings from the models into actionable strategies aimed at customer retention.
3.2. Dataset
The dataset used in this study is a publicly available, large dataset. The aim was to find a sufficiently large and recent dataset regarding churn within the telecommunication industry. The dataset in question was selected based on meeting certain search criteria—it could meet a representative size and was presumed to contain relevant explanatory variables such as demographics. Finally, a Telecom CUSTOMER Churn dataset from the Maven Analytics website platform was selected. It consists of customer activity data (features), along with a churn label specifying whether a customer canceled the subscription, which we used to develop predictive models. The dataset consists of 7043 rows and 38 attributes. The attributes consist of different pieces of information about the individual customer, including the status of the customer’s subscription, which is categorized as churn and not churn.
A high-quality dataset is required for further analysis; therefore, this study examined each variable in the dataset for missing values. In order to select a predictive model with optimal predictive accuracy, the research data was divided into two groups—the Train (75%) and Test (25%) datasets—by putting into consideration the great ratio of 2:1 (non-churner: churner) based on Wei and Chiu ’s [31] research. After testing data quality and selecting variables, none of the observations or rows were deleted.
3.3. Machine Learning Algorithms
Different machine learning techniques were adopted to create a customer churn prediction system. These techniques were adopted to find the best possible way to predict customer churn through a model with high predictive accuracy. The five techniques used were the Logistic Regression model, the Random Forest classifier, the Naïve Bayes classifier, The K-Nearest Neighbor classifier, and the Decision Tree Classifier. All the models and their functions are described below.
Logistic Regression: Logistical models attempt to create a regression model based on data with the binary response variable. These models can be used, among other things, to estimate population groups where a statement may be true or false, such as churn or non-churn. In Logistic Regression, the logit function is used to determine the probability of a binary outcome.
K-Nearest Neighbor (KNN) Classifier: KNN is a non-parametric algorithm and a lazy-learner algorithm, meaning it does not make any assumptions about the underlying data and does not immediately learn from the training set until new data is obtained. The KNN algorithm assumes that the new data is similar to the existing data and assigns the new data to the category that is most similar to the existing category based on the distance between two points.
Naïve Bayes Classifier: This procedure is based on Bayes’ Theorem and the assumption that the predictors are unrelated. In contrast to other models, the Naive Bayes model is straightforward to create and is particularly successful when dealing with large datasets. Additionally, it is easy to use.
Decision Tree: Decision Tree learning is a fundamental technique in decision theory. These trees comprise a root at the top and knots that are interconnected by branches. Nodes can be classified as either internal or terminal. At each internal node, a specific attribute is tested, and the result guides the selection of different branches, eventually leading to a terminal node. The terminal nodes, or “leaves”, correspond to a classification [46].
Random Forest Classifier: A Random Forest is a combination method that works with Decision Trees as building blocks. The algorithm generates a predefined number of trees and takes a cut of the total number of trees and uses it as its predictor [47]. In the task, the Random Forest model can be adjusted to achieve the best possible performance by adjusting the number of trees and the number of cuts used in the algorithm.
3.4. Model Performance Evaluation
In classification problems, the following measures are used to assess the performance of ML models.
3.4.1. Confusion Matrix
A Confusion Matrix is a well-known way of examining how a good classification model achieves the observed classes. The Confusion Matrix is presented in Table 1 as follows.
The four boxes in the classification table are all assigned a name: FN, FP, TP, and TN.
TN stands for true negative. Here, the customers are observed as not being churners, and the model has also classified the customers as non-churners.
FP stands for false positive. Here, customers are observed as being non-churners, but the model has classified the customers as churners.
FN stands for false negative. Customers are observed as being churners, but the model has classified the customers as non-churners.
TP stands for true positive. Here, customers are both observed and classified as churners.
The Confusion Matrix is not only a visually advantageous way of measuring the model’s ability to classify correctly. Several calculations can also be made based on the four values. These calculations are all targets for more specific evaluations of the model and can, therefore, be used to identify the model’s strengths and weaknesses.
3.4.2. Four Evaluation Metrics
This section describes four metrics used to assess the performance of churn prediction models, including accuracy, error rate, specificity, and sensitivity.
Accuracy: Accuracy is the proportion of correctly classified observations out of all customers classified. This is an overall score, which counts on the overall performance of the model. Accuracy is determined as follows:
(1)
Error rate: As opposed to accuracy, the error rate is the proportion of incorrectly classified observations out of all customers classified. The error rate is calculated as follows:
Error rate = (FP + FN)/(TP + TN + FP + FN). (2)
Specificity: This is the true negative rate. The rate, thus, indicates how large a proportion of the customers estimated as non-churners are correctly classified. Specificity is determined as follows:
Specificity = TN/(TN + FP). (3)
Sensitivity: This is the true positive rate. The rate indicates what percentage of the customers were estimated as churners and classified correctly. Sensitivity is calculated as below:
Sensitivity = TP/(TP + FN).
3.4.3. Other Evaluation Metrics
ROC stands for Receiver Operating Characteristic and is a visual representation of the ability to classify correctly. The graph shows the relationship between the false positive rate on the x-axis and the true positive rate on the y-axis. Three different ROC curves are shown in Figure 2.
The ROC curve in the graph to the left indicates a model that correctly predicts 50% of the time and, therefore, also indicates random classification. The graph in the middle indicates a model with improved predictive ability but that still leads to misclassification. In the graph to the right, the ROC curve indicates that the model classifies close to perfect, as the curve is very close to the upper left corner.
AUC stands for area under the curve and measures the area under the Receiver Operating Characteristic (ROC) curve. It is one way to obtain a more accurate measurement of the model’s performance. A perfect model will have an AUC of 1, after which the model’s quality decreases as the AUC value decreases.
3.5. Explainable AI Techniques
Due to the black-box nature of machine learning algorithms, the operating principles of the algorithms are difficult to understand and cannot be easily explained to decision-makers in the telecom industry, especially if they do not have a computer science background. As a result, although ML algorithms excel in terms of accuracy, they may not gain the trust of decision-makers or users or be accepted in real-world enterprise management. While aiming to address this issue, after evaluating the classifiers, this paper introduces the concept of explainable AI in the study and illustrates the results of the best customer churn prediction models through two visualization tools, including LIME and SHAP.
First, Local Interpretable Model-Agnostic Explanations (LIMEs) are employed to optimize the local interpretability of the Decision Tree classifiers and Random Forest classifiers. It emphasizes training locally interpretable models that may be used to explain specific predictions and help decision-makers understand why a particular class was predicted for a certain instance [46].
However, the purely local character of the LIME explanation, which only predicts how the present prediction will change in response to minute changes in the input values, is one of its limitations. As a result, rather than serving as an interpretation of why the forecast was made in the first place, it acts more like a sensitivity study. Therefore, this study also employs another explanation technique, SHAP, to show the overall importance of the factors.
The SHapley Additive exPlanation (SHAP) is a method of interpreting the results of any ML model by attributing an importance score to each feature in the data [48]. In this study, summary plots are drawn to allow visualization of the importance of features and their influence on prediction. The features are ranked according to the sum of the SHAP values of all samples. The colors indicate the high and low SHAP values of the features, i.e., blue indicates low importance and red indicates high importance. It makes use of the SHAP values to show the impact distribution of each feature as well.
4. Analysis and Results
This section will present the results from the application of the machine learning model described in Section 3 of this paper to the research dataset, as well as the model evaluation procedure. The initial step of the analysis section involves the descriptive statistics, which will help to build an understanding of the data for the subsequent analysis. The second area consists of, Section 4.2 and Section 4.3, showing different but related results and analysis. The section will conclude with a review of the variable and the machine learning technique that provides the best algorithm to predict customer churn.
4.1. Descriptive Statistics
To develop a better understanding of the attributes, it is required to construct summary statistics for all the features contained in the dataset used in this study. Table 2 shows the outcomes.
From the over 4601 customer samples used in this research study, the number of referrals is 1.95, with an SD of 2.96, which means that, on average, each customer referred this telecom company to two friends. The average download volume in gigabytes to the end of the second quarter is 26.12 GB, and the customer’s total charges for additional data downloads for the same quarter is 8.94, with an SD of 28.62. Table 2 also shows details of the customers’ charge history, where the average values are 3042.59, 2.16, 8.94, and 888.91 for total charges, total refunds, total extra data charges, and total long-distance charges, respectively.
The bar chart presented below was computed to examine the distribution of the target variable across the customer service call variable. It can be observed from the chart that the dataset is unbalanced, with almost twice as many churning samples than not churning samples. See Figure 3.
To examine the inter-correlation among the features, a correlation matrix was computed and displayed in the form of a heatmap to indicate the pairwise relationship among the call activities and features of the telecom customers. See Figure 4.
4.2. Results Based on Confusion Matrix
Customer churn is a critical issue for telecom companies, as it can significantly impact their revenue and profitability. Therefore, accurate prediction of customer churn is an important task in the industry. This study applied five different ML algorithms to predict customer churn in the telecom industry, including Logistic Regression, KNN model, Naïve Bayes, Decision Tree, and Random Forest. Their performance was first evaluated using a Confusion Matrix, which consisted of values of true negatives (TN) and true positives (TP), false negatives (FN) and false positives (FP). As stated in Section 3.4.1, TNs are cases where the actual negative is also predicted to be negative, while TPs are cases where the actual positive is also predicted to be positive. FNs are cases that are positive but predicted to be negative, while FPs are cases that are actually negative but predicted to be positive. See Figure 5.
A clustered bar chart was drawn to illustrate the performance of these algorithms in terms of these four metrics. Random Forest appeared to have the highest number of TP and TN and the lowest number of FN and FP, indicating that it may be the best-performing model of the five. It can be observed that 641 customers from the test dataset are correctly classified as non-churner, 109 of the customers who are non-churners are incorrectly classified as churners, 88 of the customers that are churners are misclassified as non-churners, and 670 of the customers are classified correctly as churners. The prediction accuracy of the Random Forest algorithm was computed to be equal to 0.8694. This implies that 86.94% of the customers are correctly classified by the Random Forest model. Random Forest is effective in identifying both customers at risk of churn (TP) and those unlikely to churn (TN), while avoiding misclassification of non-churning customers, which can lead to customer dissatisfaction, or churning customers as non-churning and missed opportunities to take action to retain them, leading to loss of profit.
In terms of the TP metric, Decision Tree was the second-best choice after Random Forest, indicating that it was excellent at identifying customers at risk of churn (TP: 614). However, its relatively lower TN values of 593 compared to the other algorithms indicate that it performs poorly in identifying non-churning customers. Furthermore, its relatively low FN values (144) and high FP values (157) confirm that the Decision Tree has a tendency to classify customers as those at risk of churn. Therefore, telecom companies may want to carefully assess the accuracy of the algorithm’s positive predictions before taking any retention action.
Naïve Bayes and Logistic Regression performed similarly with moderate levels of performance, with high numbers of true negatives (TNNB: 617; TNLR: 555) and true positives (TPNB: 587; TPLR: 584) as well as high numbers of false negatives (FNNB: 171; FNLR: 174) and false positives (FPNB: 133; FPLR: 195). This suggests that these algorithms can effectively identify customers who are likely to churn (TP) and those who are unlikely to churn (TN), but it may also misclassify some customers as either those who will not churn (FN) or those who will churn (FP).
The KNN model is the worst performer of all the algorithms. Although it performs well in identifying non-churning customers, as evidenced by the high number of TNs (635 customers), it performs rather poorly in identifying customers at risk of churning (TP: 440 customers). In addition, it has a high number of FN (318 customers) and a relatively low number of FP (115 customers) compared to the other models, which could indicate a tendency to misclassify positive instances (i.e., customers who will churn) as negative (i.e., customers who will not churn). This could potentially lead to missed opportunities for telecom companies to take retention action.
The area under the Receiver Operating Characteristic (ROC) curve is a metric widely used to assess the overall performance of binary classification models. The results of AUC-ROC for all algorithms tested in this study are shown in Figure 6. Random Forest had the highest score of 0.95, followed closely by Naïve Bayes, with a score of 0.88. These two models were the most effective at predicting customer churn in the telecom industry in this study. Logistic Regression also performed well, with an AUC score of 0.84, indicating that it is a reliable algorithm for predicting customer churn. KNN and Decision Tree had lower AUC scores of 0.81 and 0.8, respectively, suggesting that they may be less accurate in distinguishing between the customers at risk of churn and customers who will not churn. See Figure 6.
In summary, from the analysis of the Confusion Matrix and AUC-ROC curve, Random Forest was the most effective model for automatically detecting customer churn risk for telecom companies. The performance of the five models will be further evaluated in the next section.
4.3. Results of Analysis and Discussion
This study aims to identify the system that produces good predictions on customer churn through machine learning models. Therefore, it is necessary to compare the models selected in this study and identify the model with the best predictive ability. Presented in Table 3 are the model evaluation criteria considered in this study.
The accuracy measure is the ability of a machine learning model to make a correct prediction. As observed from Table 3 above, the machine learning models based on Naïve Bayes, Decision Tree, and Random Forest algorithms computed high prediction accuracy, as they computed well with around 80% or above accuracy measures. The Random Forest classifier computed the highest accuracy (86.94%) measures with around 7% points better than the decision tree model, which computed an 80.04% accuracy measure. Figure 7 depicts a graphical representation of the model evaluation metrics.
4.4. Model Interpretation
According to the model comparison results, the best customer churn prediction model was created from the random forest algorithm, which had a prediction accuracy of close to 90%. After that, to address with the black-box nature of machine learning algorithms, this paper illustrates the results of customer churn prediction models through two explainable AI tools, including LIME and SHAP.
4.4.1. Local Interpretable Model-Agnostic Explanations (LIME)
Figure 8 shows the local explanations of the Random Forest classifier for the first sample and the second sample using the LIME technique. According to Figure 8, the five most significant factors in Random-Forest-based churn detection for the first customer are the ‘Contract’, ‘Number of Dependents’, ‘Online Security’, ‘Premium Tech Support’, and ‘Number of Referrals’. Furthermore, an increase in the value of these five factors will drive the prediction toward not churning, while an increase in the value of ‘Payment Method’ and ‘Monthly Charge’ will increase the likelihood of churning. For the second customer, ‘Monthly Charge’ was more important than ‘Premium Tech Support’ and was one of the top five factors that influenced a customer’s subscription decision. In addition, contrary to the first customer, LIME indicates that an increase in the value of ‘Contract’, ‘Number of Dependents’, and ‘Number of Referrals’ values can increase the likelihood of the second customer’s churn. See Figure 8.
However, the LIME explanation only predicts how the present prediction will change in response to minute changes in the input values. Therefore, this study also employs another explanation technique, SHAP, to show the overall importance of the factors. The results are shown in the following subsection.
4.4.2. SHapley Additive exPlanation (SHAP)
The summary plots for the random forest classifier computed using the SHAP technique are shown in Figure 9. In terms of the overall importance of the features, ‘Contract’, ‘Number of Referrals’, ‘Tenure in Months’, ‘Monthly Charge’, and ‘Online Security’ are the five factors that make the most contribution to the prediction outcomes of the Random Forest classifier. To be specific, high values of ‘Contract’, ‘Number of Referrals’, ‘Tenure in Months’, and ‘Online Security’ increase the possibility of customer churn predicted by the classifiers, while the low value of ‘Monthly Charge’ increases the possibility.
In addition, to provide a global explanation, SHAP values can also be used to explore the prediction made for an individual sample. Therefore, this paper computes the SHAP explanation for the same two samples, as investigated using LIME in Section 4.4.1, and conducts a comparison.
According to Figure 10, the most significant factors for the first customer sample that push the prediction toward churning are ‘Payment Method’, ‘Age’, and ‘Monthly Charge’, and the ones that push the prediction toward not churning are ‘Contract’, ‘Online Security’, ‘City’, and ‘Number of Referrals’. Although ‘Age’ is not included in the top 10 groups of LIME, LIME and SHAP are able to corroborate each other’s judgment about the significance and direction of the factors for the first sample. For the second customer sample, as shown in Figure 10, SHAP shows that the ‘Number of Referrals’, ‘Contract’, ‘Number of Dependents’, and ‘Monthly Charge’ are the most significant factors that push the prediction toward churning, which is consistent with LIME explanation as well.
In summary, the results of the model explanations presented by the LIME and SHAP techniques, respectively, are consistent. By showing decision-makers in the telecom industry how customer-related factors affect customer retention through these interpretation figures, decision-makers can understand how AI makes predictions in a way they understand, even if they do not understand the complex algorithms of AI.
5. Discussion and Recommendations
5.1. Relevance to This Emerging Area and Its Contributions
Due to the black-box nature of machine learning algorithms, it may be difficult for decision-makers in any industry to comprehend the advanced customer churn system and may, therefore, lead to the systems being rejected. While aiming to address this issue, this paper also introduces the concept of explainable ensemble learning, with the use of ensemble-based machine learning and innovative ensemble learning, into business analytics and illustrates the results of the best customer churn prediction model through two visualization tools, including LIME and SHAP.
We implemented multiple classifiers including Logistic Regression, Random Forests, Naïve Bayes, Decision Trees, and KNN to compare ensemble techniques like bagging and boosting against individual methods [49]. The ensemble approach of combining diverse models clearly provided superior predictive performance over any individual learner for this key business analytics challenge.
Consequently, the developed predictive system is sufficient for distinguishing churners and non-churners and helps business developers in the telecommunication industry to conduct a more efficient campaign for customer retention, which will help to reduce marketing costs and churn rate effectively. The use of analytics and explainable ensemble learning can deliver critical insights effectively to all customers, stakeholders, and employees. Therefore, it makes decision-making more streamlined and straightforward, and the employees and other stakeholders can revise their work focus.
5.2. Implications Relevance to This Special Issue
This research aimed to develop a predictive model for customer turnover in the telecommunications industry that could distinguish between customers who are likely to churn in the near future and those who are loyal to the company. The value of such a strategy to a firm is that it avoids wasting money on useless mass marketing approaches and allows organizations to target true churners by removing customers who are likely to churn. Furthermore, as explained in Section 1, obtaining a new customer costs eight times as much as keeping an existing one; as a result, because the churn predictive model can predict future churners, businesses seeking to keep their clientele can concentrate on retention strategies rather than acquisition strategies, which are less expensive.
In addition, this study also improves the interpretability of customer churn predictive models through the use of LIME and SHAP to optimize their local and global interpretability. It provides decision-makers with an overall explanation of the factors affecting the customer’s decision to churn, as well as a specific analysis for every single customer. As a result, decision-makers can more easily understand the results of advanced predictive models and use these explanations to develop global strategies and customize strategies for specific customer segments.
The conclusions of this study have significant consequences for telecommunication companies. Apart from the notion of establishing a predictive system for predicting customer turnover in the telecommunication industry, the findings of this study may be applied to the banking industry in terms of developing a churn prediction model for debit card customers.
To generalize the previous discussion, we may advocate that companies employ machine learning approaches to transform existing consumer information in their databases into meaningful information that can assist them with their efforts to market based on the research results. They would also benefit from using machine learning to create a projected churn model, which could act as an alert system for organizations and help them to spend their retention money effectively.
5.3. Limitations of the Study
There are limitations to this study. Some of the constraints were that we could not access some types of customer data, such as billing and credit information, due to telecom data categorization and confidentiality restrictions. This was a significant research constraint. Another constraint in conducting this study was the lack of demographic information on the clients. Therefore, it was not possible to include such criteria in the classification process, which would have improved the classification accuracy and interpretability.
6. Conclusions and Recommendations
The telecommunication industry has been hit the hardest and at a high stake, with an average annual churn rate of 30%, resulting in a tremendous waste of money and effort. It is essential to understand why customers leave to minimize the damage to the bottom line of the business.
Since churn is certain to result in lost revenue, and the cost of customer acquisition is equivalent to 5–10 times the cost spent on customer retention or satisfaction, churn forecasting has become a major task for companies to remain competitive. While aiming to help companies reduce customer churn to survive or grow, customer churn forecasting can be used in the long term to obtain continuously updated, real-time customer data. Therefore, the principal aim of this study was to build a prediction system that can help to identify customers who are likely to leave, specifically in the telecommunication industry. Various data mining methods like Decision Trees, Random Forests, and Logistic Regression were utilized by experts to build the predictive model. This study created a comparison system, since it is a more interpretable and intelligible approach to visualizing several machine learning algorithms. The performance of the models was evaluated using the accuracy measures, area under the curve, and sensitivity and specificity measures. The Random Forest classifier is the best classifier in this study, with an accuracy of around 90%, and it also performed well in other evaluation metrics.
Moreover, this paper contributes to the field by introducing the concept of explainable ensemble learning. By integrating various tools, such as LIME and SHAP, this study provides decision-makers with clear, actionable insights into the factors driving customer churn. This advancement is particularly significant in addressing the opaque nature of machine learning algorithms, making advanced analytics accessible and applicable for strategic decision-making in business settings.
Conceptualization, V.C. and F.O.A.; methodology, V.C. and F.O.A.; software, V.C., F.O.A. and M.A.G.; validation, V.C. and Q.A.X.; formal analysis, V.C. and K.H.; investigation, V.C.; resources, V.C.; data curation, F.O.A. and M.A.G.; writing—original draft preparation, V.C. and F.O.A.; writing—review and editing, V.C., K.H., Q.A.X. and V.B.; visualization, K.H., Q.A.X., F.O.A. and M.A.G.; supervision, V.C.; project administration, V.C.; funding acquisition, V.C. All authors have read and agreed to the published version of the manuscript.
The authors do not own the data.
Thank you to reviewers and editor for their reviews.
The authors declare no conflicts of interest.
Footnotes
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Enlarge this image.Figure 8. Local explanation for (a) the first sample (upper) and (b) the second sample (lower) of the Random Forest classifier.
Enlarge this image.Figure 10. SHAP explanation for (a) the first sample (upper) and (b) the second sample (lower) of the Random Forest classifier.
Confusion Matrix.
| Observed 0 | Observed 1 | |
|---|---|---|
| Estimated 0 | TN | FN |
| Estimated 1 | FP | TP |
Table of descriptive statistics.
| Count | Mean | Std | Min | 25% | 50% | 75% | Max | |
|---|---|---|---|---|---|---|---|---|
| Age | 4601 | 47.89307 | 17.362 | 19 | 33 | 47 | 62 | 80 |
| Number of Dependents | 4601 | 0.380569 | 0.880541 | 0 | 0 | 0 | 0 | 8 |
| Number of Referrals | 4601 | 1.947403 | 2.957352 | 0 | 0 | 0 | 3 | 11 |
| Tenure in Months | 4601 | 34.63117 | 24.19849 | 1 | 12 | 32 | 58 | 72 |
| Avg Monthly Long-Distance Charges | 4601 | 25.5811 | 14.26574 | 1.01 | 13.02 | 25.84 | 37.97 | 49.99 |
| Avg Monthly GB Download | 4601 | 26.12889 | 19.53716 | 2 | 13 | 21 | 30 | 85 |
| Monthly Charge | 4601 | 81.20272 | 21.14967 | −10 | 69.9 | 83.75 | 96.2 | 118.75 |
| Total Charges | 4601 | 3042.595 | 2391.057 | 42.9 | 847.3 | 2564.3 | 4968 | 8684.8 |
| Total Refunds | 4601 | 2.163306 | 8.286778 | 0 | 0 | 0 | 0 | 49.57 |
| Total Extra Data Charges | 4601 | 8.943708 | 28.62071 | 0 | 0 | 0 | 0 | 150 |
| Total Long-Distance Charges | 4601 | 888.9163 | 866.5069 | 1.13 | 178.89 | 582 | 1417.92 | 3536.64 |
| Total Revenue | 4601 | 3938.291 | 3054.189 | 46.92 | 1119.4 | 3378.79 | 6412.05 | 11979.34 |
Table of descriptive statistics.
| Model | Accuracy | AUC | Sensitivity | Specificity |
|---|---|---|---|---|
| Logistic Regression | 0.7553 | 0.84 | 0.7400 | 0.7704 |
| KNN | 0.7129 | 0.81 | 0.8467 | 0.5805 |
| Naïve Bayes | 0.7984 | 0.88 | 0.8227 | 0.7744 |
| Decision Tree | 0.8004 | 0.80 | 0.7907 | 0.8100 |
| Random Forest | 0.8694 | 0.95 | 0.8547 | 0.8839 |
References
1. Eklof, J.; Podkorytova, O.; Malova, A. Linking customer satisfaction with financial performance: An empirical study of Scandinavian banks. Total Qual. Manag. Bus. Excell.; 2020; 31, pp. 1684-1702. [DOI: https://dx.doi.org/10.1080/14783363.2018.1504621]
2. Madhani, P.M. Enhancing Customer Value Creation with Market Culture: Developing 7Cs Framework. IUP J. Manag. Res.; 2018; 17, pp. 46-64.
3. Chouiekh, A. Deep convolutional neural networks for customer churn prediction analysis. Int. J. Cogn. Inform. Nat. Intell.; 2020; 14, pp. 1-16. [DOI: https://dx.doi.org/10.4018/IJCINI.2020010101]
4. Duval, A. Explainable Artificial Intelligence (XAI); MA4K9 Scholarly Report Mathematics Institute, The University of Warwick: Coventry, UK, 2019; pp. 1-53.
5. Reilly, J. The Machine Learning Revolution: Telco Customer Churn Prediction. Technical Paper 2023; Available online: https://www.akkio.com/post/telecom-customer-churn (accessed on 13 April 2024).
6. Yulianti, Y.; Saifudin, A. Sequential feature selection in customer churn prediction based on Naive Bayes. IOP Conf. Ser. Mater. Sci. Eng.; 2020; 879, 012090. [DOI: https://dx.doi.org/10.1088/1757-899X/879/1/012090]
7. Leung, C.K.; Pazdor, A.G.; Souza, J. Explainable artificial intelligence for data science on customer churn. Proceedings of the 2021 IEEE 8th International Conference on Data Science and Advanced Analytics (DSAA); Porto, Portugal, 6–9 October 2021; pp. 1-10.
8. Mishra, A.; Reddy, U.S. A comparative study of customer churn prediction in telecom industry using ensemble based classifiers. Proceedings of the 2017 International Conference on Inventive Computing and Informatics (ICICI); Coimbatore, India, 23–24 November 2017; pp. 721-725.
9. De Caigny, A.; Coussement, K.; De Bock, K.W. A new hybrid classification algorithm for customer churn prediction based on logistic regression and decision trees. Eur. J. Oper. Res.; 2018; 269, pp. 760-772. [DOI: https://dx.doi.org/10.1016/j.ejor.2018.02.009]
10. Lalwani, P.; Mishra, M.K.; Chadha, J.S.; Sethi, P. Customer churn prediction system: A machine learning approach. Computing; 2022; 104, pp. 271-294. [DOI: https://dx.doi.org/10.1007/s00607-021-00908-y]
11. Yu, R.; An, X.; Jin, B.; Shi, J.; Move, O.A.; Liu, Y. Particle classification optimization-based BP network for telecommunication customer churn prediction. Neural Comput. Appl.; 2018; 29, pp. 707-720. [DOI: https://dx.doi.org/10.1007/s00521-016-2477-3]
12. Fujo, S.W.; Subramanian, S.; Khder, M.A. Customer churn prediction in telecommunication industry using deep learning. Inf. Sci. Lett.; 2022; 11, 24.
13. Kampani, N.; Jhamb, D. Analyzing the role of e-crm in managing customer relations: A critical review of the literature. J. Crit. Rev.; 2020; 7, pp. 221-226.
14. Nelson, C.A.; Walsh, M.F.; Cui, A.P. The role of analytical CRM on salesperson use of competitive intelligence. J. Bus. Ind. Mark.; 2020; 35, pp. 2127-2137. [DOI: https://dx.doi.org/10.1108/JBIM-07-2019-0328]
15. Al-Homery, H.; Asharai, H.; Ahmad, A. The core components and types of CRM. Pak. J. Human. Soc. Sci.; 2019; 7, pp. 121-145. [DOI: https://dx.doi.org/10.52131/pjhss.2019.0701.0076]
16. Calzada-Infante, L.; Óskarsdóttir, M.; Baesens, B. Evaluation of customer behavior with temporal centrality metrics for churn prediction of prepaid contracts. Expert Syst. Appl.; 2020; 160, 113553. [DOI: https://dx.doi.org/10.1016/j.eswa.2020.113553]
17. Hendriyani, C.; Auliana, L. Transformation from relationship marketing to electronic customer relationship management: A literature study. Rev. Integr. Bus. Econ. Res.; 2018; 7, pp. 116-124.
18. Chagas, B.N.R.; Viana, J.A.N.; Reinhold, O.; Lobato, F.; Jacob, A.F.; Alt, R. Current applications of machine learning techniques in CRM: A literature review and practical implications. Proceedings of the 2018 IEEE/WIC/ACM International Conference on Web Intelligence (WI); Santiago, Chile, 3–6 December 2018; pp. 452-458.
19. Eachempati, P.; Srivastava, P.R.; Kumar, A.; de Prat, J.M.; Delen, D. Can customer sentiment impact firm value? An integrated text mining approach. Technol. Forecast. Soc. Chang.; 2022; 174, 121265. [DOI: https://dx.doi.org/10.1016/j.techfore.2021.121265]
20. Lamrhari, S.; El Ghazi, H.; Oubrich, M.; El Faker, A. A social CRM analytic framework for improving customer retention, acquisition, and conversion. Technol. Forecast. Soc. Chang.; 2022; 174, 121275. [DOI: https://dx.doi.org/10.1016/j.techfore.2021.121275]
21. Mahmoud, M.A.; Hinson, R.E.; Adika, M.K. The effect of trust, commitment, and conflict handling on customer retention: The mediating role of customer satisfaction. J. Relat. Mark.; 2018; 17, pp. 257-276. [DOI: https://dx.doi.org/10.1080/15332667.2018.1440146]
22. De Caigny, A.; Coussement, K.; De Bock, K.W.; Lessmann, S. Incorporating textual information in customer churn prediction models based on a convolutional neural network. Int. J. Forecast.; 2020; 36, pp. 1563-1578. [DOI: https://dx.doi.org/10.1016/j.ijforecast.2019.03.029]
23. De Caigny, A.; Coussement, K.; Verbeke, W.; Idbenjra, K.; Phan, M. Uplift modeling and its implications for B2B customer churn prediction: A segmentation-based modeling approach. Ind. Mark. Manag.; 2021; 99, pp. 28-39. [DOI: https://dx.doi.org/10.1016/j.indmarman.2021.10.001]
24. Seymen, O.F.; Dogan, O.; Hiziroglu, A. Customer Churn Prediction Using Deep Learning. Proceedings of the 12th International Conference on Soft Computing and Pattern Recognition (SoCPaR 2020). SoCPaR 2020; Advances in Intelligent Systems and Computing; Springer: Cham, Switzerland, 2021; Volume 1383, [DOI: https://dx.doi.org/10.1007/978-3-030-73689-7_50]
25. Guo, W.W.; Xue, H. Crop yield forecasting using artificial neural networks: A comparison between spatial and temporal models. Math. Probl. Eng.; 2014; 2014, 857865. [DOI: https://dx.doi.org/10.1155/2014/857865]
26. Shahi, T.B.; Shrestha, A.; Neupane, A.; Guo, W. Stock price forecasting with deep learning: A comparative study. Mathematics; 2020; 8, 1441. [DOI: https://dx.doi.org/10.3390/math8091441]
27. Aldunate, Á.; Maldonado, S.; Vairetti, C.; Armelini, G. Understanding customer satisfaction via deep learning and natural language processing. Expert Syst. Appl.; 2022; 209, 118309. [DOI: https://dx.doi.org/10.1016/j.eswa.2022.118309]
28. Raza, K. Improving the prediction accuracy of heart disease with ensemble learning and majority voting rule. U-Healthcare Monitoring Systems; Academic Press: Devon, UK, 2019; pp. 179-196.
29. Ahmad, I.; Yousaf, M.; Yousaf, S.; Ahmad, M.O. Fake news detection using machine learning ensemble methods. Complexity; 2020; 2020, 8885861. [DOI: https://dx.doi.org/10.1155/2020/8885861]
30. Sahoo, A.K.; Pradhan, C.; Das, H. Performance evaluation of different machine learning methods and deep-learning based convolutional neural network for health decision making. Nat. Inspired Comput. Data Sci.; 2020; 871, pp. 201-212.
31. Wei, C.P.; Chiu, I.T. Turning telecommunications call details to churn prediction: A data mining approach. Expert Syst. Appl.; 2002; 23, pp. 103-112. [DOI: https://dx.doi.org/10.1016/S0957-4174(02)00030-1]
32. Anees, R.T.; Nordin, N.A.; Anjum, T.; Cavaliere, L.P.L.; Heidler, P. Evaluating the Impact of Customer Relationship Management (CRM) Strategies on Customer Retention. Bus. Manag. Strateg.; 2020; 11, pp. 117-133. [DOI: https://dx.doi.org/10.5296/bms.v11i2.17934]
33. Shrestha, Y.R.; Ben-Menahem, S.M.; Von Krogh, G. Organizational decision-making structures in the age of artificial intelligence. Calif. Manag. Rev.; 2019; 61, pp. 66-83. [DOI: https://dx.doi.org/10.1177/0008125619862257]
34. Buchanan, B.G.; Wright, D. The impact of machine learning on UK financial services. Oxf. Rev. Econ. Pol.; 2021; 37, pp. 537-563. [DOI: https://dx.doi.org/10.1093/oxrep/grab016] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34642572]
35. Fernández-Rovira, C.; Valdés, J.Á.; Molleví, G.; Nicolas-Sans, R. The digital transformation of business. Towards the datafication of the relationship with customers. Technol. Forecast. Soc. Chang.; 2021; 162, 120339. [DOI: https://dx.doi.org/10.1016/j.techfore.2020.120339]
36. Bauer, K.; von Zahn, M.; Hinz, O. Expl(AI)ned: The impact of explainable artificial intelligence on users’ information processing. Inf. Syst. Res.; 2023; 34, pp. 1582-1602. [DOI: https://dx.doi.org/10.1287/isre.2023.1199]
37. Teng, Y.; Zhang, J.; Sun, T. Data-driven decision-making model based on artificial intelligence in higher education system of colleges and universities. Expert Syst.; 2023; 40, e12820. [DOI: https://dx.doi.org/10.1111/exsy.12820]
38. Djeffal, C. The Normative Potential of the European Rule on Automated Decisions: A New Reading for Art. 22 GDPR. Z. Ausländisches Öffentl. Recht Völkerr.; 2020; 81, pp. 847-879.
39. Nkolele, R.; Wang, H. Explainable Machine Learning: A Manuscript on the Customer Churn in the Telecommunications Industry. Proceedings of the 2021 Ethics and Explainability for Responsible Data Science (EE-RDS); Johannesburg, South Africa, 27–28 October 2021; pp. 1-7.
40. Shapley, L.S. A value for n-person games. Contributions to the Theory of Games; Princeton University Press: Princeton, NJ, USA, 1953; Volum 2, pp. 307-317.
41. Štrumbelj, E.; Kononenko, I. Explaining prediction models and individual predictions with feature contributions. Knowl. Inf. Syst.; 2014; 41, pp. 647-665. [DOI: https://dx.doi.org/10.1007/s10115-013-0679-x]
42. Gramegna, A.; Giudici, P. SHAP and LIME: An evaluation of discriminative power in credit risk. Front. Artif. Intell.; 2021; 4, 752558. [DOI: https://dx.doi.org/10.3389/frai.2021.752558] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34604738]
43. Kwon, O.; Lee, N.; Shin, B. Data quality management, data usage experience and acquisition intention of big data analytics. Int. J. Inf. Manag.; 2014; 34, pp. 387-394. [DOI: https://dx.doi.org/10.1016/j.ijinfomgt.2014.02.002]
44. Provost, F.; Fawcett, T. Data science and its relationship to big data and data-driven decision making. Big Data; 2013; 1, pp. 51-59. [DOI: https://dx.doi.org/10.1089/big.2013.1508] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/27447038]
45. Martínez-Plumed, F.; Contreras-Ochando, L.; Ferri, C.; Hernández-Orallo, J.; Kull, M.; Lachiche, N.; Ramírez-Quintana, M.J.; Flach, P. CRISP-DM twenty years later: From data mining processes to data science trajectories. IEEE Trans. Knowl. Data Eng.; 2019; 33, pp. 3048-3061. [DOI: https://dx.doi.org/10.1109/TKDE.2019.2962680]
46. Hakkoum, H.; Idri, A.; Abnane, I. Artificial neural networks interpretation using LIME for breast cancer diagnosis. Trends and Innovations in Information Systems and Technologies; Springer: Cham, Switzerland, 2020; pp. 15-24.
47. Hutter, F.; Xu, L.; Hoos, H.H.; Leyton-Brown, K. Algorithm runtime prediction: Methods & evaluation. Artif. Intell.; 2014; 206, pp. 79-111.
48. Antwarg, L.; Miller, R.M.; Shapira, B.; Rokach, L. Explaining anomalies detected by autoencoders using Shapley Additive Explanations. Expert Syst. Appl.; 2021; 186, 115736. [DOI: https://dx.doi.org/10.1016/j.eswa.2021.115736]
49. Lessmann, S.; Baesens, B.; Seow, H.V.; Thomas, L.C. Benchmarking state-of-the-art classification algorithms for credit scoring: An update of research. Eur. J. Oper. Res.; 2015; 247, pp. 124-136. [DOI: https://dx.doi.org/10.1016/j.ejor.2015.05.030]
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Hall, Karl 2
; Xu, Qianwen Ariel 1 ; Amao, Folakemi Ololade 2 ; Meghana Ashok Ganatra 1 ; Benson, Vladlena 1 1 Department of Operations and Information Management, Aston Business School, Aston University, Birmingham B4 7ET, UK
2 School of Computing, Engineering and Digital Technologies, Teesside University, Middlesbrough TS1 3BX, UK;