Content area
Improving customer experience is a critical component to maintaining a successful business and can be accomplished by actively monitoring customer feedback. Online retailers typically capture feedback through ratings, comments, and surveys. While surveys broadly capture various aspects of customers' experience, focusing on returned products can deliver greater insight on how a product did not meet the customer's expectations. When a product return is initiated, the customer fills out a form describing the reason(s) for return. Return reason categories are often provided by the retailer in a broad manner, while the customers' description for the return reason provides more information on why this product did not meet their expectations. Understanding product returns provides the retailer with information useful for improving customer experience and cutting down on return costs. This research analyzes return data using Latent Dirichlet Allocation (LDA) topic modeling. Analyzing product returns using LDA provides a more detailed tool to track reasons for product returns which helps observe new emerging patterns that encompass the majority of the returns. This study concluded that studying product returns using LDA is an insightful tool to understand how a product did not meet customers' expectations. Discovering and understanding hidden patterns in customers' product returns provides the retailer with information needed to improve the product's online description, which helps enhance the customers' online shopping experience and drive improved business..
Abstract
Improving customer experience is a critical component to maintaining a successful business and can be accomplished by actively monitoring customer feedback. Online retailers typically capture feedback through ratings, comments, and surveys. While surveys broadly capture various aspects of customers' experience, focusing on returned products can deliver greater insight on how a product did not meet the customer's expectations. When a product return is initiated, the customer fills out a form describing the reason(s) for return. Return reason categories are often provided by the retailer in a broad manner, while the customers' description for the return reason provides more information on why this product did not meet their expectations. Understanding product returns provides the retailer with information useful for improving customer experience and cutting down on return costs. This research analyzes return data using Latent Dirichlet Allocation (LDA) topic modeling. Analyzing product returns using LDA provides a more detailed tool to track reasons for product returns which helps observe new emerging patterns that encompass the majority of the returns. This study concluded that studying product returns using LDA is an insightful tool to understand how a product did not meet customers' expectations. Discovering and understanding hidden patterns in customers' product returns provides the retailer with information needed to improve the product's online description, which helps enhance the customers' online shopping experience and drive improved business..
Keywords
Natural language processing, topic modeling, LDA, customer experience
1.Introduction
Online shopping has been increasing over the past decade, as the convenience of the click of a button has transformed the way people shop around the world. Newer technologies and the increasing number of online stores have made it easier for shoppers to get what they need from simple grocery items to home furniture from the comfort of their home at no extra cost. The new online shopping era have redefined the metrics for customer satisfaction across different industries. Location and opening hours are no longer a factor, since online shoppers can purchase goods from all around the world at the time of their choosing, but on the other hand new quality metrics have emerged to gauge the performance of these virtual stores. Customer experience is one of the key metrics to measure the success of a business. In online shopping, customers are always looking for convenience, whether that is defined by the ease of using the website or mobile app to navigate through products, or by providing different shipping speeds at affordable prices. But one of the most important aspects of an online shopping experience is assuring the customer that they can return the product if it did not meet their expectations.
In a survey published on Invesp, it was documented that 30% of all products ordered online are returned compared to 8.9% in brick-and-mortar stores [1]. While this feature offers convenience for the customer, it adds to the financial burden of the business in terms of shipping and restocking fees, and since over 50% of online retailers offer free returns, the store is responsible for all costs associated with the return [1]. Reducing the number of returns is essential to decreasing the extra expenses associated with product returns. In order to reduce the number of returns, understanding why customers returned a specific purchases is critical to improving the process and eventually eliminating a significant number of returns. To make a return, customers go through an online return request process where they select the item, quantity to return, and one of six parent return reason codes: Missing Parts & Pieces, Product Damaged, Inaccurate Website Description, Received Wrong Item, Changed Mind, or Item Not Received. Should a customer choose Missing Parts & Pieces, Product Damaged, or Item Not Received, they are presented with options to either replace or refund their item; should they choose Changed Mind or Inaccurate Website Description, they are asked to select a secondary return reason. For the former, the options are: Found better price, Found better alternative, Poor quality, and Arrived later than expected; for the latter, their options are: Size/ Dimensions, Color, Material, Not as Pictured or Described, and Product Features. The customer is then prompted to provide additional details into a free-text field which is the 'Return Comment'.
This paper studies the behavior of customer returns of a luxury home goods ecommerce retailer through analyzing return comments and extracting the different topics that emerge from the data. Topic modeling is used to understand how to edit and customize product description, shipping details, and different aspects related to the online shopping experience tailored to increase customer satisfaction with their products and eventually reduce the number of returns. The next section discusses previous work published in the literature to address understanding customers' comments and uncovering hidden patterns. The third section provides a description of topic modeling in general and Latent Dirichlet Allocation (LDA) in specific, followed by the results and conclusion in sections 4 and 5, respectively.
2. Related Research
Customers' comments are documented and collected along with other forms of unstructured data that the business can use to gain insights on different areas. Manually analyzing this unstructured data is extremely time consuming, inefficient, and introduces human bias to the results, therefore, the use of natural language processing (NLP) and machine learning have seen more research interest for analyzing huge text corpora. Term Frequency-Inverse Document Frequency (TF-IDF) accompanied with word2vec was used to represent data collected from verbatim inputs from transactional systems of quality office, warranty claims, issues matrix, surveys, and social media content, and was followed by implementing a support vector machines (SVM) model to classify customer comments into concern codes and make them easier to interpret [2].
Another study focused on customer satisfaction collected from unstructured data available as text and available through social media, comments from a survey, voice recordings of customer interactions, and chat transcripts. NLP and deep learning were employed to bridge the gap between understanding patterns available in textual data and providing recommendations for the business. The research discussed applying rule-based annotation, semantic tagging and business ontology creation on a dataset of car reviews, with a conclusion that deep learning models such as convolutional neural networks (CNNs) and Recurrent Neural Networks (RNNs) are more suitable for analyzing text on the sentence level. [3]. Sentiment of customers' comments is another important aspect to evaluate customer satisfaction. To extract how the customer feels towards their purchase, sentiment analysis, which is also known as opinion mining, is used to classify comments into 3 respective classes: negative, neutral and positive. In a study focused on analyzing the sentiment of customer reviews in the Arabic language for an online-bought product, adjectives were used as the key point to highlight important information in the comment then were classified based on these attributes [4]. More recently, advances in topic modeling techniques have addressed the challenges inherent to visualizing the high dimensionality of a fitted model by means of interactivity [5].
3. Methodology
Topic modeling is an unsupervised machine learning technique that can be applied to large amounts of unlabeled text documents to discover the hidden topics. As a natural language processing (NLP) technique, topic modeling helps transform the difficult nuances of human language into a format recognizable and analyzable by a computer. This is done by converting text into a numerical representation in the form of a term frequency (TF) matrix, which contains the word count for each unique word per document [6]. The total number of words constitute the vocabulary, whereas the matrix is referred to as the corpus.
3.1Pre-Processing for NLP
Pre-processing refers to the series of steps used to turn large bodies of text into a viable mathematical model. The first pre-processing step is tokenization, wherein the text is split into sentences, and the sentences split into words (e.g. "This rug was too big for the room" would become "This", "rug", "was", "too", "big", "for", "the", "room"). Next steps were to filter out common English stop words (e.g. "of', "if', "do", "an", etc.) and blanket removing words shorter than three letters, as well as words intermixed with numbers. These steps helped reduce the total amount of noise cluttering the data.
3.2Latent Dirichlet Allocation (LDA)
In statistics, latent variables involve elements that are inferred rather than directly observed. LDA is a mathematical model based on probability distributions (Dirichlet) to infer hidden (non-observable) variables and allocate words into topics. LDA models represent documents as a mixture of topics that output words with certain probabilities. There are two hyperparameters: the number of topics the model should discover, and the rules by which those topics are assigned to a document. The initial step requires choosing the total number of topics (K) to discover within the set of documents. As the algorithm scans through each document, it randomly assigns each word to one of the K topics. For each document (d) there are individual words (w) in the document, where two things are computed per topic (t): the proportion of words in document d currently assigned to document d, and the proportion of words currently assigned to this topic. Next, w is reassigned a new topic t, choosing t with the probability p(topic t in document d) · p(word w | topic t) - the probability that topic t has generated word w. This is the Dirichlet distribution where the sum probabilities of K topics will equal 1. This must be repeated until the chosen topics change very little.
After the documents are pre-processed, the LDA algorithm requires the data to be transformed into a Bag of Words (BoW) model, which serves as a dictionary to show the frequency that each word appears in the training set. The dictionary has a property that also allows us to look up the total occurrences of a single word across the entire dataset. Next the tokens must be filtered, where anything occurring in less than 15 documents or more than half the documents are removed. After, a BoW must be created per document to store the total number of words in the document and the number of times each word occurs. Each document results in a list of tuples, where each word comes with the integer id of the word and the number of times it occurs in each document. Passing the BoW corpus, number of desired topics, and the id-to-word dictionary through the algorithm at least twice results in a generated list of topics and the most important words occurring in each topic. For each item in the dataset, the LDA model provides a list of topic indexes and scores for each document in the collection, which allows one to sort by the highest score.
4.Implementation
This section provides a brief description of the dataset used as well as the experimental results of the applied LDA on the customer returns data, followed by a discussion of the applications and limitations of this approach.
4.1 Dataset Description
For this experiment, an LDA model was applied to 11 months of customer returns data collected from February 2019 through January 2020 for a luxury home-goods company based in the United States. The 11,998 requested return orders represented roughly 15,600 items and $3.2M of merchandise, and were extracted and compiled from the Customer Experience department return report data table.
The data consists of 45 categorical and numerical features, including the original customer order number; date of purchase and initial return date; tracking number(s); item SKU ID; return comment text; primary and secondary return reason codes; shipping method and carrier; item quantity, brand, vendor, product name, merchandise price, tax, shipping, and total refund amount; restocking fees; and final return disposition of the item; Figure 1 displays a snapshot of the raw data. After applying the pre-processing steps listed above, PyLDAvis [5] was used to visualize the topics generated by the model.
4.2Experimental Setup
The hyperparameters for this experiment included the number of topics (K) to be discovered, set at 10, as well as the specific term factors to be vectorized. These included removing any word appearing in over 50% of the documents or any word appearing less than 10 times. The batch size was set to 128, with a learning decay of 0.7 and offset of 10. The 1, representing the relevance metric used to fine-tune the exclusivity or generality of the words to a specific topic, can be described as such: relevance(term w | topic t) = Л· p(w|t) + 1 - X; this value was set just under 0.5 at 0.44.
4.3Results and Discussion
The LDA was visualized through an interactive PyLDAvis Intertopic Distance Map (IDM) as shown in Figure 2. The IDM was arranged in a side-by-side layout depicting a global view of uncovered topics on the left and bar charts portraying the most frequently found terms on the right. The IDM showed three soft clusters of topics; from all topics, the LDA returned the 30 most salient terms from the customer comments, with the top five general results being "item", "color", "quality", "expected", and "pictured". These are consistent with the parent Return Reason of "Inaccurate Website Description", which is the second-most common reason for customer returns after "Changed Mind". Topic 2, located in the top right-hand cluster of the IDM, presented a consistent breakdown of these terms, with "looks", "cheap", "appeared", "picture", and "online" informing that many customers had mismatched experiences between what was offered on the website versus the product they received (Figure 3). The presence of "gold", "finish", "vibrant", "real", "brass", and "painted" within Topic 2 suggest a particular breakdown between expectations and reality for products within the gold/yellow color family, which could indicate opportunity to better clarify product descriptions between gold finishes that are authentic and those that are painted-on veneers.
Likewise, Topic 10 groups "thought", "size", "room", "bigger", "smaller", "anticipated", which combine to represent a topic and sub-cluster that center on sizing miscalculations (Figure 4). Though many products include dimensions in their descriptions, this topic gives insight to a gap between customers' real and perceived ability to size furniture or other items accurately to their intended space based solely on numerical dimensional descriptions. Three potential solutions to this could include: 1) adding accessible visual guides to product pages on larger items such as sofas and rugs which help the customer in advance with expert-suggested sizing options to match their room of choice; 2) examining the existing dimensions for products experiencing high rates of returns to validate true deviations from what is listed; 3) exploring third-party Augmented Reality products which would allow customers to visualize and interact with the item in 3D through their smartphone camera.
5.Conclusion and Future Work
Analyzing unstructured textual data presents a challenging but critical task in understanding and leveraging customer feedback. This study sought to use LDA to analyze customers' return comments to uncover hidden patterns in customer behavior and inform on potential opportunities to improve customer experience. The results of this research suggested that perceived quality and product size are among the most prominent drivers for customer returns. Providing better descriptions of these characteristics could shrink the gap between customer expectations and the reality of the product offered. Additionally, providing customers with high-tech tools to bring them closer to the product would help decrease the number of returns. One of the limitations for this study was excluding a portion of the return data which was documented by customer service agents rather than customers due to the limited vocabulary allowed to describe the return reason. In the future, this research could be expanded by incorporating product reviews in addition to the return comments to provide a more holistic picture behind the product return. Finally, exploring alternative clustering techniques for text data could provide a better understanding of the main contributors toward product returns.
References
[1] Salih, K. "E-commerce Product Return Rate - Statistics and Trends". Retrieved from https://www.invespcro.com/blog/ecommerce-product-return-rate-statistics/
[2] Akella, K., Venkatachalam, N., Gokul, K., Choi, K., & Tyakal, R. (2017). "Gain Customer Insights Using NLP Techniques". SAE International Journal of Materials and Manufacturing, 10(3), 333-337.
[3] Ramaswamy, S., & DeClerck, N. (2018). "Customer Perception Analysis Using Deep Learning and NLP". Procedía Computer Science, 140, 170-178.
[4] Abuleil, S., & Alsamara, K. (2017). "Using NLP Approach for analyzing Customer Reviews." SOEN-2017, 117-124.
[5] Sievert, C., & Shirley, K. (2014). "LDAvis: A method for visualizing and interpreting topics." Proceedings of the Workshop on Interactive Language Learning, Visualization, and Interfaces. doi: 10.3115/v1/w14-3110
[6] Chuang, J., Manning, C. D., & Heer, J. (2012). Termite. Proceedings of the International Working Conference on Advanced Visual Interfaces - AVI 12. doi: 10.1145/2254556.2254572
Copyright Institute of Industrial and Systems Engineers (IISE) 2020