Exploring the Relationships among Factors Influencing Healthcare Chatbot Adoption

Abstract

Technology has revolutionized various industries; notably, chatbots impact healthcare through the efficient streamlining of medical procedures, cost reductions, and improved accessibility to healthcare services. Consequently, understanding how to promote the adoption of healthcare chatbots has become crucial for enhancing the healthcare industry and medical service quality. Despite numerous studies identifying factors influencing healthcare chatbot adoption, there is a notable lack of empirical verification regarding their interrelationships, leading to a significant knowledge gap. Therefore, this study aims to address this gap by developing a decision-making model to analyze the relationships among key factors regarding three dimensions: technology, user, and society. The model begins by filtering out insignificant factors based on experts’ opinions. Subsequently, it employs DEMATEL (Decision Making Trial and Evaluation Laboratory) to construct a causal relationships graph and the ISM (interpretive structural modeling) method to categorize these factors into a hierarchical order. To mitigate uncertainties stemming from the topic’s complexity, this study utilizes fuzzy sets and Z-number theory in the assessment process. The findings reveal a predominance of causal factors within the technological dimension. Notably, the quality of information provided by chatbots stands out as the most influential causal factor. The insights from this study suggest implications for both enterprises and governments to boost chatbot adoption in society.

1. Introduction

The progress in technology is sparking conversations about the potential transformation of various processes across industries, society, and the economy, with particular emphasis on its impact on the healthcare sector. The healthcare industry is known as an industry that comprises entities offering medical services, producing medical devices or medications, delivering medical coverage, or supporting the delivery of healthcare to individuals [1]. Healthcare facilities and caregivers globally, particularly in advanced economies, are actively integrating digital advancements, which aim to enhance both the quality of care provided and the operational efficiency of these establishments [2]. People’s awareness is moving towards patient-centered services. Hospitals and medical services have been moving their attention from treatment to disease prevention and health management to deal with the global aging population and healthcare costs around the world [3,4]. Therefore, this is the perfect time for new technologies, such as chatbots with their huge potential, to fulfill the demands of society.

Chatbots, also referred to as artificial conversational entities, interactive agents, smart bots, and digital assistants, are entities designed to engage in conversation and interact with users in various contexts [5]. A chatbot can also be defined as an artificial intelligence (AI) program that operates within the framework of the human–computer interaction model [6]. Chatbots save information in a database to find important words in sentences and decide how to respond to questions [7]. These are mimic systems that can copy how people talk to each other using techniques that understand language, process it naturally, and learn from machines [8]. The use of chatbots in healthcare has many practical applications, from seeking medical advice and booking medical appointments [9] to dealing with anxiety and depression [10] and even helping with cancer [11]. In addition, chatbots are said to be the solution to increasing mental healthcare access [12]. In particular, chatbots showed potential for fighting COVID-19 [13]. Despite these great benefits, healthcare chatbots, in fact, are not popular among the general public, as explained in a research study wherein only 6% of the participants had heard of a health chatbot [9]. These days, human–technology interactions have gained importance in modern life, so there have been a large number of studies revealing factors affecting the adoption of health chatbots. Nevertheless, the inquiry into which factors constitute key factors and the relationships among them remains unanswered. The process leading to that answer reflects a multi-attribute decision-making approach.

Multiple criteria decision-making (MCDM) techniques have been widely applied in many aspects of healthcare industry such as healthcare service quality [14,15,16], medical treatments [17,18,19], and technology applications [20,21,22]. However, regarding health chatbots, the applications of MCDM are limited and new. Santa Barletta et al. [23] applied the very traditional technique of analytic hierarchy process (AHP) to evaluate clinical health chatbots. Hsu [24] constructed a model for implementing chatbots within military mental health services using DEMATEL. AHP was also used to select chatbots in the healthcare industry in Thailand [25].

Considering the noted shortcomings in previous research, this study proposes a model to empirically investigate the key factors influencing the acceptability of healthcare chatbots. The model seeks to highlight the relationships among these factors and identify the pivotal factors within the system. Initially, the study will filter out insignificant factors and then employ DEMATEL and ISM techniques to explore these relationships. Furthermore, fuzzy sets and Z-numbers will be utilized to address uncertainty in expert assessments stemming from the novelty of the subject matter.

This research provides a methodological framework to facilitate decision-makers in systematically identifying root causes within complex systems. This systematic approach lays a robust foundation for the development of effective strategies. The study has achieved the following contributions:

• (1). After listing important factors that impact healthcare chatbot adoption in a broader context, the study can highlight these, which can assist with decision-making processes.

• (2). By pinpointing causal relationships and visually representing them, this study simplifies the complex evaluation system. This simplification aids in effectively exploring root problems and developing strategies.

• (3). Based on the results, this study provides managerial and theoretical implications for health chatbot adoption.

2. Literature Review

This section will present the list of factors identified from the literature and describe the methods used to obtain these factors.

2.1. Factor Identification Process

• (1). Identifying key terms: To discover relevant factors, we first generated key words related to the theme of healthcare chatbot adoption factors. This process yields a list of key terms, as shown in Table 1.

• (2). The identified keywords were used to search research databases for relevant factors.

2.2. Factors Influencing Health Chatbot Adoption

Twenty factors were identified and grouped into three dimensions: technology, user, and society.

2.2.1. Technology Dimension

The technology aspect pertains to the specific traits and capabilities of health chatbots.

• Usefulness

Perceived usefulness is one of the two factors in the technology acceptance model [26], used to study user acceptance of new technology. The acceptance of AI-driven chatbot services in healthcare shows a positive correlation with the usefulness of the chatbot [9]. The perceived benefits of healthcare chatbots are a direct predictor of adoption [27,28,29]. Moreover, perceived usefulness is a significant predictor of intention to continue using chatbots [30]. In this study, we propose usefulness to be the ability to meet the demand of patients.

• Integration

In a scoping review, it is suggested that chatbots’ interconnectedness with other devices can help to enhance the adoption of AI applications in healthcare [31].

• Security

Information insecurity is considered a barrier to chatbot adoption [9]. Data privacy is also a concern of patients using health AI applications [31].

• Quality of information

It is emphasized that initial failures of chatbots may adversely impact a patient’s willingness to look for mental health assistance in the future [32]. In another research study, most of the participants believed that reliable and accurate information about healthcare chatbots is important [9].

• Customization

It is suggested that, to improve the adoption of AI applications in healthcare, personalization and customization should be taken into account [31].

• Natural language ability

Natural language is an advanced ability of chatbots, which make them sound like humans. In healthcare, empathy is a crucial factor in building trust between service providers and patients; however, it should be displayed properly in verbal and nonverbal expressions [31].

• Human involvement

Miles et al. [33] found out that a general practitioners–chatbot approach is more acceptable than a chatbot-only approach. In addition, patients were less accepting of chatbots due to their perceived lack of empathy and the absence of a professional human approach [9].

2.2.2. User Dimension

The user dimension encompasses factors associated with individuals engaging with chatbots, including various aspects such as their attitudes toward these systems, educational backgrounds, health-related considerations, etc.

• Chatbot perception

Healthcare chatbot acceptability is highly correlated with positive attitudes and perceived trustworthiness towards heath chatbots [9]. Individuals held favorable views regarding healthcare chatbots and expressed a willingness to adopt these innovative technologies [27,34]. Furthermore, medical AI resistance bias can have a negative impact on the intention to switch to an AI chatbot [35].

• IT literacy

Chatbot acceptability in healthcare is negatively correlated with poor IT skills [9].

• Level of education

A cross-sectional study showed that highly educated users were more likely to engage with healthcare chatbots [36].

• Personal innovativeness

People are curious about how new technologies could improve health, and this is highly correlated with an intention to use health chatbots [9]. Those who look for complementary and alternative medicine tend to look for new solutions to improve their well-being and health [37]. Personal innovativeness was found to be a crucial factor influencing people’s intention to use a health chatbot [28].

• Health issues

For health problems such as sexual and mental health issues, it is preferable that these be dealt with by chatbots [9]. Nevertheless, these technological solutions are deemed less suitable for managing health problems of greater severity and, thus, should not be suggested in such cases [33].

• Hedonic motivation

Hedonic motivation is a popular factor in technology adoption research [38]. In a research about COVID-19 chatbots, the most influential predictor of readiness to use chatbots was shown to originate from the internal drive factor [27].

• Habit

People who regularly sought health information online were more prone to finding nearby healthcare services using healthcare chatbots. [36]. In addition, users’ habits influence their intention to use a health chatbot [28,36].

2.2.3. Society Dimension

This dimension holds a wider scope compared to others. These factors lie beyond the control of chatbot users or developers.

• Promotion

Although the popularity of chatbots in the field of public health has increased since COVID-19 [39], there existed a widespread lack of familiarity and understanding regarding health chatbots [9]. Numerous studies have underscored the limited public and clinical awareness regarding the capabilities of AI in healthcare and most respondents expressed a willingness to learn more about these capabilities [31].

• Cost

Acceptance hinges on the cost of AI diagnosis in comparison to that of doctors [40]. Moreover, the cost of healthcare chatbots can be an important element in regions with inadequate health insurance coverage [41].

• Social influence

People tend to conform to the opinions of others regarding healthcare chatbots, as found in an analysis [41].

• Healthcare infrastructure

There is a strong correlation between the technology adoption index and infrastructure in agriculture [42,43], which may also be applicable to the healthcare industry.

• Cultural norms

Some cultural dimensions were discovered to have notable moderation effects on the acceptance of service robots [44].

• Network externalities

Network externality was found to play a crucial role in influencing one’s adoption of a healthcare chatbot [28].

Table 2 summarizes the list of identified factors.

3. Methodology

This section introduces the methods utilized in the study and outlines the steps for their applications.

3.1. Experts’ Importance Integration

Considering varying levels of experience and expertise, each expert is assigned a specific weight indicating the significance of their assessment to the study. This is because giving priority to expertise during the aggregation stage can help mitigate unstable preferences [45]. Levels of expertise can be categorized according to years of experience in the field, consistency levels, or other pertinent indicators [46].

3.2. Exploring Relationships among Factors

DEMATEL is a technique introduced by the Geneva Research Centre of the Battelle Memorial Institute to solve sophisticated and intertwined issues [47]. In contrast to other traditional approaches such as the analytic hierarchy process, this technique can show relationships among factors through a causal diagram [48,49]. The result of DEMATEL is a diagraph showing interdependence among factors, which divide factors into cause and effect groups, with numbers showing strength of influence [50]. DEMATEL has been successfully applied to discover relationships and rankings among adoption factors in various different industries such as the manufacturing sector [51], healthcare [22], banking [52], and retailing [53].

ISM was developed by Warfield [54] to deal with complex problems with the help of experts. The method is interpretive, in that the experts’ expertise decides whether and how items are related, an overall structure is drawn from the set of items, and the specific relationships and overall structure are demonstrated in a digraph model [55]. ISM has been applied successfully in many fields to discover the relationships among adoption enablers and their ratings such as the adoption of solar energy products [56], barriers to the agriculture supply chain [57], cloud computing adoption [58], and barriers to m-commerce adoption [59].

Due to the effectiveness of these two techniques in exploring relationships among adoption factors, we advocate for their use in our study.

3.3. Addressing Uncertainty

When completing the surveys, experts use their experiences and knowledge to reflect the real situation. According to Tseng, Ming-Lang et al. [60], crisp numbers cannot present their evaluation. Experts need to depend on linguistic variables to express their evaluation. However, these linguistic preferences are not directly comparable values; therefore, fuzzy set theory is utilized to convert linguistic variables into crisp values. Moreover, along with fuzzy logic, Z-numbers facilitate the decision-making process when the information is imprecise and/or incomplete. Since health chatbots are still in the initial phases of development [12,39], there is a high level of uncertainty regarding the topic; therefore, this approach is excellent for addressing this issue.

3.4. The Proposed Methods

This section details the outline of our model, organized into three stages, as illustrated in Figure 1. The goals of the stages are listed as follows:

• (1). Developing the list of validated factors by filtering out insignificant ones.

• (2). Evaluating the relationships among factors within each dimension and across the entire system.

• (3). Establishing a hierarchical order for the factors based on their relationships.

• (1). First stage: Developing the list of validated factors by filtering out insignificant ones.

After defining the factors, a survey is created to ask experts about the levels of significance of the factors in practice. If the expert k thinks the factor j is significant in real life, he/she will give the answer “Yes”, which equals to 1 ${(e}_k)$; otherwise, it will be “No”, which equals to 0 ${(e}_k)$. A sample of the first questionnaire is shown in Appendix A. With the integration of weight of each expert $\left(w_k\right)$, if the average scores, $A_j$, equal at least 75%, the factors are selected for further research. The equation for this calculation is as follows:

(1)$A_j=\sum \left(e_k*w_k\right)$

• (2). Second stage: Evaluating the relationships among factors within each dimension and across the entire system.

The steps below show how the DEMATEL technique is conducted in the study after obtaining the list of validated factors.

Step 2.1: Constructing the pairwise comparison matrix X

In this step, the questionnaire asks the experts to give their opinions on pairwise comparisons of influence among factors, using a scale spanning from no influence to very high influence. Triangular fuzzy numbers (TFNs) are applied to demonstrate the fuzziness in experts’ decisions. Table 3 illustrates the linguistic terms [61] used in the study.

A TFN is demonstrated by a triplet, i.e., $a_{\mathit{ij}},{ b}_{\mathit{ij}},{ c}_{\mathit{ij}}$. Suppose $x_{\mathit{ij}}^k=\left(a_{\mathit{ij}}^k,{ b}_{\mathit{ij}}^k,{ c}_{\mathit{ij}}^k\right),$ where 1 ≤ k ≤ K is a fuzzy judgment that the kth expert makes about the degree to which factor i has influence on factor j. If there are K experts on the decision panel to estimate the causality $x_{\mathit{ij}}^k$ between n adoption factors, the inputs are a n × n matrix, i.e., X^k= ${\left[x_{\mathit{ij}}^k\right]}_{n \times n}$; where k = 1, 2,…, K.

Also, while giving answers on degree of causality, experts are also asked to give their confidence level $\stackrel{~}{{{\mathit{R}}^k}_{\mathit{ij}}}$ for each answer on a fuzzy five-level scale [61], as shown in Table 4. A TFN is applied again to capture the fuzziness in experts’ judgements, i.e., $m_{\mathit{ij}},o_{\mathit{ij}},{ p}_{\mathit{ij}}$. A sample of the second questionnaire for pairwise comparison is shown in Appendix B.

Therefore, each expert’s evaluation can be denoted as follows:

(2)${\mathit{Z}}_{\mathit{ij}}=\left(\stackrel{~}{{{\mathit{X}}^k}_{\mathit{ij}}}, \stackrel{~}{{{\mathit{R}}^k}_{\mathit{ij}}}\right)$

(3)${\mathit{Z}}_{\mathit{ij}}=\left(a_{\mathit{ij}}^k, b_{\mathit{ij}}^k,{ c}_{\mathit{ij}}^k;{ m}_{\mathit{ij}}^k,{o }_{\mathit{ij}}^k,{ p}_{\mathit{ij}}^k\right)$

where $\stackrel{~}{{{\mathit{X}}^k}_{\mathit{ij}}}$ represents the influence membership of factor i on factor j of expert k, and $\stackrel{~}{{{\mathit{R}}^k}_{\mathit{ij}}}$ represents the corresponding reliability membership.

After that, Z_ij will be converted into classical fuzzy numbers, before moving to the next step using the following method by Kang et al. [62] as in Equations (4)–(6):

First, convert the reliability part into a crisp number, as follows:

(4)$\alpha =\frac{\int x{\mathsf{\mu }}_{\stackrel{\mathrm{~}}{\mathit{R}}}\left(x\right)\hspace{0.17em}dx}{\int {\mathsf{\mu }}_{\stackrel{\mathrm{~}}{\mathit{R}}}\left(x\right)\hspace{0.17em}dx}$

when $\tilde{\mathit{R}}$ denotes the reliability membership.

Second, add the weight of the reliability part $\left(\alpha \right)$ to the restriction part using Equation (5), as follows:

(5)${\stackrel{\mathrm{~}}{\mathit{Z}}}^{\alpha }=⟨x,{\mathsf{\mu }}_{{\tilde{\mathit{X}}}^{\alpha }}\left(x\right)⟩|{\mathsf{\mu }}_{{\tilde{\mathit{X}}}^{\alpha }}\left(x\right)=\alpha {\mathsf{\mu }}_{{\tilde{\mathit{X}}}^{\alpha }}, x\in \left[0,1\right]$

Third, convert the irregular fuzzy number (${\stackrel{\mathrm{~}}{\mathit{Z}}}^{\alpha }$) into a classical fuzzy number, as follows:

(6)$\widetilde{{\mathit{Z}}^{\prime }}=⟨x,{ \mathsf{\mu }}_{\widetilde{{\mathit{Z}}^{\prime }}}\left(x\right)⟩|{\mathsf{\mu }}_{\widetilde{{\mathit{Z}}^{\prime }}}(x)={\mathsf{\mu }}_{\tilde{\mathit{X}}}\left(\frac{x}{\sqrt{\alpha }}\right), x \in \left[0,1\right]$

The result of the calculation is a classical fuzzy set, which is denoted as follows:

(7)${\mathit{Z}}^{\prime }=\left(a_{\mathit{ij}}^{\prime k}, b_{\mathit{ij}}^{\prime k},{ c}_{\mathit{ij}}^{\prime k}\right)$

After that, ${\mathit{Z}}^{\prime }$ will go through the defuzzification using the center of gravity method, which is set out in the equation below:

(8)$x_{\mathit{ij}}^{\prime k}=\frac{a_{\mathit{ij}}^{\prime k}+b_{\mathit{ij}}^{\prime k }+c_{\mathit{ij}}^{\prime k}}{3}$

Finally, the pairwise comparison matrix ${\mathit{X}}^k$ of expert k is constructed, using Equation (9), as follows:

(9)${\mathit{X}}^k={\left[{x^{\prime }}_{\mathit{ij}}^k\right]}_{n \times n}$

Step 2.2: Compute the direct relationship matrix with weight integration A

The n × n matrix A is found by averaging all the experts’ scores from matrices ${\mathit{X}}^k$, as follows:

(10)$\mathit{A}= {\left[a_{\mathit{ij}}\right]}_{n \times n}$

(11)$a_{\mathit{ij}}=\frac{\sum w_k\times {x\prime }_{\mathit{ij}}^k}{k}$

where $a_{\mathit{ij}}$ is the average influence score of factor i on factor j, $w_k$ is the weight of the expert, $x_{\mathit{ij}}^{\prime k}$ is the influence score of expert kth for factor i on factor j, and k is the number of experts.

Step 2.3: Computing the normalized initial direct relationship matrix D

The matrix D is calculated by normalizing the average matrix A.

Let the following be true:

(12)$s=\mathbf{max}\left(\underset{1\le i\le n}{\mathbf{max}}{\sum }_{j=1}^n{a}_{\mathit{ij}},\underset{1\le j\le n}{\mathbf{max}}{\sum }_{i=1}^n{a}_{\mathit{ij}}\right)$

Then, as follows:

(13)$\mathit{D} =\frac{\mathit{A}}{s}$

Step 2.4: Compute the total influence matrix T

The total influence matrix T is defined as an n × n matrix, as in Equation (14) below:

(14)$\mathit{T}={\left[t_{\mathit{ij}}\right]}_{n \times n}$

where (i, j = 1, 2, 3, 4,…, n), and

T = D × (I − D)⁻¹ (15)

when

(16)$\underset{r\to \infty }{\lim }{\mathit{D}}^r={\left[0\right]}_{n \times n}$

where I is the n × n identity matrix.

Also, the sums of rows and the sums of columns of the matrix T are denoted as R and G in Equations (11) and (12), respectively, as follows:

(17)$\mathit{R}={\left[r_i\right]}_{n \times 1}={\left({\sum }_{j=1}^n{t}_{\mathit{ij}}\right)}_{n \times 1}$

(18)$\mathbf{G}={\left[g_j\right]}_{1 \times n}^{\prime }={\left({\sum }_{i=1}^n{t}_{\mathit{ij}}\right)}_{1 \times n}$

where $r_i+{ g}_j$ show the combined effect both provided and received by factor i, and $r_i-{ g}_j$ gives the net effect by factor i.

Step 2.5: The thresholds are calculated to filter out insignificant influence relationships in the total influence matrix using Equation (19), as follows:

(19)$\beta =\frac{{\sum }_{i=1}^n{\sum }_{j=1}^n{t}_{\mathit{ij}}}{N}$

where N is the number of items in the matrix T.

Step 2.6: Visualize the cause–effect relationships

• (3). Stage 3: Establishing hierarchical orders for the factors based on their relationships.

The following steps show how ISM technique is employed in the study. The approach is based on the method by Mathiyazhagan, K. et al. [63], with some modifications.

Step 3.1: Create the initial reachability matrix L

If the value $t_{\mathit{ij}}$ is equal or greater than the threshold $\beta$, it will be denoted as having a relevant amount of influence on other factors ($l_{\mathit{ij}}=1$) or having negligible impact ($l_{\mathit{ij}}=0)$. The matrix L can be obtained by the equation below:

(20)$\mathit{L}={\left[l_{\mathit{ij}}\right]}_{n \times n}$

Step 3.2: Develop the final reachability matrix F based on the direct and indirect relationships among the factors. The logic is that, when factor i has an impact on factor j, and factor j has an impact on factor k, factor i will be considered to have an impact on factor k.

Step 3.3: Level partition

The reachability and antecedent sets for each factor are determined using the final reachability matrix F variables. The reachability set includes the variable itself and other variables that the variable influences. Similarly, the antecedent set includes the variable itself and others that influence the variable. The intersection of these sets is then computed across all variables. The Level I variable in the ISM hierarchy, which cannot assist in achieving any higher-level variables, is identified as the one where the reachability and intersection sets coincide. Once level I is identified, it is removed from consideration, along with any other variables in that level.

Step 3.4: Development of the cause–effect digraph

From the results of level partitioning, the model is constructed. The relationships among levels are shown by an arrow pointing from the level that exerts influence to the level that receives influence.

4. Case Study Analysis and Results

This section will describe how the above methods were used to analyze the case study and the results.

4.1. Identification of Key Factors

Chakrabortty et al. [64] and Phooriyaphan et al. [25] indicate that, for reliable results, this type of study typically requires the participation of three to nine experts with expertise in AI and chatbots to complete the surveys. Thus, our expert panel comprises six males and three females with expertise in healthcare chatbot adoption, encompassing roles such as professors, researchers, developers, professionals, and users across various organizations. The experts received a questionnaire containing twenty factors from the literature to validate. Table 5 summarizes the experts’ information alongside the weights they were assigned.

The experts were tasked with providing their evaluations regarding the importance of the factors. The significance level of each factor was determined using Equation (1). Subsequently, only factors with values exceeding 75% were retained for further investigation, as depicted in Table 6.

4.2. Casual Relationships

This study will employ the DEMATEL technique to elucidate the interrelationships among factors within a dimension, as well as the relationships among different dimensions.

After collecting the results from the pairwise questionnaire, the next step involves the conversion of linguistic terms into Z-numbers. For instance, if an expert assesses that factor C₁ exhibits low influence (L) on factor C₂ with a reliability level of “Neutral” (N), the corresponding Z-number would be represented as (L, N) = (1, 2, 3; 0.3, 0.5, 0.7), based on Table 3 and Table 4.

The following steps describe how the calculations are performed to convert the Z-number into a classical fuzzy set [62]. Utilizing the aforementioned example, the calculation of α is initiated using Equation (4), as follows:

(21)$\mathsf{\alpha }=\frac{{\int }_0^{\mathrm{x}}{x\mathsf{\mu }}_{\stackrel{\mathrm{~}}{\mathit{R}}}dx}{{\int }_0^{\mathrm{x}}{\mathsf{\mu }}_{\stackrel{\mathrm{~}}{\mathit{R}}}dx}=\frac{{\int }_{0.3}^{0.5}x\left(\frac{0.5-x}{0.5-0.3}\right)dx+{\int }_{0.5}^{0.7}x\left(\frac{0.7-x}{0.7-0.5}\right)dx}{{\int }_{0.3}^{0.5}\left(\frac{x-0.3}{0.5-0.3}\right)dx+{\int }_{0.5}^{0.7}\left(\frac{0.7-x}{0.7-0.5}\right)dx}=0.5$

Then, the α value is incorporated with the influence fuzzy number, as follows:

(22)$\widetilde{Z^{\mathsf{\alpha }}}=\left(1, 2, 3; 0.5\right)$

This fuzzy number then is converted into a crisp set using Equation (6), as follows:

(23)$Z^{\prime }=\left(\sqrt{0.5}\times 1,\sqrt{0.5}\times 2,\sqrt{0.5}\times 3\right)$

(24)$Z^{\prime }=\left(0.707, 1.414, 2.212\right)$

Applying the same calculation method as above, we derive the values of crisp sets for all combinations of fuzzy linguistic terms representing influence and fuzzy reliability terms, as presented in Table 7 [61].

Table 8 displays the assessment of expert 1. Using Table 7 as a reference, we can convert experts’ assessment into fuzzy sets, as shown in Table 9.

After that, the scores provided by the experts undergo a defuzzification process using Equation (8). For example, Table 10 shows the defuzzified number of fuzzy sets in Table 9.

4.2.1. Causal Relationships between Dimensions

In this part, we discuss the relationships in each dimension, rather than the whole system. Therefore, to create the average matrix for each dimension, we first categorize the data of direct relation matrices X^k into dimensions as mentioned earlier (technology—C_1, C_2, C_3, C_4, C_5, and C₆, user—C₇, C₈, and C₉, and society—C₁₀, C₁₁, C₁₂, C₁₃, and C₁₄).

Then, we apply Equation (11) for each dimension to create the weighted average matrices, as shown in Table 11, Table 12 and Table 13.

The next step involves normalizing these matrices. We use Equation (12) to find the value s of each matrix and utilize Equation (13) to obtain the normalized values. Total influence matrices are created using Equations (15) and (16), as shown in Table 14, Table 15 and Table 16. Scores higher than the predetermined thresholds are highlighted in bold, indicating significant influences between factors. The thresholds are calculated for each dimension using Equation (19).

The graphs below are created based on significant relationships among factors in each dimension, after removing insignificant relationships (Figure 2, Figure 3 and Figure 4).

Regarding the technological dimension, certain attributes such as the customizability of chatbots (C₄), language capabilities (C₅), and information quality (C₂) only influence other technological variables without being affected. Noteworthy is the considerable degree to which external factors influence the usefulness aspect (C₁). Moreover, usefulness (C₁) and human involvement (C₆) are subject to the influence of various other factors.

Looking at the user-related factors, the perceptions held by users regarding chatbots (C₇) exert an influence on habits (C₉). Moreover, such perceptions (C₇) significantly affect pleasure from their usage (C₈).

In the societal dimension, social influence (C₁₂) and the cost of chatbots (C₁₀) significantly impact the promotional aspect (C₁₁). Conversely, cultural dynamics (C₁₃) remain insulated from the influences of other factors. Social influence (C₁₂) is significantly affected by promotion (C₁₁) and culture (C₁₃).

4.2.2. Casual Relationships between Factors

In a manner similar to the previous procedures, the direct relationship matrix X, the average matrix A, the normalized direct relationship matrix D, and the total influence relationship T are formulated for the 14 factors. Table 17 and Table 18 show the matrices A and T.

Table 19 represents rankings for significance level, cause, and effect, based on the values of r + g and r − g.

Figure 5 displays both the combined effect (r + g) and the net effect (r – g) of each factor, with three dimensions represented by different colors. Figure 6 shows the word clouds of factors based on their net effect scores.

From Table 18 and Table 19 and Figure 5, we can state the following conclusions:

Regarding their impact on other factors, information quality (C₂), customizability (C₄), and language proficiency (C₅) stand out as the most potent factors, whereas integration features (C₃) and cultural elements (C₁₃) rank fourth and fifth in influence, respectively. Interestingly, a relationship between quality of information and usefulness is again highlighted, with a score of 0.215. Also, usefulness (C₁) shows the strongest impact on human involvement (C₆). In the broader context, the technological dimension and user-related factors assume greater significance than the social dimension. Moreover, technology-related factors predominantly contribute to causation, whereas factors stemming from the users’ dimension exert the most significant influence in terms of effects.

Regarding the influenced variables, human involvement (C₆), habit formation (C₉), and promotional strategies (C₁₁) are notably impacted by other variables. Human involvement (C₆) emerges as the most affected factor, necessitating the consideration of technological-related factors with significant influence on it. Furthermore, the perception of chatbots (C₇) is notably influenced by chatbot-related technology, while promotion (C₁₁) is significantly impacted by both technology factors (C₁, C₂, C₃, C₄, C₅, and C₆) and societal factors (C₁₀ and C₁₂).

4.3. Hierarchical Relationships

Data from total influence T are used to create the initial reachability matrix L. After that, final reachability matrix F and level partition tables are also obtained, as shown below (Table 20, Table 21, Table 22, Table 23 and Table 24).

Finally, the hierarchical structure is developed based on the level partitioning results, as shown in Figure 6. As expected, some of the factors shown to be enablers in each dimension are the enablers for the whole system, namely the quality of information (C₂), customization (C₄), language ability (C₅), cost (C₁₀), and culture (C₁₃). Interestingly, network externalities (C₁₄) are a bridge between the base of the system and other factors (Figure 7).

5. Discussion

This section will suggest managerial and theoretical implications based on the results.

5.1. Managerial Implications

The study sheds light on the relatively nascent domain of healthcare chatbot adoption, offering insights that can inform strategies aimed at enhancing public acceptance of these technological solutions. The findings underscore the paramount importance of information quality (C₂) provided by healthcare chatbots, as it emerges as the most influential causal factor with the highest net effect score of 0.792, significantly impacting other adoption factors, including the usefulness of chatbots (C₁). Several studies [9,65] also emphasize the importance of information quality due to its crucial role in delivering high-quality healthcare chatbot services. Furthermore, the effective exchange of information enhances the adoption of public transportation chatbots [66]. Therefore, developers and enterprises should prioritize efforts to enhance the quality of information delivered by chatbots to increase their adoption.

Secondly, beyond information quality (C₁), various other capabilities of chatbots (C₃, C₄, and C₅) also exhibit substantial impacts on their usefulness (C₁), a critical quality of chatbots and the whole adoption system. This aligns with Hsu‘s research [24], which identifies the technological dimension as a key influence in the healthcare chatbot introduction system. Thus, efforts to enhance these capabilities represent promising avenues for future improvement in chatbots’ usefulness and chatbot adoption. Thirdly, while promotional campaigns (C₁₁) can serve as effective tools to bolster chatbot acceptance among the public [31], it is noteworthy that this factor is primarily influenced by technological factors. This can be explained by the fact that these factors, including information quality (C₂), significantly impact usefulness as the core quality of chatbots, while this quality can lead to more effective advertising [67]. Consequently, the advanced technology of chatbots should be a focal point to ensure the success of promotional initiatives. Fourthly, the perception of chatbots (C₇) is influenced by their usefulness (C₁) and other technological factors. In a study involving a data-collecting chatbot [68], respondents’ perceptions about chatbots changed after engaging with them because of their natural language abilities. Another study on a voice chatbot (Apple Siri) found that the chatbot’s technological advancements, which enable it to perform tasks like a human, lead to a positive attitude towards it [69]. Therefore, efforts to alter public perceptions of chatbots should commence with enhancing the capabilities of these technological solutions.

Fifthly, in addition to technical capabilities, the usefulness of chatbots is significantly influenced by human assistance (C₆), with the combination of chatbots and human support being perceived as more trustworthy than chatbots alone [33]. A similar result was also found in a study on journaling guidance chatbots [70]. Hence, when optimizing the usefulness of chatbots in healthcare settings, their acceptance necessitates the consideration of the extent of human involvement. Lastly, the cost of health chatbots (C₁₀) is also one of the factors constructing the base of the ISM model, which affects many other factors in the system. The demand for low-cost and available healthcare solutions is increasing in developing countries [71]. Therefore, if the cost of health chatbots is lowered or even becomes free, it will not only facilitate health chatbot health adoption in general, but will also facilitate other factors in the overall system.

5.2. Theoretical Implications

Firstly, experts’ input is weighted based on their level of expertise, which mirrors their deep understanding of the subject matter. To attain a comprehensive perspective on health chatbot adoption, it is crucial to assemble a diverse panel of experts who engage with the topic from various angles. However, this diversity may lead to inconsistencies in their responses. Nevertheless, this approach helps mitigate the variability stemming from differing levels of expertise while incorporating all stakeholders’ viewpoints. Secondly, the utilization of Z-numbers is employed to handle the uncertainty surrounding the topic of health chatbot adoption. Given the novelty of the subject, there are still many misconceptions and areas of ambiguity. Therefore, assigning a reliability level to each response helps address this issue. This method enables experts to evaluate the subject while reducing the impact of uncertainty. Thirdly, the combined research framework of DEMATEL and ISM offers enterprises and governments clearer insights into the relationships among factors influencing health chatbot adoption. This approach not only explores relationships among factors but also organizes them hierarchically, aiding stakeholders in further research and future investments. The study also categorizes factors into three dimensions: technology, users, and society. This categorization helps stakeholders associated with each dimension better understand their role in the system and facilitates the decision-making process.

6. Conclusions and Remarks

The study sheds light on the factors influencing healthcare chatbot adoption. The study discovered key factors and explored the relationships among them. The findings underscore the critical role of technological factors in the adoption of healthcare chatbots, particularly emphasizing the significance of information quality as a primary causal determinant within the system. Technological factors predominantly influence factors within other dimensions, including user and societal aspects. Based on these insights, practical applications are proposed. Particularly, to promote the widespread adoption of healthcare chatbots, it is essential to allocate resources towards enhancing their capabilities.

However, this study has some limitations, which warrant consideration for future refinement. Firstly, the extensive array of factors and questions within the questionnaire poses a risk of inconsistency in evaluation due to the potential burden on respondents [46]. Streamlining the questionnaire to mitigate this workload could enhance the reliability of responses. Secondly, while the study adeptly elucidates the relationships among factors influencing chatbot adoption, it falls short of discerning the relative importance of the factors compared to others. To address this limitation, future research could employ an analytic network process methodology to rank factors, thereby offering clearer insights into their significance. Lastly, the reliance on survey-based data collection may restrict the depth of insights obtained, compared to face-to-face interviews. Embracing interview-based approaches in future studies could afford researchers a more nuanced understanding of the issue at hand, potentially yielding richer data and insights.

Author Contributions

Conceptualization, J.J.H.L.; methodology, J.J.H.L.; validation, J.J.H.L.; formal analysis, T.T.V.; investigation, T.T.V.; data writing—original draft preparation, T.T.V.; writing—review and editing, J.J.H.L.; visualization, T.T.V.; supervision, J.J.H.L. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Not applicable.

Data Availability Statement

The original contributions presented in the study are included in the article; further inquiries can be directed to the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1. Research flow chart.

Figure 2. Cause and effect graph—technology.

Figure 3. Cause and effect graph—users.

Figure 4. Cause and effect graph—society.

Figure 5. Casual diagram.

Figure 6. Word cloud based on net effect.

Figure 7. Hierarchical structure.

Table 1

Keywords for identifying factors.

Terms	Keywords
healthcare	medical care, health service, hospital
chatbot	virtual assistant, conversational agent, AI chat interface
adoption	acceptance, integration, acceptability, implementation
factor	element, component, determinant, aspect

Table 2

List of factors.

Code Name	Factors	Factor Description	References
C 1	Usefulness	Healthcare chatbots’ ability to meet users’ demands, such as seeking information, booking appointments, diagnosis, etc.	[9,27,28,29,30]
C 2	Integration	Healthcare chatbots are integrated in popular systems such as social media platforms, smart devices, etc.	[31]
C 3	Security	Healthcare chatbots are safe to use without any potential harms such as data privacy, misleading information, etc.	[9,31]
C 4	Quality of information	Healthcare chatbots are able to provide reliable and accurate information.	[9,32]
C 5	Customization	Healthcare chatbots are customizable to suit each individual user.	[31]
C 6	Natural language ability	Healthcare chatbots are able to communicate with users naturally like a human.	[31]
C 7	Human involvement	Healthcare chatbot services are partly supported by humans.	[9,33]
C 8	Chatbot perception	Users’ beliefs and attitudes towards healthcare chatbots.	[9,27,34,35]
C 9	IT literacy	Users’ IT skills such as programming, software applications, data analysis, etc.	[9]
C 10	Level of education	Users’ academic qualifications.	[36]
C 11	Personal innovativeness	Users have a high level of innovativeness and tend to look for new things.	[9,28,37]
C 12	Health issue	Health problems do not require specialist knowledge or have a low level of severity or/and high-level stigma.	[9,33]
C 13	Hedonic motivation	Users experience pleasure when using healthcare chatbots.	[27,38]
C 14	Habit	Users have daily life habits related to healthcare chatbots such as using smartphones, seeking medical advice online, etc.	[28,36]
C 15	Promotion	Healthcare chatbots are promoted to the public population.	[9,31]
C 16	Cost	Healthcare chatbots are cheap or free to use.	[40,41]
C 17	Social influence	Healthcare chatbots are recommended by important people such as friends, parents, relatives, etc.	[41]
C 18	Healthcare infrastructure	Healthcare infrastructure where users live is good.	[42,43]
C 19	Cultural norms	Cultural values and orientations support the adoption of healthcare chatbots.	[44]
C 20	Network externalities	Value of chatbots increases when the number of users increases.	[28]

Table 3

Influence linguistic variables.

Influence Linguistic Variables	Code	Equivalent TFNs
No Influence	NO	(0, 0, 1)
Very Low Influence	VL	(0, 1, 2)
Low Influence	L	(1, 2, 3)
High Influence	H	(2, 3, 4)
Very High Influence	VH	(3, 4, 4)

Table 4

Reliability linguistic variables.

Reliability Linguistic Terms	Code	Fuzzy Numbers
Strongly Unlikely (Very Low)	SU	(0, 0, 0.3)
Unlikely (Low)	U	(0.1, 0.3, 0.5)
Neutral (Medium)	N	(0.3, 0.5, 0.7)
Likely (High)	L	(0.5, 0.7, 0.9)
Strongly Likely (Very High)	SL	(0.7, 1.0, 1.0)

Table 5

Expert panel.

Code	Gender	Job Title	Area	Education	Experience (Years)	Weight
1	M	Professor	IT and AI application	PhD	>10	12.92%
2	M	Professor	Public health and healthcare technology application	PhD	>10	14.35%
3	M	Researcher	Healthcare technology application	PhD	>10	14.35%
4	F	Researcher	Healthcare technology application	Master	3–5	11.48%
5	M	Developer	Healthcare chatbots	Bachelor	3–5	10.05%
6	M	Developer	Healthcare chatbots	Bachelor	3–5	10.05%
7	F	Professional	Chatbot deployment	Master	1–2	9.09%
8	F	Experienced user	Healthcare chatbots	Bachelor	1–2	8.61%
9	M	Experienced user	Healthcare chatbots	Master	1–2	9.09%

Table 6

Factor filtering.

Code Name	Factors	Significance Level	Status	Recode
C 1	Usefulness	100%	Selected	C 1
C 2	Integration	77%	Selected	C 2
C 3	Security	64%	Deleted
C 4	Quality of information	87%	Selected	C 3
C 5	Customization	100%	Selected	C 4
C 6	Natural language ability	78%	Selected	C 5
C 7	Human involvement	100%	Selected	C 6
C 8	Chatbot perception	77%	Selected	C 7
C 9	IT literacy	22%	Deleted
C 10	Level of education	36%	Deleted
C 11	Personal innovativeness	48%	Deleted
C 12	Health issue	49%	Deleted
C 13	Hedonic motivation	78%	Selected	C 8
C 14	Habit	76%	Selected	C 9
C 15	Promotion	87%	Selected	C 10
C 16	Cost	82%	Selected	C 11
C 17	Social influence	90%	Selected	C 12
C 18	Healthcare infrastructure	64%	Deleted
C 19	Cultural norms	87%	Selected	C 13
C 20	Network externalities	82%	Selected	C 14

Table 7

Z-number linguistic variables.

Z-Number Linguistics	Lower Bound	Middle Bound	Upper Bound
NO, SU	0	0	0.316
NO, U	0	0	0.548
NO, N	0	0	0.707
NO, L	0	0	0.837
NO, SL	0	0	0.949
VL, SU	0	0.316	0.632
VL, U	0	0.548	1.095
VL, N	0	0.707	1.414
VL, L	0	0.837	1.673
VL, SL	0	0.949	1.897
L, SU	0.316	0.632	0.949
L, U	0.548	1.095	1.643
L, N	0.707	1.414	2.121
L, L	0.837	1.673	2.510
L, SL	0.949	1.897	2.846
H, SU	0.632	0.949	1.265
H, U	1.095	1.643	2.191
H, N	1.414	2.121	2.828
H, L	1.673	2.510	3.347
H, SL	1.897	2.846	3.795
VH, SU	0.949	1.265	1.265
VH, U	1.644	2.191	2.191
VH, N	2.122	2.829	2.829
VH, L	2.51	3.347	3.347
VH, SL	2.847	3.795	3.795

Table 8

Expert 1’s assessment.

	C 1	C 2	C 3	…	C 14
C 1	0, 0	H, L	VH, SL	…	L, N
C 2	H, SL	0, 0	L, L	…	VL, L
C 3	L, N	L, N	0, 0	…	L, N
…	…	…	…	…	…
C 14	VH, L	L, U	L, U	…	0, 0

Table 9

Expert 1′s assessment converted into fuzzy numbers.

	C 1	C 2	C 3	…	C 14
C 1	(0, 0, 0)	(1.674, 2.51, 3.347)	(2.847, 3.795, 3.795)	…	(0.708, 1.415, 2.122)
C 2	(1.898, 2.847, 3.795)	(0, 0, 0)	(0.837, 1.674, 2.51)	…	(0, 0.837, 1.674)
C 3	(0.708, 1.415, 2.122)	(0.708, 1.415, 2.122)	(0, 0, 0)	…	(0.708, 1.415, 2.122)
…	…	…	…	…	…
C 14	(2.51, 3.347, 3.347)	(0.548, 1.096, 1.644)	(0.548, 1.096, 1.644)	…	(0, 0, 0)

Table 10

Defuzzified numbers of expert 1’s assessment.

	C 1	C 2	C 3	…	C 14
C 1	0	2.51	3.479	…	1.415
C 2	2.847	0	1.674	…	0.837
C 3	1.415	1.415	0	…	1.415
…	…	…	…	…	…
C 14	3.068	1.096	1.096	…	0

Table 11

Average matrix—technology.

A T	C 1	C 2	C 3	C 4	C 5	C 6	Sum
C 1	0.000	0.111	0.128	0.106	0.115	0.288	0.748
C 2	0.329	0.000	0.097	0.077	0.103	0.289	0.893
C 3	0.219	0.075	0.000	0.116	0.113	0.209	0.732
C 4	0.290	0.097	0.182	0.000	0.116	0.209	0.894
C 5	0.297	0.061	0.160	0.112	0.000	0.213	0.843
C 6	0.214	0.150	0.174	0.129	0.093	0.000	0.760
Sum	1.349	0.494	0.740	0.540	0.540	1.208

Table 12

Average matrix—users.

A U	C 7	C 8	C 9	Sum
C 7	0.000	0.232	0.156	0.388
C 8	0.102	0.000	0.313	0.416
C 9	0.127	0.184	0.000	0.311
Sum	0.229	0.417	0.469

Table 13

Average matrix—society.

A S	C 10	C 11	C 12	C 13	C 14	Sum
C 10	0.000	0.152	0.168	0.079	0.068	0.467
C 11	0.144	0.000	0.154	0.092	0.124	0.514
C 12	0.045	0.156	0.000	0.119	0.113	0.433
C 13	0.069	0.075	0.207	0.000	0.111	0.463
C 14	0.135	0.077	0.157	0.095	0.000	0.465
Sum	0.394	0.460	0.687	0.385	0.417

Table 14

Total influence matrix—technology.

T T	C 1	C 2	C 3	C 4	C 5	C 6
C 1	0.218	0.177	0.223	0.177	0.178	0.388
C 2	0.456	0.119	0.225	0.175	0.188	0.429
C 3	0.352	0.151	0.133	0.181	0.175	0.339
C 4	0.435	0.183	0.278	0.121	0.197	0.380
C 5	0.424	0.155	0.256	0.192	0.111	0.369
C 6	0.360	0.200	0.251	0.191	0.167	0.215

Threshold is 0.244.

Table 15

Total influence matrix—users.

T U	C 7	C 8	C 9
C 7	0.749	1.483	1.572
C 8	0.946	1.158	1.756
C 9	0.845	1.249	1.115

Threshold is 1.208.

Table 16

Total influence matrix—society.

T S	C 10	C 11	C 12	C 13	C 14
C 10	0.257	0.501	0.619	0.371	0.377
C 11	0.454	0.341	0.639	0.406	0.458
C 12	0.313	0.477	0.398	0.399	0.412
C 13	0.338	0.407	0.652	0.264	0.419
C 14	0.417	0.414	0.604	0.385	0.278

Threshold is 0.424.

Table 17

Average matrix.

A	C 1	C 2	C 3	C 4	C 5	C 6	C 7	C 8	C 9	C 10	C 11	C 12	C 13	C 14
C 1	0.000	0.041	0.047	0.039	0.042	0.055	0.106	0.080	0.096	0.038	0.058	0.058	0.021	0.059
C 2	0.121	0.000	0.036	0.028	0.038	0.057	0.106	0.077	0.066	0.033	0.055	0.059	0.027	0.038
C 3	0.080	0.028	0.000	0.043	0.042	0.053	0.077	0.052	0.059	0.033	0.061	0.061	0.023	0.079
C 4	0.107	0.036	0.067	0.000	0.043	0.044	0.077	0.068	0.055	0.039	0.055	0.041	0.040	0.026
C 5	0.109	0.023	0.059	0.041	0.000	0.053	0.078	0.051	0.034	0.030	0.052	0.037	0.024	0.026
C 6	0.014	0.023	0.036	0.030	0.032	0.000	0.059	0.085	0.057	0.049	0.057	0.036	0.020	0.036
C 7	0.079	0.055	0.064	0.047	0.034	0.071	0.000	0.038	0.046	0.028	0.060	0.022	0.026	0.033
C 8	0.032	0.020	0.026	0.054	0.015	0.038	0.073	0.000	0.115	0.051	0.061	0.057	0.050	0.051
C 9	0.056	0.030	0.025	0.036	0.033	0.047	0.093	0.068	0.000	0.029	0.033	0.040	0.019	0.029
C 10	0.031	0.020	0.031	0.027	0.028	0.033	0.086	0.061	0.052	0.000	0.056	0.062	0.029	0.025
C 11	0.019	0.035	0.032	0.030	0.021	0.037	0.062	0.052	0.043	0.053	0.000	0.057	0.034	0.046
C 12	0.042	0.019	0.034	0.025	0.029	0.043	0.052	0.053	0.066	0.016	0.058	0.000	0.044	0.042
C 13	0.037	0.039	0.023	0.017	0.025	0.053	0.068	0.045	0.084	0.025	0.027	0.076	0.000	0.041
C 14	0.059	0.047	0.040	0.029	0.047	0.027	0.062	0.045	0.052	0.050	0.028	0.058	0.035	0.000

Table 18

Total influence matrix.

T	C 1	C 2	C 3	C 4	C 5	C 6	C 7	C 8	C 9	C 10	C 11	C 12	C 13	C 14
C 1	0.102	0.098	0.117	0.102	0.099	0.230	0.137	0.179	0.201	0.103	0.146	0.141	0.075	0.129
C 2	0.215	0.060	0.108	0.093	0.096	0.234	0.140	0.178	0.178	0.099	0.145	0.143	0.081	0.112
C 3	0.171	0.083	0.068	0.101	0.096	0.196	0.130	0.147	0.161	0.094	0.143	0.140	0.074	0.144
C 4	0.198	0.091	0.133	0.062	0.098	0.201	0.124	0.164	0.162	0.101	0.140	0.123	0.090	0.098
C 5	0.189	0.073	0.119	0.095	0.051	0.188	0.124	0.139	0.131	0.086	0.129	0.111	0.069	0.090
C 6	0.160	0.102	0.121	0.099	0.083	0.113	0.138	0.126	0.139	0.084	0.133	0.096	0.070	0.095
C 7	0.087	0.064	0.086	0.076	0.073	0.151	0.060	0.155	0.136	0.095	0.120	0.098	0.060	0.088
C 8	0.117	0.071	0.086	0.105	0.066	0.182	0.109	0.089	0.203	0.105	0.134	0.129	0.095	0.110
C 9	0.129	0.074	0.079	0.083	0.075	0.185	0.108	0.141	0.084	0.077	0.100	0.103	0.060	0.083
C 10	0.103	0.063	0.083	0.073	0.070	0.176	0.094	0.133	0.132	0.048	0.120	0.122	0.069	0.078
C 11	0.091	0.075	0.081	0.074	0.062	0.151	0.095	0.123	0.121	0.097	0.064	0.116	0.072	0.095
C 12	0.111	0.061	0.083	0.070	0.069	0.143	0.101	0.123	0.142	0.064	0.118	0.062	0.081	0.092
C 13	0.112	0.082	0.076	0.066	0.069	0.164	0.115	0.123	0.164	0.074	0.095	0.137	0.041	0.094
C 14	0.139	0.092	0.096	0.079	0.093	0.165	0.095	0.126	0.139	0.099	0.100	0.125	0.077	0.059

Threshold is 0.110.

Table 19

Cause and effect rankings.

	r	g	r + g	r − g	Significance Rankings	Cause Rankings	Effect Rankings
C 1	1.861	1.923	3.784	−0.062	2	8	7
C 2	1.881	1.089	2.970	0.792	7	1	14
C 3	1.747	1.337	3.084	0.410	5	4	11
C 4	1.786	1.179	2.965	0.607	9	2	13
C 5	1.593	1.098	2.692	0.495	12	3	12
C 6	1.558	2.479	4.037	−0.921	1	14	1
C 7	1.349	1.570	2.919	−0.221	10	9	6
C 8	1.601	1.945	3.546	−0.344	3	11	4
C 9	1.382	2.092	3.474	−0.711	4	13	2
C 10	1.363	1.225	2.589	0.138	13	6	9
C 11	1.317	1.689	3.007	−0.372	6	12	3
C 12	1.320	1.646	2.967	−0.326	8	10	5
C 13	1.413	1.013	2.426	0.400	14	5	10
C 14	1.483	1.367	2.850	0.116	11	7	8

Table 20

Initial reachability matrix.

L	C 1	C 2	C 3	C 4	C 5	C 6	C 7	C 8	C 9	C 10	C 11	C 12	C 13	C 14
C 1	1	0	1	0	0	1	1	1	1	0	1	1	0	1
C 2	1	1	0	0	0	1	1	1	1	0	1	1	0	1
C 3	1	0	1	0	0	1	1	1	1	0	1	1	0	1
C 4	1	0	1	1	0	1	1	1	1	0	1	1	0	0
C 5	1	0	1	0	1	1	1	1	1	0	1	1	0	0
C 6	0	0	0	0	0	1	1	1	1	0	1	0	0	0
C 7	1	0	1	0	0	1	1	1	1	0	1	0	0	0
C 8	1	0	0	0	0	0	1	1	1	0	1	1	0	0
C 9	1	0	0	0	0	0	1	1	1	0	0	0	0	0
C 10	0	0	0	0	0	0	1	1	1	1	1	1	0	0
C 11	0	0	0	0	0	0	1	1	1	0	1	1	0	0
C 12	1	0	0	0	0	0	1	1	1	0	1	1	0	0
C 13	1	0	0	0	0	1	1	1	1	0	0	1	1	0
C 14	1	0	0	0	0	0	1	1	1	0	0	1	0	1

Table 21

Final reachability matrix.

F	C 1	C 2	C 3	C 4	C 5	C 6	C 7	C 8	C 9	C 10	C 11	C 12	C 13	C 14
C 1	1	0	1	0	0	1	1	1	1	0	1	1	0	1
C 2	1	1	1	0	0	1	1	1	1	0	1	1	0	1
C 3	1	0	1	0	0	1	1	1	1	0	1	1	0	1
C 4	1	0	1	1	0	1	1	1	1	0	1	1	0	1
C 5	1	0	1	0	1	1	1	1	1	0	1	1	0	1
C 6	1	0	1	0	0	1	1	1	1	0	1	1	0	1
C 7	1	0	1	0	0	1	1	1	1	0	1	1	0	0
C 8	1	0	1	0	0	1	1	1	1	0	1	1	0	1
C 9	1	0	1	0	0	1	1	1	1	0	1	1	0	1
C 10	1	0	1	0	0	1	1	1	1	1	1	1	0	1
C 11	1	0	1	0	0	1	1	1	1	0	1	1	0	1
C 12	1	0	1	0	0	1	1	1	1	0	1	1	0	1
C 13	1	0	1	0	0	1	1	1	1	0	1	1	1	1
C 14	1	0	1	0	0	1	1	1	1	0	1	1	0	1

Table 22

Level partition—Level I.

Factor	Reachability Set	Antecedent Set	Intersection	Level
C 1	C1, C3, C6, C7, C8, C9, C11, C12, C14	C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12, C13, C14	C1, C3, C6, C7, C8, C9, C11, C12, C14	I
C 2	C1, C2, C3, C6, C7, C8, C9, C11, C12, C14	C 2	C 2
C 3	C1, C3, C6, C7, C8, C9, C11, C12, C14	C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12, C13, C14	C1, C3, C6, C7, C8, C9, C11, C12, C14	I
C 4	C1, C3, C4, C6, C7, C8, C9, C11, C12, C14	C 4	C 4
C 5	C1, C3, C5, C6, C7, C8, C9, C11, C12, C14	C 5	C 5
C 6	C1, C3, C6, C7, C8, C9, C11, C12, C14	C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12, C13, C14	C1, C3, C6, C7, C8, C9, C11, C12, C14	I
C 7	C1, C3, C6, C7, C8, C9, C11, C12, C14	C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12, C13, C14	C1, C3, C6, C7, C8, C9, C11, C12, C14	I
C 8	C1, C3, C6, C7, C8, C9, C11, C12, C14	C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12, C13, C14	C1, C3, C6, C7, C8, C9, C11, C12, C14	I
C 9	C1, C3, C6, C7, C8, C9, C11, C12, C14	C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12, C13, C14	C1, C3, C6, C7, C8, C9, C11, C12, C14	I
C 10	C1, C3, C6, C7, C8, C9, C10, C11, C12, C14	C 10	C 10
C 11	C1, C3, C6, C7, C8, C9, C11, C12, C14	C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12, C13, C14	C1, C3, C6, C7, C8, C9, C11, C12, C14	I
C 12	C1, C3, C6, C7, C8, C9, C11, C12, C14	C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12, C13, C14	C1, C3, C6, C7, C8, C9, C11, C12, C14	I
C 13	C1, C3, C6, C7, C8, C9, C11, C12, C13, C14	C 13	C 13
C 14	C1, C3, C6, C7, C8, C9, C11, C12, C14	C1, C3, C7, C8, C9, C11, C12, C14	C1, C3, C7, C8, C9, C11, C12, C14

Table 23

Level partition—Level II.

Factor	Reachability Set	Antecedent Set	Intersection	Level
C 2	C2, C14	C 2	C 2
C 4	C4, C14	C 4	C 4
C 5	C5, C14	C 5	C 5
C 10	C10, C14	C 10	C 10
C 13	C13, C14	C 13	C 13
C 14	C 14	C 14	C 14	II

Table 24

Level partition—Level III.

Factor	Reachability Set	Antecedent Set	Intersection	Level
C 2	C 2	C 2	C 2	III
C 4	C 4	C 4	C 4	III
C 5	C 5	C 5	C 5	III
C 10	C 10	C 10	C 10	III
C 13	C 13	C 13	C 13	III

Appendix A

Table A1

The first questionnaire (sample).

Expert’s Evaluation
Dimension	Factors	Factor Description	Expert’s Opinion
			Yes	No
Technology	Usefulness	Healthcare chatbots are able to meet users’ demands such as seeking information, booking appointments, and diagnosis, etc.
	Integration	Healthcare chatbots are integrated in popular systems such as social media platforms, smart devices, etc.
	Security	Healthcare chatbots are safe to use without any potential harms such as data privacy, misleading information, etc.
	Quality of information	Healthcare chatbots are able to provide reliable and accurate information
	Customization	Healthcare chatbots can be customized to suit each individual user.
	Natural language ability	Healthcare chatbots are able to communicate with users naturally like a human.
	Human involvement	Healthcare chatbot services is partly supported by humans.

Appendix B

Table A2

The second questionnaire (sample).

For example: Factor “Usefulness” has no influence on factor “Cost” with very high confidence level and has high influence on factor “Chatbot perception” with high confidence level, we have the table, as follow.
Usefulness		Confidence Level
Cost	NO	SL
Chatbot perception	H	L
Expert’s Evaluations
Usefulness		Confidence Level
Integration
Quality of information
Customization
Natural language ability
Human involvement
Chatbot perception
Hedonic motivation
Habit
Promotion
Cost
Social influence
Cultural norms
Network externalities

References

1. Team, T.I. Healthcare Sector: Industries Defined and Key Statistics. 2021; Available online: https://www.investopedia.com/terms/h/health_care_sector.asp (accessed on 25 April 2024).

2. Lee, D. Effects of key value co-creation elements in the healthcare system: Focusing on technology applications. Serv. Bus.; 2019; 13, pp. 389-417. [DOI: https://dx.doi.org/10.1007/s11628-018-00388-9]

3. Chung, K.; Park, R.C. Chatbot-based healthcare service with a knowledge base for cloud computing. Clust. Comput.; 2019; 22, pp. 1925-1937. [DOI: https://dx.doi.org/10.1007/s10586-018-2334-5]

4. Touati, F.; Tabish, R. U-healthcare system: State-of-the-art review and challenges. J. Med. Syst.; 2013; 37, 9949. [DOI: https://dx.doi.org/10.1007/s10916-013-9949-0] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/23640734]

5. Adamopoulou, E.; Moussiades, L. Chatbots: History, technology, and applications. Mach. Learn. Appl.; 2020; 2, 100006. [DOI: https://dx.doi.org/10.1016/j.mlwa.2020.100006]

6. Bansal, H.; Khan, R. A review paper on human computer interaction. Int. J. Adv. Res. Comput. Sci. Softw. Eng.; 2018; 8, pp. 53-56. [DOI: https://dx.doi.org/10.23956/ijarcsse.v8i4.630]

7. Athota, L.; Shukla, V.K.; Pandey, N.; Rana, A. Chatbot for healthcare system using artificial intelligence. Proceedings of the 2020 8th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO); Noida, India, 4 June 2020.

8. Bhirud, N.; Tataale, S.; Randive, S.; Nahar, S. A literature review on chatbots in healthcare domain. Int. J. Sci. Technol. Res.; 2019; 8, pp. 225-231.

9. Nadarzynski, T.; Miles, O.; Cowie, A.; Ridge, D. Acceptability of artificial intelligence (AI)-led chatbot services in healthcare: A mixed-methods study. Digit. Health; 2019; 5, 2055207619871808. [DOI: https://dx.doi.org/10.1177/2055207619871808] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31467682]

10. Ahmed, A.; Ali, N.; Aziz, S.; A Abd-Alrazaq, A.; Hassan, A.; Khalifa, M.; Elhusein, B.; Ahmed, M.; Ahmed, M.A.S.; Househ, M. A review of mobile chatbot apps for anxiety and depression and their self-care features. Comput. Methods Programs Biomed. Update; 2021; 1, 100012. [DOI: https://dx.doi.org/10.1016/j.cmpbup.2021.100012]

11. Xu, L.; Sanders, L.; Li, K.; Chow, J.C.L. Chatbot for health care and oncology applications using artificial intelligence and machine learning: Systematic review. JMIR Cancer; 2021; 7, e27850. [DOI: https://dx.doi.org/10.2196/27850]

12. Viduani, A.; Cosenza, V.; Araújo, R.M.; Kieling, C. Chatbots in the Field of Mental Health: Challenges and Opportunities. Digital Mental Health: A Practitioner’s Guide; Springer: Cham, Switzerland, 2023; pp. 133-148.

13. Almalki, M.; Azeez, F. Health chatbots for fighting COVID-19: A scoping review. Acta Inform. Medica; 2020; 28, 241. [DOI: https://dx.doi.org/10.5455/aim.2020.28.241-247]

14. Akdag, H.; Kalaycı, T.; Karagöz, S.; Zülfikar, H.; Giz, D. The evaluation of hospital service quality by fuzzy MCDM. Appl. Soft Comput.; 2014; 23, pp. 239-248. [DOI: https://dx.doi.org/10.1016/j.asoc.2014.06.033]

15. Esfahani, A.A.; Ahmadi, H.; Nilashi, M.; Alizadeh, M.; Bashiri, A.; Farajzadeh, M.A.; Shahmoradi, L.; Rasouli, H.R.; Hekmat, M. An evaluation model for the implementation of hospital information system in public hospitals using multi-criteria-decision-making (MCDM) approaches. Int. J. Eng. Technol. (UAE); 2018; 7, pp. 1-18. [DOI: https://dx.doi.org/10.14419/ijet.v7i1.8404]

16. Verma, P.; Dutta, S. Critical success factors for better healthcare: An MCDM approach. Int. J. Technol. Mark.; 2021; 15, pp. 287-315. [DOI: https://dx.doi.org/10.1504/IJTMKT.2021.118222]

17. Sir, E.; Batur Sir, G.D. Evaluating treatment modalities in chronic pain treatment by the multi-criteria decision making pro-cedure. BMC Med. Inform. Decis. Mak.; 2019; 19, 191. [DOI: https://dx.doi.org/10.1186/s12911-019-0925-6] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31615483]

18. Tamanini, I.; de Castro, A.K.; Pinheiro, P.R.; Pinheiro, M.C. Applied neuroimaging to the diagnosis of Alzheimer’s disease: A multicriteria model. Proceedings of the Best Practices for the Knowledge Society. Knowledge, Learning, Development and Technology for All: Second World Summit on the Knowledge Society, WSKS 2009, Chania; Crete, Greece, 16–18 September 2009.

19. Yildirim, F.S.; Sayan, M.; Sanlidag, T.; Uzun, B.; Ozsahin, D.U.; Ozsahin, I. Comparative evaluation of the treatment of COVID-19 with multicriteria decision-making techniques. J. Healthc. Eng.; 2021; 2021, 8864522. [DOI: https://dx.doi.org/10.1155/2021/8864522] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33552457]

20. Büyüközkan, G.; Mukul, E. Evaluation of smart health technologies with hesitant fuzzy linguistic MCDM methods. J. Intell. Fuzzy Syst.; 2020; 39, pp. 6363-6375. [DOI: https://dx.doi.org/10.3233/JIFS-189103]

21. Liou, J.J.H.; Lu, M.-T.; Hu, S.-K.; Cheng, C.-H.; Chuang, Y.-C. A hybrid MCDM model for improving the electronic health record to better serve client needs. Sustainability; 2017; 9, 1819. [DOI: https://dx.doi.org/10.3390/su9101819]

22. Lu, M.-T.; Lin, S.-W.; Tzeng, G.-H. Improving RFID adoption in Taiwan’s healthcare industry based on a DEMATEL technique with a hybrid MCDM model. Decis. Support Syst.; 2013; 56, pp. 259-269. [DOI: https://dx.doi.org/10.1016/j.dss.2013.06.006]

23. Santa Barletta, V.; Caivano, D.; Colizzi, L.; Dimauro, G.; Piattini, M. Clinical-chatbot AHP evaluation based on “quality in use” of ISO/IEC 25010. Int. J. Med. Inform.; 2023; 170, 104951. [DOI: https://dx.doi.org/10.1016/j.ijmedinf.2022.104951]

24. Hsu, M.-C. The construction of critical factors for successfully introducing chatbots into mental health services in the army: Using a hybrid MCDM approach. Sustainability; 2023; 15, 7905. [DOI: https://dx.doi.org/10.3390/su15107905]

25. Phooriyaphan, S.; Rachsiriwatcharabul, N. Identifying the Criteria for Selection of Healthcare Chatbot in Thailand Using A Multi-Criteria Decision Making Approach. J. Posit. Sch. Psychol.; 2022; 6, pp. 3757-3763.

26. Davis, F.D. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q.; 1989; 13, pp. 319-340. [DOI: https://dx.doi.org/10.2307/249008]

27. Almalki, M. Exploring the Influential Factors of Consumers’ Willingness Toward Using COVID-19 Related Chatbots: An Empirical Study. Med. Arch.; 2021; 75, 50. [DOI: https://dx.doi.org/10.5455/medarh.2021.75.50-55]

28. Huang, C.Y.; Yang, M.C.; Chen, I.M.; Hsu, W.C. Modeling consumer adoption intention of an AI-powered health chatbot in Taiwan: An empirical perspective. Int. J. Perform. Eng.; 2022; 18, 338.

29. Patil, K.; Kulkarni, M. Can we trust Health and Wellness Chatbot going mobile? Empirical research using TAM and HBM. Proceedings of the 2022 IEEE Region 10 Symposium (TENSYMP); Mumbai, India, 1–3 July 2022.

30. Li, X.; Xie, S.; Ye, Z.; Ma, S.; Yu, G. Investigating patients’ continuance intention toward conversational agents in outpatient departments: Cross-sectional field survey. J. Med. Internet Res.; 2022; 24, e40681. [DOI: https://dx.doi.org/10.2196/40681] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36342768]

31. Chew, H.S.J.; Achananuparp, P. Perceptions and needs of artificial intelligence in health care to increase adoption: Scoping review. J. Med. Internet Res.; 2022; 24, e32939. [DOI: https://dx.doi.org/10.2196/32939] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35029538]

32. Miner, A.S.; Milstein, A.; Hancock, J.T. Talking to machines about personal mental health problems. JAMA; 2017; 318, pp. 1217-1218. [DOI: https://dx.doi.org/10.1001/jama.2017.14151] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28973225]

33. Miles, O.; West, R.; Nadarzynski, T. Health chatbots acceptability moderated by perceived stigma and severity: A cross-sectional survey. Digit. Health; 2021; 7, 20552076211063012. [DOI: https://dx.doi.org/10.1177/20552076211063012]

34. Hussain, S.A.; Ogundimu, F.; Bhattarai, S. Mobile phone-based chatbot for family planning and contraceptive information. Proceedings of the Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management. Healthcare Applications: 10th International Conference, DHM 2019, Held as Part of the 21st HCI International Conference, HCII 2019; Orlando, FL, USA, 26–31 July 2019.

35. Lai, Y.; Lioliou, E.; Panagiotopoulos, P. Understanding Users’ switching Intention to AI-Powered Healthcare Chatbots. Proceedings of the ECIS 2021; Marrakech, Morrocco, 14–16 June 2021.

36. Almalki, M. Perceived utilities of COVID-19 related chatbots in Saudi Arabia: A cross-sectional study. Acta Inform. Medica; 2020; 28, 218. [DOI: https://dx.doi.org/10.5455/aim.2020.28.219-224]

37. Barnes, P.M.; Bloom, B.; Nahin, R.L. Complementary and alternative medicine use among adults and children: United States, 2007. Natl. Health Stat. Rep.; 2008; 12, pp. 1-23.

38. Tamilmani, K.; Rana, N.P.; Prakasam, N.; Dwivedi, Y.K. The battle of Brain vs. Heart: A literature review and meta-analysis of “hedonic motivation” use in UTAUT2. Int. J. Inf. Manag.; 2019; 46, pp. 222-235. [DOI: https://dx.doi.org/10.1016/j.ijinfomgt.2019.01.008]

39. Wilson, L.; Marasoiu, M. The development and use of chatbots in public health: Scoping review. JMIR Hum. Factors; 2022; 9, e35882. [DOI: https://dx.doi.org/10.2196/35882] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36197708]

40. Liu, T.; Tsang, W.; Huang, F.; Lau, O.Y.; Chen, Y.; Sheng, J.; Guo, Y.; Akinwunmi, B.; Zhang, C.J.; Ming, W.K. Patients’ preferences for artificial intelligence applications versus clinicians in disease diagnosis during the SARS-CoV-2 pandemic in China: Discrete choice experiment. J. Med. Internet Res.; 2021; 23, e22841. [DOI: https://dx.doi.org/10.2196/22841]

41. Prakash, A.V.; Das, S. Intelligent conversational agents in mental healthcare services: A thematic analysis of user perceptions. Pac. Asia J. Assoc. Inf. Syst.; 2020; 12, 1. [DOI: https://dx.doi.org/10.17705/1thci.12201]

42. Jain, R.; Arora, A.; Raju, S. A novel adoption index of selected agricultural technologies: Linkages with infrastructure and productivity. Agric. Econ. Res. Rev.; 2009; 22, pp. 109-120.

43. Da Silveira, F.; Da Silva, S.L.; Machado, F.M.; Barbedo, J.G.; Amaral, F.G. Farmers’ perception of barriers that difficult the implementation of agriculture 4.0. Agric. Syst.; 2023; 208, 103656. [DOI: https://dx.doi.org/10.1016/j.agsy.2023.103656]

44. Chi, O.H.; Chi, C.G.; Gursoy, D.; Nunkoo, R. Customers’ acceptance of artificially intelligent service robots: The influence of trust and culture. Int. J. Inf. Manag.; 2023; 70, 102623. [DOI: https://dx.doi.org/10.1016/j.ijinfomgt.2023.102623]

45. Bulut, E.; Duru, O.; Yoshida, S. Market entry, asset returns, and irrational exuberance: Asset management anomalies in dry cargo shipping. Int. J. Shipp. Transp. Logist.; 2013; 5, pp. 652-667. [DOI: https://dx.doi.org/10.1504/IJSTL.2013.056851]

46. Bulut, E.; Duru, O. Analytic hierarchy process (AHP) in maritime logistics: Theory, application and fuzzy set integration. Multi-Criteria Decision Making in Maritime Studies and Logistics: Applications and Cases; Springer: Cham, Switzerland, 2018; pp. 31-78.

47. Gabus, A.; Fontela, E. World Problems, an Invitation to Further Thought within the Framework of DEMATEL; Battelle Geneva Research Center: Geneva, Switzerland, 1972; Volume 1, pp. 12-14.

48. Tzeng, G.-H.; Chiang, C.-H.; Li, C.-W. Evaluating intertwined effects in e-learning programs: A novel hybrid MCDM model based on factor analysis and DEMATEL. Expert Syst. Appl.; 2007; 32, pp. 1028-1044. [DOI: https://dx.doi.org/10.1016/j.eswa.2006.02.004]

49. Wu, W.-W.; Lee, Y.-T. Developing global managers’ competencies using the fuzzy DEMATEL method. Expert Syst. Appl.; 2007; 32, pp. 499-507. [DOI: https://dx.doi.org/10.1016/j.eswa.2005.12.005]

50. Wu, W.-W. Choosing knowledge management strategies by using a combined ANP and DEMATEL approach. Expert Syst. Appl.; 2008; 35, pp. 828-835. [DOI: https://dx.doi.org/10.1016/j.eswa.2007.07.025]

51. Sharma, M.; Sehrawat, R. Quantifying SWOT analysis for cloud adoption using FAHP-DEMATEL approach: Evidence from the manufacturing sector. J. Enterp. Inf. Manag.; 2020; 33, pp. 1111-1152. [DOI: https://dx.doi.org/10.1108/JEIM-09-2019-0276]

52. Lin, W.-R.; Wang, Y.-H.; Hung, Y.-M. Analyzing the factors influencing adoption intention of internet banking: Applying DEMATEL-ANP-SEM approach. PLoS ONE; 2020; 15, e0227852. [DOI: https://dx.doi.org/10.1371/journal.pone.0227852]

53. Yadav, H.; Soni, U.; Gupta, S.; Kumar, G. Evaluation of barriers in the adoption of E-commerce technology in SMEs: A fuzzy DEMATEL approach. J. Electron. Commer. Organ. (JECO); 2022; 20, pp. 1-18. [DOI: https://dx.doi.org/10.4018/JECO.292472]

54. Warfield, J.N. Societal systems planning, policy and complexity. Cybern. Syst.; 1978; 8, pp. 113-115. [DOI: https://dx.doi.org/10.1080/01969727808927573]

55. Sage, A.P. Methodology for Large-Scale Systems; McGraw-Hill College: New York, NY, USA, 1977.

56. Agarwal, R.; Bhadauria, A.; Kaushik, H.; Swami, S.; Rajwanshi, R. ISM-MICMAC-based study on key enablers in the adoption of solar renewable energy products in India. Technol. Soc.; 2023; 75, 102375. [DOI: https://dx.doi.org/10.1016/j.techsoc.2023.102375]

57. Kumar, S.; Raut, R.D.; Nayal, K.; Kraus, S.; Yadav, V.S.; Narkhede, B.E. To identify industry 4.0 and circular economy adoption barriers in the agriculture supply chain by using ISM-ANP. J. Clean. Prod.; 2021; 293, 126023. [DOI: https://dx.doi.org/10.1016/j.jclepro.2021.126023]

58. Raut, R.D.; Gardas, B.B.; Jha, M.K.; Priyadarshinee, P. Examining the critical success factors of cloud computing adoption in the MSMEs by using ISM model. J. High Technol. Manag. Res.; 2017; 28, pp. 125-141. [DOI: https://dx.doi.org/10.1016/j.hitech.2017.10.004]

59. Rana, N.P.; Barnard, D.J.; Baabdullah, A.M.; Rees, D.; Roderick, S. Exploring barriers of m-commerce adoption in SMEs in the UK: Developing a framework using ISM. Int. J. Inf. Manag.; 2019; 44, pp. 141-153. [DOI: https://dx.doi.org/10.1016/j.ijinfomgt.2018.10.009]

60. Tseng, M.L.; Sujanto, R.Y.; Ardaniah, V.; Bui, T.D.; Lan, S.; Negash, Y.T.; Tseng, J.W. Product-service system toward circularity: Hierarchical framework in Indonesia’s home appliance manufacturing industry under uncertainties. Ann. Oper. Res.; 2023; 327, pp. 1-27. [DOI: https://dx.doi.org/10.1007/s10479-023-05538-5]

61. Hsu, W.-C.J.; Liou, J.J.; Lo, H.-W. A group decision-making approach for exploring trends in the development of the healthcare industry in Taiwan. Decis. Support Syst.; 2021; 141, 113447. [DOI: https://dx.doi.org/10.1016/j.dss.2020.113447]

62. Kang, B.; Wei, D.; Li, Y.; Deng, Y. A method of converting Z-number to classical fuzzy number. J. Inf. Comput. Sci.; 2012; 9, pp. 703-709.

63. Mathiyazhagan, K.; Govindan, K.; NoorulHaq, A.; Geng, Y. An ISM approach for the barrier analysis in implementing green supply chain management. J. Clean. Prod.; 2013; 47, pp. 283-297. [DOI: https://dx.doi.org/10.1016/j.jclepro.2012.10.042]

64. Chakrabortty, R.K.; Abdel-Basset, M.; Ali, A.M. A multi-criteria decision analysis model for selecting an optimum customer service chatbot under uncertainty. Decis. Anal. J.; 2023; 6, 100168. [DOI: https://dx.doi.org/10.1016/j.dajour.2023.100168]

65. Liu, C.; Zowghi, D.; Peng, G.; Kong, S. Information quality of conversational agents in healthcare. Inf. Dev.; 2023; 02666669231172434. [DOI: https://dx.doi.org/10.1177/02666669231172434]

66. Kuberkar, S.; Singhal, T.K. Factors influencing adoption intention of AI powered chatbot for public transport services within a smart city. Int. J. Emerg. Technol. Learn.; 2020; 11, pp. 948-958.

67. Chenavaz, R.Y.; Jasimuddin, S.M. An analytical model of the relationship between product quality and advertising. Eur. J. Oper. Res.; 2017; 263, pp. 295-307. [DOI: https://dx.doi.org/10.1016/j.ejor.2017.05.016]

68. Rhim, J.; Kwak, M.; Gong, Y.; Gweon, G. Application of humanization to survey chatbots: Change in chatbot perception, interaction experience, and survey data quality. Comput. Hum. Behav.; 2022; 126, 107034. [DOI: https://dx.doi.org/10.1016/j.chb.2021.107034]

69. Io, H.N.; Lee, C.B. Understanding the Adoption of Chatbot: A Case Study of Siri. Advances in Information and Communication Networks: Proceedings of the 2018 Future of Information and Communication Conference (FICC); Springer International Publishing: Cham, Switzerland, 2019; Volume 1.

70. Lee, Y.-C.; Yamashita, N.; Huang, Y. Exploring the effects of incorporating human experts to deliver journaling guidance through a chatbot. Proc. ACM Hum.-Comput. Interact.; 2021; 5, pp. 1-27. [DOI: https://dx.doi.org/10.1145/3449196]

71. Nelekar, S.; Abdulrahman, A.; Gupta, M.; Richards, D. Effectiveness of embodied conversational agents for managing academic stress at an Indian University (ARU) during COVID-19. Br. J. Educ. Technol.; 2022; 53, pp. 491-511. [DOI: https://dx.doi.org/10.1111/bjet.13174]