Improved DEMATEL-ISM integration approach for complex systems

Abstract

Purpose

Decision Making Trial and Evaluation Laboratory (DEMATEL) and Interpretive Structural Modeling (ISM) are commonly used separately, but also may be combined per their common characteristics to identify causal relationships and hierarchical structure among factors in complex systems with a relatively small computational burden. The purpose of this study is to establish an improved DEMATEL-ISM integration approach to remedy the disadvantages of the traditional DEMATEL-ISM integration method. A case study was conducted to compare the proposed improved integration approach against the traditional integration method, and to validate its feasibility and effectiveness.

Methods

The proposed improved DEMATEL-ISM integration approach has two main parts: a threshold determination via maximum mean de-entropy (MMDE) method and an additional transitivity check process. The factors influencing China’s rural-urban floating population’s willingness to participate in social insurance was analyzed as a case study.

Results

The traditional and improved methods show notable differences in the hierarchical factor structure and the inner influence relationship among factors that they respectively reveal. The traditional integration approach results in some irrationality, while the improved approach does not.

Originality

This study confirms the importance of proper threshold determination and reachability matrix transitivity checking during DEMATEL-ISM integration. The improved approach includes a scientific threshold determination method based on the MMDE method, plus a transitivity check of the reachability matrix with necessary corrections to ensure its soundness. It can be straightforwardly operated at a relatively low computational burden while providing accurate analysis results.

About the Authors:

Jih-Kuang Chen

Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Writing – review & editing

* E-mail: [email protected]

Affiliation: Economics and Management College, Zhaoqing University, Zhaoqing, China

ORCID logo https://orcid.org/0000-0003-2357-5109

Introduction

Modern organizations are faced with increasingly intense complexity and uncertainty in today’s competitive market environment. Organizational decision-makers and managers must quickly and correctly grasp a complicated array of contradictions to maintain economic competitiveness. Decision Making Trial and Evaluation Laboratory (DEMATEL) is an analytical method that was proposed to facilitate decision-making across complex factors [1]. DEMATEL serves to first simplify the multi-factor system structure and then screen out the main influence factors. Based on graph theory, DEMATEL supports the development of knowledge and experience; it allows for analysis of the logical correlations and direct-influence relationships between various factors in complex systems, thus revealing key factors. Due to the universality and simplicity of its mechanism, DEMATEL has garnered a great deal of scholarly interest in recent years across engineering, management science, supply chain management, and other fields [2–4]. Interpretive Structural Modeling (ISM) was established to decompose a complex system into several smaller systems, then build a structured model of the complex system [5]. ISM has multi-level hierarchy and can express ambiguous systems with intuitive structural relations, facilitating the objective analysis of complex problems. It has also been extensively investigated by researchers in multiple fields [6–8].

Many researchers have used DEMATEL and ISM separately to identify main influence factors and to construct multi-level hierarchical structured models of them, thus obtaining complete information [9–11]. However, separate calculation processes are overly complex and have an excessive calculation burden. DEMATEL and ISM methods both express expert evaluation information via relation matrix and reflect the influence relationships between factors. Zhao, Zhang, and Li (2006) [12] proposed a DEMATEL-ISM integration approach. They argue that as the total-relation matrix of DEMATEL contains more information than the reachability matrix of ISM, and the reachability matrix is more difficult to calculate, that the hierarchical structure of the complex system can be divided by the total-relation matrix of DEMATEL and then transformed into an ISM reachability matrix. However, this approach (from here on referred to as the “traditional integration” method or approach) has notable drawbacks and may lead to deviation in the ISM analysis results and drive down the effectiveness of any countermeasures implemented. (Details regarding its drawbacks and solutions are given in Section 3.) This traditional DEMATEL-ISM integration method has been adopted in many previous studies (Section 2). The purpose of the present study is to establish an improved DEMATEL-ISM integration approach to remedy the disadvantages of the traditional integration method.

The study of the factors influencing China’s rural-urban floating population’s willingness to participate in social insurance was analyzed by the improved integration approach versus the traditional integration method to validate its feasibility and effectiveness. The factors influencing social insurance willingness of the rural-urban floating population in the Guangdong-Hong Kong-Macao Greater Bay Area was taken as an example to test the improved integration approach by comparison against the traditional integration method. Differences emerged between the two approaches, including the hierarchical structure and inner influence relationships they respectively reveal. The results of this work may serve as a helpful reference for further research on the case study topic.

Literature review

Step-wise traditional DEMATEL-ISM integration method

To operate the traditional DEMATEL-ISM integration method, DEMATEL is first used to determine the direct relation level between two factors through expert evaluation (typically on a 0–4 scale) to form a direct-relation matrix X:(1)

After taking the maximum value of row sum as the normalized cardinality, the transformation matrix X becomes the normalized direct-relation matrix N:(2)

The total-relation matrix T is then calculated as follows:(3)where I is the identity matrix and -1 is the inverse matrix.

The influence level and the relationship level of each factor can be added to determine “prominence”, while the influence level and the relationship level of the system factor can be subtracted to determine “relation”. The prominence and relation of the factors form a two-dimensional coordinate system. The positions of each factor in the coordinate system are marked, the importance of each factor is analyzed, and suggestions are put forward for the system.

In the traditional integration approach, the total-relation matrix T does not account for the influence of a factor on itself, so the identity matrix I is added to form the matrix H (H = T+I). The value range of elements in the ISM’s reachability matrix is {0,1}, so experts or decision-makers can scientifically and reasonably set the threshold λ to suit the problem at hand. The matrix H can be directly converted into the reachability matrix K after comparison based on the following equation:(4)

The hierarchical structure of system factors can then be established based on the reachability set and antecedent set provided by the ISM reachability matrix. The steps of this process are as follows.

1. Determine the reachability set and antecedent set of each factor

The reachability set, a set of factors corresponding to all rows with 1 element in the i-th row of the reachability matrix, is represented by R(i). The antecedent set, a set of factors corresponding to all columns with 1 element in the i-th column of the reachability matrix, is represented by Q(i).

2. Identify the intersection set C(i)

C(i) is the intersection set of the reachability set R(i) and antecedent set Q(i). When C(i) = R(i) is satisfied, the corresponding factor is the first-level factor and i-th row and i-th column can be delimited from the reachability matrix K. This step is repeated until all factors are crossed out. A multi-level hierarchical structure of factors can then be established according to the order in which the factors are crossed out. This technique saves a great deal of complicated matrix calculations and is not particularly prone to user error.

Previous applications of traditional DEMATEL-ISM integration method

The traditional DEMATEL-ISM integration method has been adopted by many previous researchers. Shen, Sun, Zhang et al. (2014) [13], for example, used it to build a logical relationship of many failure subsystems; failure mode effects and criticality analysis (FMECA) was then applied qualitative analysis of the direction of reliability improvement. Hou and Xiao (2015) [14] used it to identify the critical success factors (CSFs) that impact the linkage mechanism between government and non-profit organizations tasked with geo-disaster emergency decisions. Zhang and Luo (2017) [15] used it to analyze the influencing factors of runway incursions and their mechanisms, offering a novel decision-making idea for runway incursion control. Shen, Li, Shi et al. (2018) [16] used it to construct a multi-level hierarchical structure of influencing factors on the economics of a distributed natural gas combined cooling, heating, and power (DNG-CCHP) system.

There have been many other valuable contributions to the literature. Wang, Cao, and Zhou (2018) [17] used traditional DEMATEL-ISM integration to establish a hierarchical model of influencing factors and mechanisms related to mining safety. Xie and Liu (2019) [18] used it to establish a hierarchical structure of influencing factors and to distinguish cause and effect factors. Li, Wang, Dubljevic et al. (2019) [19] used it to establish a hierarchical network model, which they mapped to a Bayesian network (BN) to further transform the conditional probability distribution; they quantified the strength of coupling relationships in an accident-causing system accordingly to determine main paths resulting in system failure. Sun, Zhou, Zhang et al. (2019) [20] used it to integrate the interaction direction and intensity between failure factors and hierarchical factor structures, then used the analytic network process (ANP) to determine the relative importance of CNC equipment fault factors.

Shakeria and Khalilzadeh (2020) [21] used this traditional method to identify and determine sequences and relationship factors affecting project communications and their clusters to inform organizational and project managers in the context of project communications. Cui, Zhu, and Wand (2020) [22] used it to explore the influencing factors of collaborative innovation in low-carbon technology in China in the context of Paris Agreement goals. Yue and Han (2020) [23] used it to determine the causative attributes of Chinese airline company safety by creating an indicator system for safety risk factors. Liu, Dou, Yang (2021) [24] used it to analyze the influencing factors of cross-border e-commerce supply chain resilience (CBSCR), so as to further enhance the competitiveness of a global supply chain.

Drawbacks and potential solutions of traditional integration method

As mentioned above, it merits consideration whether the total-relation matrix T takes into account the influence of a given factor on itself. The total-relation matrix T is derived from , which includes all direct and indirect relations. The equation T = N(I−N)−1 already includes matrix I. I lends the suspicion of repeated calculation. However, the diagonal values of the total relation matrix T may not be all 1, because the principles of the two algorithms inherently differ.

After the adjacency matrix (element is 0 or 1) of ISM is generated, the sum of the adjacency matrix and the identity matrix I performs the power operation of matrix A+ I on a certain integer n until the following equation holds: . K is the reachability matrix; this power operation is based on Boolean algebra. The elements in the ISM reachability matrix of ISM also fall under the law of transitivity, so the reachability level of each node after passing through paths has a length not greater than (n-1). For graphs with n nodes, the length of the longest path does not exceed (n-1). The total-relation matrix T of DEMATEL does not have this property, but reflects the mutual influence and various levels among different factors and contains sufficient information for a sound analysis. All diagonal elements can be directly changed to 1 in the case discussed here. (The law of transitivity characteristics is discussed further below.)

The lack of scientific standard for setting the threshold value is also problematic. Various threshold-setting techniques have been formulated by experts or decision-makers according to actual problems, but there is no unified scientific standard. Directly setting the threshold to non-0 or 1 actually negates the advantage of DEMATEL’s inclusion of the level of influence between factors. Other threshold determination techniques include the mean method [25], median method [26], μ +σ [27], and μ + 1.5σ [28]. In the present study, the threshold was set via maximum mean de-entropy (MMDE) [29]. Originally a physics concept, entropy describes the state function of system chaos. Informatics researchers later used entropy for information measurement in probability and statistical models.

MMDE and transitivity check and conversion

To operate MMDE, let a set of random variables X = {x1,x2,…,xn} while the relative probability of variables is P = (p1, p2,…,pn}; the information entropy is H(p1,p2,…pn) = -∑piln pi, where ∑pi = 1. If Pi = 0, then piln pi = 0. The de-entropy (HD) can be defined as , which reveals the amount of useful information and reduces informational uncertainty in models such as the DEMATEL-ISM.

The MMDE is implemented in the following steps.

1. Transform the total-relation matrix T into a set of three elements, then arrange the influence values from large to small to form a set T*. The collection consists of {influence level value, row where element is located, column where element is located}.

2. Separately, form the rows (columns) in which all factors in the T* set are located into the Td(Tr) set.

3. Calculate the de-entropy (MDE) values of Td (Tr) to the i-th element respectively, , where is the number of categories of factors in set is the number of categories of factors in set ).

4. List all sets composed by and , find the maximum value, then list the rows (columns) larger than and ; delete the repeated rows (columns), leaving only different kinds of elements to form the set.

5. Determine : the set established in Step 4 is a three-element set with the largest influence value in the corresponding T* set.

6. Calculate Tth: Take = Tth, then the minimum influence value in Tth set can be used as the threshold value.

It is also important to consider whether the reachability matrix K, which is compared simply with the threshold from matrix H+I, can be directly divided into layers. Some researchers have regarded matrix H as an adjacency matrix rather than a reachability matrix after comparing it with the threshold value. Zhou et al. [30], for example, changed matrix T into an ISM adjacency matrix after comparison with the threshold value and adding I, then obtained a reachability matrix after Boolean algebraic operation. Yu and Shao [31] compared the matrix T with the threshold value (mean value of T), added I to form the ISM adjacency matrix, and then obtained the reachability matrix by Boolean algebra. Weng et al. [32] obtained the reachability matrix after Boolean algebra by first adding I to matrix T, then comparing the threshold value with 0.5 to form the ISM adjacency matrix. It is yet unclear whether it is more appropriate to add I before or after the addition in setting a threshold–re-referencing the adjacency matrix in these cases drives down the overall efficiency of the integration and creates a burdensome calculation process.

Other researchers have directly converted the reachability matrix and then divided it into layers. However, under the law of transitivity of the reachability matrix, the corresponding rows or columns of nodes Si and Sj in the matrix have exactly the same corresponding elements; Si and Sj are on the same loop. The conversion directly from DEMATEL does not necessarily have this property, so an additional transitivity check conversion process is necessary.

To complete the transitivity check, first assume that there are five factors in a certain system. The direct influence relationship between them is shown in the left half of Fig 1 and the direct influence relationship is shown in the right half of Fig 1.

[Figure omitted. See PDF.]

Fig 1. Direct influence relationship and influence relationship matrix of system factors.

https://doi.org/10.1371/journal.pone.0254694.g001

If the matrix is converted from the DEMATEL calculation and compared with the threshold value in Fig 1, it may not be possible to treat it as a reachability matrix. The reachability matrix must include indirect influence and possess law of transitivity characteristics, so if A1 not only directly affects A2 but also affects A3 (because A2 affects A3), then A1 has indirect influence on A3. By analogy, the transitivity of all factors can be checked at this point. If indirect influence is identified, the missing indirect influence relationship between factors should be supplemented via 0 and converted to 1 as marked in bold-italics in the right half of Fig 2.

[Figure omitted. See PDF.]

Fig 2. Influence relation and reachability matrix after transitivity check and conversion.

https://doi.org/10.1371/journal.pone.0254694.g002

Proposed method

The proposed, improved DEAMTEL-ISM integration approach is operated in the following steps.

1. Use DEMATEL to obtain the total-relation matrix T.

2. Calculate the threshold value by MMDE method and transform it into an initial reachability matrix after comparison.

3. Modify/hold at 1 all diagonal elements from the top left to the bottom right of the reachability matrix, then build the final reachability matrix via transitivity check.

4. Finally, divide the reachability matrix into ISM layers.

Analysis

Previous studies on influence factor selection generally center on health insurance [33–35] and endowment insurance [36]. Chinese social insurance includes five categories: employment injury insurance, endowment insurance, unemployment insurance, maternity insurance, and medical insurance [37]. Issues such as regulations of the township-city floating population on the social security of a given city, and the social environment of places of residence, have significant effects on the social insurance system. Previous research on social integration [38–40] were also referenced for the purposes of this study. A total of 15 influence factors were consolidated as shown in Table 1.

[Figure omitted. See PDF.]

Table 1. 15 influence factors.

https://doi.org/10.1371/journal.pone.0254694.t001

Analysis by improved integration approach

The Guangdong-Hong Kong-Macao Greater Bay Area served as the research area for this study. Seven experts familiar with this field were interviewed. Their interviews were synthesized to establish a direct-relation matrix X, then run through the steps of DEMATEL to secure the total-relation matrix shown in Table 2.

[Figure omitted. See PDF.]

Table 2. Direct-relation matrix and total-relation matrix.

https://doi.org/10.1371/journal.pone.0254694.t002

The threshold value was then calculated by MMDE according to the total-relation matrix. The T* set was sorted first. After the calculation steps described above, the maximum MDED and MDER were found to be 0.0322 and 0.0446, respectively. The row and column element sets of the largest MDED and MDER were combined to determine the minimum influence as 0.063, so the threshold was set to 0.063. The calculation results of each stage are listed in Table 3.

[Figure omitted. See PDF.]

Table 3. Threshold results by MMDE.

https://doi.org/10.1371/journal.pone.0254694.t003

The elements with a total-relation matrix less than the threshold value were changed to 0, those with a total-relation matrix greater than or equal to the threshold value were changed to 1, and the transitivity check was performed after adding the 15-order identity matrix. Many elements showed indirect influence in this case and thus were changed to 1 (bolded in Table 4) to obtain the final reachability matrix.

[Figure omitted. See PDF.]

Table 4. Final reachability matrix.

https://doi.org/10.1371/journal.pone.0254694.t004

An ISM hierarchy analysis was performed according to the final reachability matrix; the results are shown in Table 5.

[Figure omitted. See PDF.]

Table 5. Hierarchical analysis of proposed approach.

https://doi.org/10.1371/journal.pone.0254694.t005

Analysis by traditional integration approach

After adding the 15-order identity matrix to the afore-mentioned total-relation matrix, the threshold value was set to 0.092 according to the consensus reached by the researchers and experts. Elements with a total-relation matrix less than the threshold value were changed to 0 and elements with values greater than or equal to the threshold were changed to 1. The resulting reachability matrix is shown in Table 6.

[Figure omitted. See PDF.]

Table 6. Reachability matrix.

https://doi.org/10.1371/journal.pone.0254694.t006

A factor hierarchy analysis was then performed via ISM in each step of the calculation process. The results are shown in Table 7.

[Figure omitted. See PDF.]

Table 7. Hierarchical analysis of traditional approach.

https://doi.org/10.1371/journal.pone.0254694.t007

Comparison and results

The influence factors can be divided into seven layers based on the proposed integration approach: the first layer is S1 and S6, the second layer is S4, S5, and S15, the third layer is S8, S11, and S14, the fourth layer is S3 and S12, the fifth layer is S9, S10, and S13, the sixth layer is S2, and the seventh layer is S1. The results obtained by traditional integration approach are S1, S6, S7, and S12 as the first layer, S5, S8, S11, and S14 as the second, S3 and S15 as the third, S9, S10, and S13 as the fourth, and S1 and S2 as the fifth. These are substantial differences on the whole. S5, S6, and S7 are similar while S3, S4, S8, S9, S10, S11, S13, S14, and S15 differ only slightly, but the respective approaches produce S1 and S2 as factors of a different order. The difference between S12 is especially large–the factor of the S12 unit system, logically speaking, should not be a first-order factor. This indicates that the traditional integration approach may not be accurate. The hierarchical structure analysis results of the two approaches are shown in Table 8.

[Figure omitted. See PDF.]

Table 8. Hierarchical structure analysis results of two approaches.

https://doi.org/10.1371/journal.pone.0254694.t008

An inner interaction diagram of the influence factors makes the differences between the two approaches even more stark. Fig 3(A) shows the factor layer and inner influence relationship diagram obtained by the proposed approach, where S4 directly affects S7. Fig 3(B) shows the factor layer and inner influence relationship diagram obtained by the traditional integration approach, where S1 directly affects S8 and S4 does not affect S7; S2 directly affects S7 and S8 directly affects S7. The proposed approach shows an indirect influence relationship among these three factors. The above differences are marked in the figure by thick lines. It appears that the traditional integration approach does not necessarily provide more accurate results in terms of the hierarchical factor structure and the inner relationships among influence factors.

[Figure omitted. See PDF.]

Fig 3. Factor layers and inner influence relationships of two approaches.

https://doi.org/10.1371/journal.pone.0254694.g003

Conclusion

Integrating DEMATEL and ISM is an interesting analysis technique, as it may lend simplicity while minimizing the computational burden of the two respective methods. However, the traditional approach to integration has significant drawbacks–mainly in terms of the threshold determination and transitivity checking process. The system structure obtained by this traditional integration method may not completely conform to the strict definition of “hierarchical”, so it is not necessarily possible to accurately determine the hierarchical structure of factors and the inner relationships among them. This can lead to deviation in the ISM analysis results and drive down the effectiveness of any countermeasures implemented.

In this study, an improved DEMATEL-ISM integration approach was established which includes an MMDE approach to determine the threshold, and composition of a transitivity check correction reachability matrix. The influence factors of the social insurance willingness of the rural-urban floating population in the Guangdong-Hong Kong-Macao Greater Bay Area was taken as an example to test the proposed improved integration approach by comparison against the traditional integration method. Differences emerged between the two approaches, including the hierarchical structure and inner influence relationships, which were not extreme but do still merit careful consideration. This comparative analysis also confirmed the feasibility and effectiveness of the improved integration approach. The results of this work may serve as a helpful reference for further research on this topic.

Supporting information

S1 Dataset.

https://doi.org/10.1371/journal.pone.0254694.s001

(XLSX)

Citation: Chen J-K (2021) Improved DEMATEL-ISM integration approach for complex systems. PLoS ONE 16(7): e0254694. https://doi.org/10.1371/journal.pone.0254694

References

1. Gabus A, Fontela E (1972) World problems, an invitation to further thought within the framework of DEMATEL. Geneva, Switzerland: Battelle Geneva Research Center.

2. Shao J, Taisch M, Ortega-Mier M (2016) A grey-decision-making trial and evaluation laboratory (DEMATEL) analysis on the barriers between environmentally friendly products and consumers: Practitioners’ viewpoints on the European automobile industry. Journal of Cleaner Production 112: 3185–3194.

3. Seleem SN, Attia EA, El-Assal A (2016) Managing performance improvement initiatives using DEMATEL method with application case study. Production Planning & Control 27: 637–649.

4. Chen JK (2018) The Influence of Behavioral Intention on Third-Party E-Commerce Payment. South African Journal of Economic and Management Sciences 21(1): 1–9.

5. Duperrin JC, Godet M (1973) Méthode de hiérarchisation des éléments d’un système, Rapport économique du CEA, R-45-41, Paris.

6. Hussaina M, Awasthib A, Tiwari MK (2016) Interpretive structural modeling-analytic network process integrated framework for evaluating sustainable supply chain management alternatives. Applied Mathematical Modelling 40(5/6): 3671–3687.

7. Agi MAN, Nishant R (2017) Understanding influential factors on implementing green supply chain management practices: An interpretive structural modelling analysis. Journal of Environmental Management 188(1): 351–363. pmid:28006744

8. Gupta A, Suri PK, Singh RK (2019) Analyzing the Interaction of Barriers in E-Governance Implementation for Effective Service Quality: Interpretive Structural Modeling Approach. Business Perspectives and Research.

9. Chauhan A, Singh A, Jharkharia S (2018) An ISM and DEMATEL method approach for the Analysis of Barriers of Waste Recycling in India. Journal of the Air & Waste Management Association 68(2): 100–110. pmid:28278038

10. Kumar A, Dixit G (2018) An analysis of barriers affecting the implementation of e-waste management practices in India: A novel ISM-DEMATEL approach. Sustainable Production and Consumption 14: 36–52.

11. Mousavizade F, Shakibazad M (2019) Identifying and ranking CSFs for KM implementation in urban water and sewage companies using ISM-DEMATEL technique. Journal of Knowledge Management 23(1): 200–218.

12. Zhao DQ, Zhang L, Li HW (2006) A Study of the Sysrem’s Hierarchical Strucyure through Integration of DEMATEL and ISM. Proceedings of the Fifth International Conference on Machine Learning and Cybernetics 449–1453.

13. Shen GX, Sun SG, Zhang YZ, Wang ZQ, Chen BK, MA C (2014) System failure analysis based on DEMATEL–ISM and FMECA. Journal of Central South University 21: 4518–4525.

14. Hou JD, Xiao RB (2015) Identifying critical success factors of linkage mechanism between government and non-profit in the geo-disaster emergency decision. International Journal of Emergency Management 11(2): 146–168.

15. Zhang PK, Luo F (2017) Influencing Factors of Runway Incursion Risk and Their Interaction Mechanism Based on DEMATEL–ISM. Tehnički vjesnik 4(6): 1853–1861.

16. Shen JF, Li FY, Shi D, Li HZ, Yu XH (2018) Factors Affecting the Economics of Distributed Natural Gas-Combined Cooling, Heating and Power Systems in China: A Systematic Analysis Based on the Integrated Decision Making Trial and Evaluation Laboratory-Interpretative Structural Modeling (DEMATEL-ISM) Technique. Energies 11, 2318.

17. Wang LL, Cao QG, Zhou LJ (2018) Research on the influencing factors in coal mine production safety based on the combination of DEMATEL and ISM. Safety Science 103: 51–61.

18. Xie KF, Liu ZM (2019) Factors Influencing Escalator-Related Incidents in China: A Systematic Analysis Using ISM-DEMATEL Method. International Journal of Environmental Research and Public Health 16, 2478. pmid:31336772

19. Li F, Wang WH, Dubljevic S, Khan F, Xu J, Yi J (2019) Analysis on accident-causing factors of urban buried gas pipeline network by combining DEMATEL, ISM and BN methods. Journal of Loss Prevention in the Process Industries 61: 49–57.

20. Sun SG, Zhou WJ, Zhang M, Liu XY, Mou XW (2020) Fault analysis of CNC equipment based on DEMATEL/ISM/ANP. Journal of Mechanical Science and Technology 34(8): 1–7.

21. Shakeria H, Khalilzadeh M (2020) Analysis of factors affecting project communications with a hybrid DEMATEL-ISM approach (A case study in Iran). Heliyon 6(8), e04430. pmid:32793821

22. Cui HR, Zhu XF, Wand HA (2020) Collaborative Innovation of Low-Carbon Technology from the Triple Helix Perspective: Exploring Critical Success Factors Based on DEMATEL-ISM. Polish Journal of Environmental. Study 29(2): 1579–1592.

23. Yue RT, Han YX (2020) On the DEMATEL-ISM model for analyzing the safety risk-involving factors of the airline companies. Journal of Safety and Environment 20(6): 2091–2097.

24. Liu XL, Dou ZW, Yand W (2021) Research on Influencing Factors of Cross Border E-commerce Supply Chain Resilience Based on Integrated Fuzzy DEMATEL-ISM. IEEE Access 9: 36140–36153.

25. Cebi S (2013) Determining importance degrees of website design parameters based on interactions and types of websites. Decision Support Systems 54(2): 1030–1043.

26. Shaiee M, Lotfi FH, Saleh H (2014) Supply chain performance evaluation with data envelopment analysis and balanced scored approach. Applied Mathematical Modeling 38(21/22): 5092–5112.

27. Xue W, Geng ZW, Wang HB, Zhang HC, Guo DY (2019) A Study on Influencing Factors of Fire Protection Risk in Integrated DEMATEL/ISM Timber Industry Park. Journal of Nankai University (Natural Science Edition) 52(6): 99–104.

28. Rajesh R, Ravi V (2015) Modeling enablers of supply chain risk mitigation in electronic supply chains: A Grey-DEMATEL approach. Computers & Industral Engineering 87: 126–139.

29. Lee PTW, Lin CW (2013) The cognition map of fnancial ratios of shipping companies using DEMATEL and MMDE. Maritime Policy & Management 40(2): 133–145.

30. Zhou W, Zhao TS, Xu SM, Liu W (2017) Modeling of influencing factors of safety behavior of construction workers based on DEMATEL and ISM. Journal of Civil Engineering and Management 34(6): 126–132.

31. Yu Q, Shao D (2018) Construction of Multi-level Hierarchical Driving Model of Miners’ Situational Consciousness. China Science and Technology for Safety Production 14(4): 164–169.

32. Weng WJ, Deng QZ, Du L (2015) A System Analysis of Factors Affecting the Scale of Local Government Administrative Establishment from the Perspective of Complex System—Based on DEMATEL-ISM Integration Method. Journal od Practice and Understanding of Mathematics 45(23): 110–119.

33. Lofgren C, Thanh NX, Chuc NTK, Emmelin A, Lindholm L (2008) People’s willingness to pay for health insurance in rural Vietnam. Cost Effectiveness and Resource Allocation 6(16): 1–16. pmid:18691440

34. Al-Hanawi MK, Vaidya K, Alsharqi O, Onwujekwe O (2018) Investigating the Willingness to Pay for a Contributory National Health Insurance Scheme in Saudi Arabia: A Cross-sectional Stated Preference Approach. Applied Health Economics and Health Policy 16: 259–271. pmid:29307076

35. Ogundeji YK, Akomolafe B, Ohiri K, Butawa NN (2019) Factors influencing willingness and ability to pay for social health insurance in Nigeria. PLoS ONE 14(8), e0220558. pmid:31374083

36. Wang KX, Xiao HA, Zhang WX (2011) Analysis on Influencing Factors of Farmers Involving in Rural Social Endowment Insurance—Take Huiping Town, Qidong City, Jiangsu Province as Example. Journal of Agricultural Science 3(1): 250–257.

37. Zong XL, Guan X, Gao Y, Chen ZL, Zhang GX (2017) Analysis of the Influencing Factors of Migrant Worker Social Insurance in Lanzhou. EURASIA Journal of Mathematics, Science and Technology Education 13(12): 7949–7960.

38. Louw JS, Kirkpatrick B, Leader G (2019) Enhancing social inclusion of young adults with intellectual disabilities: A systematic review of original empirical studies. Journal of Applied Research in Intellectual Disabilities 33(5): 1–15. pmid:31652027

39. Hsieh MO, Leung P (2019) Social exclusion experienced by older adults: factors for designing healthcare settings in Taipei, Taiwan. Social Work in Health Care 58(4): 368–381. pmid:30739597

40. Tikhomirova O (2020) E-Governance and Social Inclusion of Entrepreneurship and Businesses: Toward the Social Inclusive Digital Society. International Journal of E-Entrepreneurship and Innovation 10(2): 1–25.