1. Introduction
1.1. Study Background and Purpose
Developer experience (DX) refers to the overall experience of developers while they develop systems, products, or services. The concept also includes the developer’s feelings, motivations, characteristics, and activities [1]. DX is a special case of user experience (UX), which has been studied extensively. DX shares both the idea and philosophy of the UX design. However, the main actor of DX is dualistic—the developer is a tool user who enables the system use and a producer who develops the tool [2]. DX may have a similar meaning to UX, but the former is limited only to developers responsible for designing or developing the system for the end-user. The developer should have interest in and passion for the values of the application created for the user and should motivate interest in its development.
In contrast with the past, current developers are considered key participants or stakeholders in various business environments. Furthermore, because they are decision-makers, much attention has been paid to the developer’s experience [3]. For example, the success of the Apple App Store and Google Play—which transformed the mobile market ecosystem—was due to the vast development ecosystem or DX in which developers can actively develop and register apps with interest.
1.2. Research Goal
This study aims to derive sub-factors and assessment questions to evaluate DX perceived by developers who use a deep-learning (DL) platform. DL refers to a set of machine-learning (ML) algorithms that attempt high-level abstraction by combining various non-linear transformation techniques [4]. It also refers to the framework that assists in developing a DL model faster. DL platforms, which can implement DL technology, are expected to receive significant attention from developers considering the advancement potential of related industries with DL-based technologies. It is essential to use a DL-based platform to develop voice recognition, pattern recognition, and computer vision systems based on DL technology, which is considered the core technology of the Fourth Industrial Revolution.
For example, major information technology companies including Netflix, Uber, and Airbnb, use DL platforms to analyze big data collected from consumers [5]. Developers must implement DX at a high level to provide UX at a high level for software users [6]. If DX can be evaluated using a systematic methodology, it can help improve development tools and environments and provide users with excellent UX [7]. This study extracted the sub-factors and assessment questions that can monitor DX as a benchmark tool to evaluate the performance of the DL platform. It also evaluated the nomological validity concerning whether the benchmark tool can explain DX (the single concept) at the level of statistical significance. Figure 1 illustrates the research flow for the present work.
2. Literature Review
2.1. Developer Experience (DX)
DX refers to the experience involving the interaction between development tools and developers in the software development process [8]. A complete understanding of DX can facilitate an understanding of the expectation, perceptions, and feelings of developers who participate in development tools. Furthermore, DX has a dualistic nature of UX-based DX, in which the developer is both a system tool user and a system producer who predicts the UX [2]. Understanding the relationship between the developer and the platform that a developer uses is essential because we can thereby predict whether the platform can satisfy developers and ensure usability and functionality.
Fontão et al. (2017) [9] claimed that factors that affected DX could be identified from three types of information. They included the factor that affected the developer’s cognition toward the software development infrastructure, the developer’s perceptions or emotions of the contribution to the ecosystem, and the value recognized by the developer’s contribution. They claimed that if the effect on DX was analyzed from these three viewpoints, it could help to maintain organizational strategy and raise the quality of the goal pursued by the organization and the framework that supported developers [9].
Parviainen et al. (2015) [10] argued that because software developments were conducted through collaboration with team members, the DX perceived by team members would affect the integrated development environment. They classified the factors that consisted of DX into development rules and policies, dynamic work division according to circumstantial roles, and collaboration mode between communities. For example, they suggested that DX may change depending on whether team members who developed software agreed on the coding style. Furthermore, they discussed whether assigning a clear role for software development (e.g., manager, user interface developer, and backend developer) affected the perceived experience.
2.2. Deep-Learning (DL) Platform
The demand for ML-based artificial intelligence (AI) has increased. ML is an algorithm that self-learns from data and can execute work without relying on a programmed code [11]. The ML technique has been established as an essential tool to process complex data and predict patterns in all sectors, including medicine, finance, environment, military, and science [12,13] DL refers to an ML technique that performs tasks including image classification, voice recognition, and natural language processing [14,15].
A DL platform is a development environment provided to developers to build a DL model. Because potential business success can be obtained using DL techniques in various areas where logic and inference, prediction, or cognition function processing are required, investment in and development of the DL platform have increased, predominantly in global companies and academia [16]. The use of DL technology has increased in various sectors. Thus, case studies on developing and improving DL platforms toward developing and supporting new algorithms have also increased [17].
For example, studies on DL platform development combined with parallel processing technology, cloud computing power, and distributed storage technology have been conducted in recent years [18,19]. Nonetheless, no studies have been conducted on the experience of developers who use DL platforms concerning platform processes or tools or an evaluation of those experiences. A DL platform should provide the perception to developers that it can create unique value and user-friendly platform functions. Developers should fully understand the value of the DL platform and perceive that the deliverables provided by the platform are critical to value creation and the tool’s value in improving productivity.
3. Research Methods
3.1. Sub-Constructs of DX
Fagerholm and Münch (2012) [1] proposed a conceptual framework to evaluate DX (Figure 2). They showed that because software development requires creative characteristics, developers specifically consider infrastructure and perspectives on work, feeling, and value created when achieving goals. The DX framework proposed by the researchers consisted of cognition, affection, and intention or conation aspects of experience. Cognition includes attention, memory, and problem-solving ability. Affection includes the developer’s feelings or emotions, such as positive emotion or pleasure. Finally, intention, or conation, includes impulse, motivation, and desire required for software development. Inspired by the work of Fagerholm and Münch et al. (2012), the present study explores the research question of “three sub-constructs of cognitive, affective, and conative factor consist of developer experience toward the DL platform”.
3.2. Evaluation Tool
3.2.1. Development of Preliminary Survey Questionnaire
Based on the studies by Ahn and Back (2018) [20], Back and Parks (2003) [21], Chowdhury and Salam (2015) [22], Khanal (2018) [23], Venkatesh, Speier, and Morris (2002) [24], and Venkatesh and Davis (2000) [25], this study constructed a preliminary questionnaire to evaluate the sub-constructs of DX. The operational definition of “cognition” in this study refers to the rational basis providing competitiveness, efficacy, value, or motivation for developers to use the DL platform. “Affection” is an emotional state that developers expect from the use of the DL platform. Finally, “conation” refers to the developer’s will or desire to use the DL platform autonomously under volitional control.
3.2.2. Delphi Survey
This study conducted a Delphi survey with a panel of experts to evaluate the content validity of the preliminary survey questionnaire [26]. The Delphi survey was conducted between 4 January and 23 January 2021. Lynn (1986) [27] suggested that 3 to 10 expert panelists are sufficient for evaluating content validity. In this study, four experts who have been in academia and industries related to DL technology and development for at least 10 years participated in the panel. Of the four experts participated in the Delphi, two respondents were classified as having a master’s degree, and the remaining two as having a doctoral degree. In addition, they consist of one four-year university faculty, one principal researcher in a research institute, and two project managers with 15 years of industry and practice experience.
The Delphi evaluation tool was constructed based on closed-ended questions using a 7-point Likert scale (e.g., one point = “strongly disagree”, and seven points = “strongly agree”) and open-ended questions. The closed-ended questions were created to evaluate whether the preliminary survey questionnaire could assess the sub-constructs that corresponded to each question. Furthermore, free opinions about the suitability, understanding, and consistency of the terms used in the questions were collected through open-ended questions in addition to the question’s appropriateness. In the present work, the one-round Delphi study was conducted by email.
The content validity ratio (CVR) of the response values to the closed-end questions was calculated to verify the degree of agreement of the expert opinion obtained from the Delphi survey. According to Lawshe (1975) [28], the minimum acceptable CVR was 0.99 if the number of panelists was five. Thus, this study selected only the preliminary survey questions whose CVR was 1.0 to develop the questionnaire for the main survey. The excluded questions were not used in the main survey and final analysis. As illustrated in Table 1, for instance, the calculated CVRs for the measurement items of 1, 4, 6, 7, 11 through 14, 20, and 21 are less than 1.0, and these ten items were discarded for the final analysis. With respect to the ten pilot items for affective factors, calculated CVRs for the items of 1 through 4, 9, and 10 are less than 1.0 and again, these six measurement items were eliminated and not included in the final analysis (Table 2). Among the calculated CVRs for the measurement items of behavioral factors, the values for the items of 4 and 6 have CVRs less than 1.0 (Table 3). Thus, a total of nine measurement items for behavioral factors are considered for the final analysis.
4. Study Results
4.1. Study Subjects
For the main survey in this study, a questionnaire was distributed to 260 employees in the industry, including S and L companies in South Korea. Their job description involves DL technology planning, design, and development. In detail, S and L companies are characterized by information technology organization that primarily provides software solutions and service integration service. Their business is highly driven by the use of an AI-based natural language and learning model, smart factory, blockchain technology, and cloud data analysis. The survey was conducted based on an online survey tool from 8 February to 26 March 2021. Insincere responses were excluded, and survey data from 225 employees were used in the final analysis.
Table 4 presents the demographic characteristics of the respondents who evaluated the data used in the final analysis. The proportions of male and female respondents were 79.1% and 20.9%, respectively. The proportion of respondents in their 30s was 53.8 (the largest), followed by those in their 20s (35.1%) and 40s (10.7%). The respondents who graduated from four-year university programs accounted for 84.4%, while those who held Master’s and Ph.D. degrees were 10.7% and 4.9%, respectively.
4.2. Descriptive Statistics
Table 5 presents the calculated mean, standard deviation, skewness, and kurtosis for the collected questionnaire values. For the structural equation model analysis based on the maximum likelihood estimation using IBM® SPSS® Amos, the collected data pattern should have a normal distribution characteristic [29]. In this study, the skewness and kurtosis of each survey question value were calculated to evaluate the normality of the survey data [30]. The absolute values of the calculated skewness and kurtosis were less than 3.0 and 10.0, respectively—the minimum acceptable ranges proposed by Kline (2011) [31]. Consequently, the collected survey data did not violate normality.
4.3. Exploratory Factor Analysis Results
An exploratory factor analysis was performed to verify the latent variable structure for the collected survey questionnaire values. For the fitness test used to perform the factor analysis, Bartlett’s test for sphericity (χ2 = 4296.664 (325), p = 0.000 < 0.05) and the Kaiser–Meyer–Olkin (KMO) measure value (=0.946 > 0.6) were calculated. All values satisfied the minimum acceptable score proposed by Kaiser (1974).
Direct oblimin and principal component analysis were used for factor analysis rotation and factor extraction, respectively, to perform the exploratory factor analysis. The execution results demonstrated that the factor rotation converged in 14 iterations, and three components whose initial eigenvalues were greater than 1.0 were extracted (Table 6). The minimum acceptable score in the factor loading was set to 0.5 in this study [32]. The factor loadings for C1, C9, C12, and C13—survey questions in which cognition was included—were less than 0.5. Accordingly, these four survey questions and their values were excluded from the final analysis.
4.4. Internal Consistency Evaluation Results
Cronbach’s alpha coefficients were calculated to verify the internal consistency of the evaluation tool designed to measure the three sub-constructs. As presented in Table 6, all of the coefficients were greater than 0.7—the minimum acceptable value proposed by Cortina (1993) [33] and DeVellis (2003) [12].
4.5. Confirmatory Factor Analysis Results
A confirmatory factor analysis was performed to evaluate the construct validity, model fit, and nomological validity of the measurement proposed in this study. Convergent validity and discriminant validity were assessed to evaluate construct validity.
4.5.1. Convergent Validity Evaluation Results
The guideline proposed by Fornell and Larcker (1981) [34] was used to evaluate the convergent validity of the measurement. Their guideline suggested that all the standardized factor loadings of the calculated observed variables should be greater than 0.6 at a statistical significance level of p < 0.05, and the average variance extracted (AVE) and composite reliability values should be greater than 0.5 and 0.7, respectively. The calculated standardized factor loadings were observed. B8 (=0.599) and B9 (=0.500), which were used to measure the conation sub-construct, did not satisfy the minimum acceptable score. Furthermore, the root mean square error of approximation (RMSEA) value, one of the goodness of fit indices calculated to evaluate the model fit, was 0.086, which did not satisfy the acceptable score of 0.08 or less [35]. Thus, this study improved the model fit of the measurement by observing the modification indices of the model fit obtained through the confirmatory factor analysis after removing the two observed variables of B8 and B9.
The covariance modification index of the observed variables C4 and C5, C6 and C7, and C6 and C10 were all 20.0 or greater. Thus, covariance was additionally set for each of them [35]. Table 7 displays the confirmatory factor analysis results for the modified measurement. The standardized factor loading of each observed variable was 0.6 or greater, and the AVE and composite reliability values were greater than 0.5 and 0.7, respectively. Thus, the measurement of this study satisfied the appropriate convergent validity.
4.5.2. Discriminant Validity Evaluation Results
The discriminant validity of the measurement was evaluated based on whether the AVE square root of each construct was larger than the estimated correlation coefficient between constructs based on the guideline proposed by Fornell and Larcker (1981) [34].
As presented in Table 8, the AVE square root of the cognition sub-construct was 0.787, which was greater than the estimated correlation coefficient (=0.678) between the cognition and affection sub-constructs and the correlation coefficient (=0.754) between the cognition and conation sub-constructs. The AVE square root of the affection sub-construct was 0.778, which was greater than the correlation coefficient 0.709 between the affection and conation sub-constructs. Thus, the discriminant validity of the measurement satisfied the criterion proposed by Fornell and Larcker (1981) [34].
4.5.3. Model Fit Evaluation Results
In evaluating the model fit of the measurement (Table 9) he chi-square (χ2) statistics and normalized fit index (NFI), comparison fit index (CFI), Tucker–Lewis index (TLI), incremental fit index (IFI), and RMSEA were calculated. All indices satisfied the minimum acceptable values proposed in previous literature [32,36,37].
4.5.4. Nomological Validity Evaluation Results
The nomological validity was measured to evaluate whether the first-order sub-constructs (cognition, affection, and conation) could construct the second-order construct (DX) at a statistical significance level [38]. The confirmatory factor analysis results in Table 7 confirm that the standardized factor loadings of three first-order constructs were all greater than 0.6 at the significance level p < 0.05 (cognition = 0.849; affection = 0.799; and conation = 0.888) (Figure 3). Thus, the nomological validity verified that the three first-order constructs can be constructed as the second-order construct.
4.6. Results Summary
The results of this study verified three factors—cognition, affection, and conation—as critical sub-constructs that can construct the DX of a DL platform at a statistically significant level. Based on a confirmatory factor analysis, the calculated standardized factor loadings of three sub-constructs were observed. The results revealed that the factor loadings of cognition, affection, and conation were 0.849, 0.799, and 0.888, respectively. The standardized factor loading represents the correlation between latent constructs and observed variables. Each of the three sub-constructs had a high level of relatively uniform correlation with DX, a latent construct.
Based on the results of this study, the DX of a DL platform can be a construct that can be evaluated based on the developer’s competitiveness and value perceived from the platform, affection (expressed positively or negatively), and willingness to use the platform.
5. Conclusions
A DL platform is a development environment provided to developers that allows them to build a DL model. Since business success can potentially be obtained using DL techniques in various areas where logic, inference, prediction, or cognition function processing are required, the DL platform has seen an increase in investments and development, predominantly in global companies and academia [17]. The use of DL technology has increased across various sectors. However, no studies have been conducted on the experience of the developers who use DL platforms, in terms of evaluating their experiences with platform processes or tools.
In terms of needing to evaluate and research the experiences of developers, who are the main users of the deep learning platform, this study has interdisciplinary significance by proposing a reflective model for evaluating DX when using a DL platform. Furthermore, the evaluation is verified at a statistically significant level. Future research should use external validity to verify whether the derived evaluation questionnaire and the sub-constructs proposed in this study can explain the DX of other developer platforms, in addition to the DL platform.
It should be acknowledged that the present work did not consider any potential confounding factors or profiles including the type of deep learning platform respondents use in their companies or level of their work experience into the data analysis. These might impact the interpretation of our findings. Future work should replicate our findings with controlling for confounding variables to provide the robust support for the nomological validity of the measurement model proposed in the present work.
Author Contributions
Conceptualization, methodology, formal analysis, data curation, writing—original draft preparation: H.L., writing—review and editing, and funding acquisition: Y.P. All authors have read and agreed to the published version of the manuscript.
Funding
The publication of this paper has been partly supported by Graduate School of Techno, Kookmin University.
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Informed consent was obtained from all subjects involved in the study.
Data Availability Statement
Not applicable.
Acknowledgments
The authors would like to thank everyone who supported us in this work.
Conflicts of Interest
The authors declare no conflict of interest.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Figures and Tables
Enlarge this image.Figure 2. Developer Experience: conceptual framework (adapted from Fagerholm and Münch, 2012).
Pilot items for the measurement of cognitive factor and calculated CVRs.
| Items | CVR | Reference | |
|---|---|---|---|
| 1 | It provides developers with a superior development environment compared to other platforms. | 0.5 | Back and Parks (2003) [21] |
| 2 | Compared to other platforms, it provides developers with a more stable development environment. | 1 | |
| 3 | It provides convenience to developers. | 1 | Chowdhury and Salam (2015) [22] |
| 4 | It provides developers with a high level of information. | 0.5 | |
| 5 | It provides developers with a variety of add-on options (e.g., apps, resources, etc.). | 1 | |
| 6 | It provides comfort for developers. | 0.5 | |
| 7 | A fair policy applies to developers without exception. | −0.5 | |
| 8 | It provides developers with high-quality value for information. | 1 | Khanal (2018) [23] |
| 9 | It provides developers with high-quality value for their resources. | 1 | |
| 10 | It provides developers with high-quality value for their technology. | 1 | |
| 11 | It provides developers with a variety of values for information. | 0 | |
| 12 | It provides developers with a variety of values for their resources. | 0.5 | |
| 13 | It provides developers with a variety of values for technology. | 0 | |
| 14 | The price of the platform is reasonable. | 0.5 | |
| 15 | The platform is fast. | 1 | |
| 16 | The interaction between the platform and the developer is clear. | 1 | Venkatesh, Speier, and Morris (2002) [24] |
| 17 | The interaction between the platform and the developer does not require much mental effort. | 1 | |
| 18 | It is easy for developers to use the platform. | 1 | |
| 19 | It is easy to do what the developer wants to do. | 1 | |
| 20 | It evolves the work the developer wants to do. | 0.5 | |
| 21 | It increases the productivity of the developer’s development work. | 0.5 | |
| 22 | It improves the efficiency of the developer’s development work. | 1 | |
| 23 | It increases the usefulness of developer’s development work. | 1 | |
Pilot items for the measurement of affective factor and calculated CVRs.
| Items | CVR | Reference |
|---|---|---|
| 1. Developers love to use the platform. | 0.5 | Back and Parks (2003) [21] |
| 2. It feels good when developers use the platform. | 0 | |
| 3. It’s fun when developers use the platform. | 0 | Chowdhury and Salam (2015) [22] |
| 4. It’s fun when developers use the platform. | 0.5 | |
| 5. Developers are intrigued when they use the platform. | 1 | |
| 6. The platform is attractive. | 1 | Ahn and Back (2018) [20] |
| 7. Developers get a positive feeling from the platform. | 1 | |
| 8. Developers feel value from the platform. | 1 | Khanal (2018) [23] |
| 9. The platform is simple. | 0 | |
| 10. Developers feel satisfied with the platform. | 0.5 | Venkatesh, Speier, and Morris (2002) [24] |
Pilot items for the measurement of behavioral factor and calculated CVRs.
| Items | CVR | Reference | |
|---|---|---|---|
| 1 | Even higher prices this platform compared to other platforms, developers are now flat form has the idea that you need to use. | 1 | Back and Parks (2003) [21] |
| 2 | Developers intend to use the platform. | 1 | |
| 3 | Developers have a plan to use the platform. | 1 | Ahn and Back (2018) [20] |
| 4 | Developers have an effort to use the platform. | 0.5 | |
| 5 | Developers have the will to pay additional costs to use the platform. | 1 | Khanal (2018) [23] |
| 6 | Developers think that even if there is a cost to use the platform, it will not significantly affect the use of the platform. | 0.5 | |
| 7 | Developers have a clear idea that they will use the platform again. | 1 | |
| 8 | If a developer has access to the platform, he/she makes the developer intend to use the platform. | 1 | Venkatesh, Speier, and Morris (2002) [24] |
| 9 | If a developer has access to the platform, it gives that developer a willingness to use the platform. | 1 | |
| 10 | It gives developers the idea that the platform can be used voluntarily. | 1 | Venkatesh and Davis (2000) [25] |
| 11 | The developer of superior flat form without the use of force developers had the idea that you can use with the free will of its platform | 1 | |
Demographic profile for respondents (N = 225).
| Category | Item | Frequency | % |
|---|---|---|---|
| Gender | Male | 178 | 79.1 |
| Female | 47 | 20.9 | |
| Age | 20s | 79 | 35.1 |
| 30s | 121 | 53.8 | |
| 40s | 24 | 10.7 | |
| 50s | 1 | 0.4 | |
| Last educational background | Four-year university program | 190 | 84.4 |
| Graduate School Master’s Degree (Engineering Major) | 24 | 10.7 | |
| Postgraduate Ph.D. | 11 | 4.9 | |
| Sum | 225 | 100 |
Descriptive statistics for item measures: Mean, SD, Skewness, Kurtosis.
| Items | Mean | S.D. | Skewness | Kurtosis |
|---|---|---|---|---|
| C1. Compared to other platforms, it provides developers with a more stable development environment. (2) | 6.02 | 1.161 | −0.76 | −0.84 |
| C2. Provides convenience to developers. (3) | 5.81 | 1.207 | −0.85 | −0.07 |
| C3. Provides developers with a variety of add-on options (e.g., apps, resources, etc.). (5) | 5.57 | 1.227 | −0.68 | 0.056 |
| C4. It provides developers with high-quality value for information. (8) | 5.89 | 1.267 | −1.17 | 0.9 |
| C5. It provides developers with high-quality value for their resources. (9) | 6.06 | 1.182 | −1.28 | 1.08 |
| C6. It provides developers with high-quality value for technology. (10) | 6.14 | 1.175 | −1.38 | 1.532 |
| C7. The platform is fast. (15) | 6.04 | 1.213 | −1.07 | 0.066 |
| C8. The interaction between the platform and the developer is clear. (16) | 5.95 | 1.16 | −1.06 | 0.645 |
| C9. The interaction between the platform and the developer does not require much mental effort. (17) | 5.7 | 1.186 | −0.69 | −0.27 |
| C10. It is easy for developers to use the platform. (18) | 6.24 | 1.213 | −1.34 | 0.443 |
| C11. It is easy to do what the developer wants to do. (19) | 5.95 | 1.173 | −0.94 | 0.09 |
| C12. Improves the efficiency of the developer’s development work. (22) | 5.98 | 0.984 | −0.81 | 0.291 |
| C13. Increases the usefulness of the developer’s development work. (23) | 5.85 | 1.212 | −0.6 | −0.97 |
| E1. Developers are intrigued when they use the platform. (5) | 5.64 | 1.11 | −0.44 | −0.53 |
| E2. The platform is attractive. (6) | 5.75 | 1.086 | −0.53 | −0.37 |
| E3. Developers get a positive feeling from the platform. (7) | 5.48 | 1.094 | −0.35 | −0.25 |
| E4. Developers feel value from the platform. (8) | 5.65 | 0.994 | −0.21 | −0.76 |
| D1. Even if the platform price is high compared to other platforms, I think that developers should use the current platform. (1) | 5.84 | 1.053 | −0.58 | −0.22 |
| D2. Developers intend to use the platform. (2) | 5.84 | 1.043 | −0.66 | −0.05 |
| D3. Developers have a plan to use the platform. (3) | 5.66 | 1.181 | −0.71 | 0.122 |
| D4. Developers have the will to pay additional costs to use the platform. (5) | 5.84 | 1.101 | −0.76 | −0.15 |
| D5. Developers have a clear idea that they will use the platform again. (7) | 5.87 | 1.146 | −0.81 | −0.03 |
| D6. If a developer has access to the platform, he/she makes the developer intend to use the platform. (8) | 5.9 | 1.095 | −0.87 | 0.118 |
| D7. If a developer has access to the platform, it gives that developer a willingness to use the platform. (9) | 5.54 | 1.138 | −0.47 | −0.42 |
| D8. It gives developers the idea that the platform can be used voluntarily. (10) | 5.52 | 1.118 | −0.56 | −0.18 |
| D9. Even if the developer’s boss does not force the use of the platform, it makes the developer feel that they can use the platform freely. (11) | 5.86 | 1.187 | −1.37 | 2.713 |
Note. The value in parenthesis denotes the number of pilot items.
Table 6The results of the exploratory factor analysis.
| Item | Component | ||
|---|---|---|---|
| 1 | 2 | 3 | |
| C1 * | 0.407 | 0.219 | 0.29 |
| C2 | 0.671 | 0.236 | −0.038 |
| C3 | 0.668 | −0.108 | 0.283 |
| C4 | 0.821 | −0.061 | 0.005 |
| C5 | 0.89 | −0.068 | −0.01 |
| C6 | 0.822 | −0.035 | 0.109 |
| C7 | 0.835 | 0.04 | 0.054 |
| C8 | 0.693 | 0.245 | −0.071 |
| C9 * | 0.346 | 0.075 | 0.423 |
| C10 | 0.741 | 0.074 | 0.07 |
| C11 | 0.599 | 0.341 | 0.009 |
| C12 * | 0.441 | 0.437 | 0.03 |
| C13 * | 0.449 | 0.37 | −0.028 |
| E1 | 0.157 | −0.095 | 0.828 |
| E2 | 0.253 | 0.031 | 0.717 |
| E3 | −0.199 | 0.356 | 0.714 |
| E4 | 0.116 | 0.099 | 0.676 |
| B1 | 0.149 | 0.589 | 0.103 |
| B2 | 0.026 | 0.546 | 0.357 |
| B3 | 0.025 | 0.742 | 0.084 |
| B4 | 0.107 | 0.753 | 0.04 |
| B5 | 0.036 | 0.687 | 0.201 |
| B6 | 0.132 | 0.678 | 0.119 |
| B7 | −0.159 | 0.801 | 0.116 |
| B8 | 0.013 | 0.732 | −0.077 |
| B9 | 0.17 | 0.567 | −0.174 |
| eigenvalue | 12.927 | 1.977 | 1.55 |
| %variance | 49.72 | 7.605 | 5.96 |
| Cronbach’s alpha | 0.939 | 0.858 | 0.905 |
Note. * Items discarded in the final analysis.
Table 7The results of the confirmatory factor analysis.
| Second-Order Construct | First-Order Construct | Items | Standardized Factor Loading | AVE | Composite Reliability |
|---|---|---|---|---|---|
| DX | 0.849 | 0.716 | 0.883 | ||
| Cognitive | |||||
| component | C2 | 0.82 | 0.62 | 0.936 | |
| C3 | 0.707 | ||||
| C4 | 0.693 | ||||
| C5 | 0.75 | ||||
| C6 | 0.799 | ||||
| C7 | 0.862 | ||||
| C8 | 0.829 | ||||
| C10 | 0.783 | ||||
| C11 | 0.827 | ||||
| 0.799 | |||||
| Affective | |||||
| component | E1 | 0.798 | 0.605 | 0.859 | |
| E2 | 0.878 | ||||
| E3 | 0.709 | ||||
| E4 | 0.714 | ||||
| 0.888 | |||||
| Behavioral | |||||
| component | B1 | 0.684 | 0.59 | 0.909 | |
| B2 | 0.74 | ||||
| B3 | 0.783 | ||||
| B4 | 0.818 | ||||
| B5 | 0.813 | ||||
| B6 | 0.838 | ||||
| B7 | 0.683 |
The results of the confirmatory factor analysis.
| Construct | AVE | Cognitive | Affective | Behavioral |
|---|---|---|---|---|
| Cognitive | 0.62 | 0.787 | ||
| Affective | 0.605 | 0.678 | 0.778 | |
| Behavioral | 0.59 | 0.754 | 0.709 | 0.768 |
The results of the calculation of model fit indexes.
| Model Fit Index | χ2/df | NFI | CFI | TLI | IFI | RMSEA |
|---|---|---|---|---|---|---|
| Measurement model | 2.432 | 0.885 | 0.928 | 0.917 | 0.929 | 0.08 |
| Minimum criteria | (≤3.0) | (≥0.8) | (≥0.8) | (≥0.9) | (≥0.9) | (≤0.08) |
© 2021 by the authors.