Validation of the mental health continuum-short form: The bifactor model of emotional, social, and psychological well-being

Abstract

The Mental Health Continuum-Short Form (MHC-SF) is aimed at measuring the three dimensions of mental health; emotional, social, and psychological well-being. The purpose of the current study was to evaluate the psychometric properties of the MHC-SF within the context of Singapore and Australia. A total of 299 Singaporeans or permanent residents (59.2% female; mean age = 24.26, SD = 6.13) and 258 Australians or permanent residents (69% female; mean age = 23.95, SD = 8.66) completed the study. Confirmatory factor analyses were used to assess the structural validity of the MHC-SF. Internal consistency reliability was assessed via the Cronbach’s α and MacDonald’s ω reliability coefficients. Concurrent validity was examined against the World Health Organisation-Five Well-Being Index, discriminant validity using the Hospital Anxiety and Depression Scale, and criterion validity using a self-rated question of “Please rate your averaged level of mental health over the past month”, all via Pearson’s correlations. A bifactor model of the MHC-SF, where each item loaded on a general factor and simultaneously on their respective uncorrelated group factors, yielded the best fit to the data across both samples. Further investigations demonstrated that the general well-being factor accounted for majority of variances of the MHC-SF. Internal consistency reliability, concurrent validity, discriminant validity, and criterion validity were all demonstrated. In conclusion, the current study provided support for the bifactor model of MHC-SF and demonstrated evidence of good psychometrics across both samples. The results highlighted the unidimensionality of the measure, suggesting that it is more informative to interpret the aggregated score than scores of independent factors standalone.

Introduction

Prevalence of mental disorders have significantly increased over the years [1, 2] and have seen an increase of attention with regard to research, policies, diagnosis, and treatments, while investigations into mental health have been side-lined [3–5]. The oversight has negatively impeded our overall ability to address the increasing deterioration of mental health amongst all ages [6–9]. Disruptions to mental health are multi-layered and present differently to the varying age groups. Inadequacies in nurturing and attachment are major risk factors to negative well-being in children [6], while adolescents are faced with challenges such as academic pressure, and exposure to risky behaviours [10, 11]. Young adults encounter occupational, financial, and social difficulties [12–14], while older adults experience the deterioration of physical health, and perceived isolation [15, 16].

Overall, challenges to maintain a positive well-being are encountered throughout the lifespan, and considering the relationship of mental health and mental disorders, a valid and reliable standardised measure of well-being is required.

Mental health and well-being

Mental health has traditionally been conceptualised as the absence of psychopathology [17, 18]. However, it has changed in recent years; the two-continua model of mental health and mental illness states that mental health is related to, but different from, mental illness: where one continuum represents the individual’s state of mental health, while the other indicates the absence or presence of mental illness [19].

Keyes [19] investigated the relationship between mental health and mental illness using the representative data of 3,032 American adults between the ages of 25 and 74, obtained from the Midlife Development in the United States (MIDUS) survey. Confirmatory factor analyses demonstrated two factors correlating negatively, where measures of well-being loaded on one factor, while measures of psychopathology loaded on a second factor. This result is expected as mental health and illness while distinct, are strongly related, and mental health status have shown to be predictive of mental illness [20–22]. Doré, O’Loughlin [17] concluded that moderate to high inverse correlations between measures of mental health and mental illness to be supportive of the two-continua model and indicative of the divergence between the two constructs.

The structural results of the two-factor model of mental health and mental illness have been replicated over different samples, including American adolescents [23], Dutch adults [24], South-African adults [25], and with measures of mental illness such as the Brief Symptom Inventory [BSI; 18], Symptom Checklist-90-Revised [SCL-90-R; 26], and Hospital Anxiety and Depression Scale [HADS; 17].

Mental health is also referred to by the World Health Organisation as a state of well-being, specifically defining it as “a state of well-being in which the individual realises his or her own abilities, can cope with the normal stresses of life, can work productively and fruitfully, and is able to make a contribution to his or her community” [WHO; 27]. Mental health, thus, is no longer tied to the presence or absence of mental illness, but rather, pertains to the combination of (i) the individual’s mental state, (ii) functioning within the social context, and (iii) the individual’s psychological functioning [18, 22]. This definition and its three components are a direct reflection of our current understanding of well-being found within the literature.

Throughout the literature of well-being, two longstanding theories are highlighted, the hedonic concept of well-being and the eudaimonic concept of well-being [28–31].

The hedonic tradition was first conceptualised and views well-being as a sense of happiness, satisfaction, and interest in life [32, 33]. The three components would later be grouped under the umbrella of emotional well-being (EWB) by Keyes [34, 35].

Alternatively, the eudaimonic concept of well-being, was conceptualised later, when psychologists started to fault the narrow portrayal of well-being via the hedonic tradition, and argued that well-being ought to consider one’s functioning within the social and individual (i.e., psychological) realms [22]. Consequently, two multidimensional models of well-being; social well-being [SWB; 29], and psychological well-being [PWB; 30] were developed. The multidimensional model of SWB was developed by Keyes [29] comprises of five social dimensions. While the multidimensional model of PWB proposed by Ryff [30] comprises of six dimensions that reflect the challenges encountered as individuals strive to maximise their potential.

Incorporating both the hedonic and eudaimonic approaches, mental health can be defined as the combination of emotional, social, and psychological well-being and, therefore, be measured via these three components [19, 34]. Under the two-continua model, the absence of mental disorder is neither necessary nor sufficient to ensure a positive mental well-being, the contribution of emotional, social, and psychological well-being ought to be the primary considerations for measuring an individual’s state of mental health moving forward.

Existing measures of mental health and well-being

There are numerous well-established and validated measures of well-being, including the World Health Organisation Quality of Life Assessment [WHOQOL-100; 36], World Health Organisation Quality of Life Assessment–Brief [WHOQOL-BREF; 37], World Health Organisation-Five Well-Being Index [WHO-5; 38], Positive and Negative Affect Schedule [PANAS; 39], 12-item General Health Questionnaire [GHQ-12; 40], Satisfaction with Life Scale [SWLS; 41], and Control, Autonomy, Self-Realisation, and Pleasure Scale [CASP-19; 42].

While the above-mentioned measures have been heavily utilised, well-validated across different populations, and available in a myriad of languages, they each hold shortcomings that may deter an accurate representation and assessment of one’s mental health according to the discussed definition and its three contributory factors.

Firstly, questionnaires such as the WHOQOL-100, WHOQOL-BREF, WHO-5, PANAS, SWLS, and CASP-19 measure only one or a few aspects of well-being or utilise theoretical constructs that differ from the hedonic and eudaimonic concepts of well-being. For instance, the WHOQOL-100 and WHOQOL-BREF includes physical health and environmental facets, while the WHO-5 and PANAS specifically targets the EWB component of mental health. The SWLS measures a combination of EWB and PWB aspects, and the CASP-19 utilises differing factors such as control, autonomy, self-realisation, and pleasure. Taking into consideration of the widely recognised definition of mental health by WHO [27] and the literature surrounding well-being, it is paramount for any measures of mental health to assume the three core components of mental health, emotional, social, and psychological well-being, so as to assess and present an all-encompassing representation of the individual’s mental health.

Secondly, instruments such as the GHQ-12 include not only well-being, but also items of psychopathology. While mental health and mental illness have shown to be strongly related [20–22], investigations into the two-continua model have demonstrated that mental illness ought to be viewed as a distinct state, and thus, its assessment to be discrete of psychopathology [17, 18, 43].

Lastly, a clinician and researcher’s choice of measure is often guided by various practical constraints, with a major consideration being administration time [44]. Instrument such as the WHOQOL-100 is rather lengthy and thus, not optimal for research or practical usage due to increased drop-out rates and fatigue [45].

Throughout the course of the current study, a new assessment tool, Well-being Numerical Rating Scales [WB-NRSs; 46], was developed and published. The instrument demonstrated numerous positives, including a concise 5 items format that comprehended most dimensions of well-being, strong psychometric properties, and the ability to detect changes in well-being after intervention. However, while it is a promising tool, it is still new and pending of further validation studies, as currently, only one study has been conducted.

Mental health continuum-short form

To combat the inability of existing mental health assessment tools to adequately represent the current definition of mental health, the Mental Health Continuum-Short Form [MHC-SF; 25] was developed. The MHC-SF is an abbreviated version of the 40-item Mental Health Continuum-Long Form [MHC-LF; 34].

Comprising of 14 items, each theoretical dimension of emotional (3 items), social (5 items), and psychological (6 items) well-being is operationalised using a single item (Table 1). Consequently, it ensured that the MHC-SF was theory driven and encompassed all the theoretical components of mental health.

[Figure omitted. See PDF.]

Participants respond on a 6-point Likert scale, ranging from 0 (Never) to 5 (Everyday). Scores can be computed either continuously or categorically.

Continuous scoring would see the score of each subscale summed (i.e., EWB [range, 0 to 15], SWB [range, 0 to 25], and PWB [range, 0 to 30]), and the sum of all scores yields the overall score for well-being (range, 0 to 70). In this scenario, higher scores would indicate more positive mental health, and vice versa. No suggested cut-offs were provided [25].

The categorical scoring proposed by Keyes [34], on the other hand, captures three categorical diagnoses: languishing, moderate, and flourishing mental health. This scoring system is dependent on a specific cut-off criterion of the various well-being factors and thus, relies on the score of each subscale rather than the overall score. Languishing mental health is diagnosed when an individual reports having experienced at least one of the three EWB symptoms and at least 6 of the 11 SWB and PWB symptoms as 0 (Never) or 1 (Once or twice) within the past month. Conversely, flourishing mental health is diagnosed when an individual reports having experienced at least one of the three EWB symptoms and at least 6 of the 11 SWB and PWB symptoms as 4 (Almost every day) or 5 (Everyday) in the past month. Individuals who are neither categorised as languishing or flourishing are considered moderately mentally healthy.

The initial evaluation of the MHC-SF was conducted by Keyes, Wissing [25] with a sample of 1,050 Setswana-speaking adults in the Northwest province of South Africa. Results indicated acceptable internal consistency of the entire MHC-SF (α = .74), and questionable-to-acceptable range over the various subscales; EWB (α = .73), SWB (α = .59), and PWB (α = .67). Later evaluation studies, however, reported internal consistency of the MHC-SF total scale and its various subscales within the acceptable-to-high range (α = .74 to .94), in various countries such as Hong Kong, India, Japan, Malaysia, Netherlands, United Kingdom, and Vietnam [18, 47, 48].

Within the initial study, Keyes, Wissing [25] reported that the MHC-SF demonstrated convergent validity by correlating moderately to strongly with measures of subjective well-being, such as the Affectometer Positive Affect Scale (r = .52, p < .001) and SWLS (r = .37, p < .001), and measures of criteria of positive mental health, such as the Generalised Self-Efficacy Scale (r = .39, p < .001), and Coping Strategies Scale (r = .34, p < .001).

Discriminant validity of the MHC-SF has also been shown. The MHC-SF has been reported to be inversely correlated (r = -.22 to -.78) with measures of mental illness, such as the BSI [18], SCL-90-R [26], and HADS [17].

With regard to the structure of the MHC-SF, Keyes, Wissing [25] conducted confirmatory factor analyses on three theoretical models: (1) one-factor model based on the assumption that all 14 items loaded on a single global factor of well-being; (2) two-factor model comprising of the two correlated dimensions of well-being: hedonic and eudaimonic well-being; and (3) three-factor model comprising of the three correlated core components of well-being: emotional, social, and psychological well-being. Results indicated that both the single and two-factor models fitted poorly, and the three-factor model was concluded to be the best-fitting model to the data, χ² = 269.40, df = 62; GFI/AGFI = .96/.94; CN = 354.50; RMSEA = .06; AIC = 345.90; N = 1050 [25]. Later studies that evaluated the MHC-SF with differing populations and translated questionnaires, including reported similar results, indicating that a three-factor model fitted the best in comparison to the single and/or two-factor models, χ² = 103.48 to 527.96, df = 74; GFI = .82 to .91; AGFI = .61 to .88; RMSEA = .03 to .09; CFI = .94 to .99; SRMR = .05 to .07 [17, 18, 49, 50].

Bifactor model of MHC-SF.

Whilst the MHC-SF was developed by Keyes [23] under the assumption of a three-factor structure (i.e., EWB, SWB, and PWB), and prior evaluation studies have demonstrated that a three-factor model to be the best fit in comparison to the single and/or two-factor models, researchers are now suggesting that the three-factor structure may be problematic [e.g., 48].

Firstly, despite the support of the three-factor model concluded in some of the previous studies [e.g., 18], the majority of studies [e.g., 17, 49] yielded fit indices that can only be considered marginally acceptable according to conventional cut-off criteria [51]. For instance, Doré, O’Loughlin [17] reported fit indices of χ² = 527.96; df = 74; χ²/df = 7.13; RMSEA = .07; CFI = .94; SRMR = .05; TLI = .93. For this study, while the three-factor model demonstrated better fit in comparison to the one- and two-factor models, only two of the four fit indices (CFI and SRMR) were within acceptable range, and therefore, the result did not provide conclusive support to the three-factor structure of MHC-SF. Jovanović [52] highlighted the need for further investigation, so as to identify the sources of misfit, improve upon the scale, and propose, if necessary, another theoretical model that better represent the MHC-SF.

Secondly, de Bruin and du Plessis [47] have indicated that theoretically, the assumed three-factor model of MHC-SF does not align with or justify the current scoring system of the instrument. A three-factor structure of MHC-SF would theoretically translate into the calculation and interpretation of scores separately for each of the subscale, EWB, SWB, and PWB. However, researchers, to the best of our knowledge, utilise a total MHC-SF score (i.e., continuous scoring system), which is computed by aggregating the scores across the 14 items [e.g., 18, 25]. This practice suggests the presence of a general well-being (GWB) factor, and contradicts the initial aim of the instrument, which was to measure three relatively distinct components of well-being [52]. Several evaluation studies, including Jovanović’s [52] and Petrillo et al.’s [58] studies, have included a second-order factor in their analyses, revealing the presence of a GWB factor amongst the 14 items.

A second-order factor analysis, however, does not allow for direct comparisons of strengths between the GWB factor and the group factors (i.e., EWB, SWB, and PWB), and thus is not able to measure any contribution of unique variance by the various subscales [47, 52]. Consequently, numerous researchers including de Bruin and du Plessis [47], Jovanović [52], and Żemojtel‐Piotrowska, Piotrowski [48] have incorporated a bifactor model in their analyses, where each item loaded on a general factor (i.e., GWB) and simultaneously on one of the uncorrelated group factors (i.e., EWB, SWB, and PWB). Bifactor model analysis has been widely utilised in understanding the structures of multidimensional constructs measures such as self-esteem [53], depression [54], and intelligence [55]. Importantly, it allows for the separation of the GWB dimension from the group factors (i.e., EWB, SWB, and PWB), enabling investigations into the proportion of variance contributed by each dimension, dimensionality of the data, and reliability of the GWB dimension and subscales [47, 48, 56].

In a cross-cultural evaluation study of MHC-SF’s structure involving 38 countries (N = 8,066), Żemojtel‐Piotrowska, Piotrowski [48] reported that in all 38 nations, the bifactor model fitted better than the three-factor model, with 16 countries demonstrating good fit, and acceptable fit in all others, except Kenya and Iran, where the fit was marginal. Excluding the two countries which demonstrated marginal fit, the fit indices were as follows: χ² = 74.20 to 155.31, df = 63; RMSEA = .32 to .77; CFI = .90 to .98; SRMR = .37 to .63. In addition, the indices evaluating the reliability and dimensionality of the MHC-SF also demonstrated prominent results in all samples. The Macdonald’s ω reliability coefficients ranged from .82 to .95 for the GWB factor and .57 to .92 for the subscales. In addition, the Macdonald’s ω_H coefficient ranged from .56 to .87 (M = .80), and the explained common variance (ECV) index of the GWB factor ranged from .40 to .76 (M = .66), while the mean Macdonald’s ω_S coefficients for the various subscales are as follows: .29 for EWB; .31 for SWB; and .12 for PWB.

Studies, including an Australian sample [57], who have incorporated a bifactor model as part of their structural analysis of the MHC-SF have also found similar results; good and better fit over the other proposed models, good reliabilities coefficients, and unidimensionality of the data [e.g., 47, 48, 52].

Overall, the results indicate that a bifactor model would structurally better represent the MHC-SF, and the combination of a high ECV and high ω_H coefficient indicates unidimensionality of the model and supports the computation and interpretation of a total MHC-SF score via the continuous scoring system.

Aims and hypotheses.

The MHC-SF is seemingly an instrument that has the ability to address the limitations of the other measures of mental health and has been translated, validated, and utilised across many different languages and countries, such as Chinese [49], Italian [58], Serbian [52], and Polish [59]. In addition, given the importance of mental health, and its relationship to mental disorders, it is paramount that such a promising instrument is properly evaluated and validated in as many different contexts as possible. To the best of our knowledge, there has only been one study conducted with an Australia sample [57], and none with a Singapore sample. Therefore, the primary aim of the current study was to examine the psychometric properties of the MHC-SF within the Singapore and Australia context. The second aim was to provide further clarification on the structural model of the MHC-SF and better inform the computation and interpretation of its scores. Thus, with consideration of the discussed literature and aims of the current study, the hypotheses of the current study (applicable to both samples) were as follows:

1. H1, A bifactor model of MHC-SF would demonstrate good, and comparatively better fit than the other models (i.e., one-factor, two-factor, and three-factor models).

2. H2, Results of the bifactor models would demonstrate a combination of high ECV (≥ .70) and high Macdonald’s ω_H coefficient (≥ .80), thus providing support for unidimensionality and interpretation for a total MHC-SF score.

3. H3, MHC-SF total scale and various subscales would demonstrate high internal consistency reliability (Cronbach’s α and Macdonald’s ω reliability coefficients ≥ .80).

4. H4, MHC-SF’s total and subscales scores would demonstrate moderate to strong positive correlations with scores in WHO-5, which measures an individual’s current mental well-being based on positive mood, vitality, and general interest (Pearson’s r ≥ .30), demonstrating concurrent validity.

5. H5, MHC-SF’s total and subscales scores would demonstrate moderate to strong positive correlations with the participants’ self-rated level of mental health (Pearson’s r ≥ .30), demonstrating criterion validity.

6. H6, MHC-SF’s total and subscales scores would demonstrate moderate to strong negative correlations with both the depression and anxiety subscales of HADS (Pearson’s r ≥ -.30), demonstrating discriminant validity.

Methods

Design and participants

The current study employed a cross-sectional and correlational research design. An individual was eligible for the study if he/she was above the 18 years old and a citizen or permanent resident (PR) of Singapore or Australia. Participants were either recruited via convenience sampling through the James Cook University’s (Singapore and Australia campuses) research participation program, to which upon completion, received 1 research credit point, or have encountered the study through snowball techniques, primarily distributed via social networks.

A total of 588 participants were recruited, but 31 records were omitted due to missing data. Of which, 410 participants were recruited via the university’s research participation program and 147 participants via the snowball technique. The data was then separated into two groups: the Singapore sample, consisting of 299 Singaporeans or PRs; and the Australia sample, consisting of 258 Australians or PRs. The characteristics of participants are shown in Table 2.

[Figure omitted. See PDF.]

Measures

Mental health continuum-short form.

The MHC-SF [25] is a 14-item self-reported measure of mental health, investigating the respondent’s emotional (3 items), social (5 items), and psychological (6 items) well-being. Participants responded on a 6-point Likert scale, ranging from 0 (Never) to 5 (Everyday). Scores can be computed either continuously or categorically. The current study predominantly utilised the continuous scoring system; the scores of each subscale summed (i.e., EWB [range, 0 to 15], SWB [range, 0 to 25], and PWB [range, 0 to 30]), and totality of the 14 items for the overall well-being score (range, 0 to 70). Higher scores would indicate more positive mental health, and vice versa.

The initial evaluation of the MHC-SF by Keyes, Wissing [25] reported acceptable internal consistency of the entire MHC-SF (α = .74), and questionable-to-acceptable range over the various subscales; EWB (α = .73), SWB (α = .59), and PWB (α = .67). While later studies, such as de Bruin and du Plessis [47], Lamers, Westerhof [18], and Żemojtel‐Piotrowska, Piotrowski [48] reported internal consistency of the MHC-SF total scale and its various subscales within the acceptable-to-high range (α = .74 to .94). The MHC-SF has good convergent validity [25] and good discriminant validity [17, 18, 26].

World health organisation-five well-being index.

The WHO-5 [38] is a 5-item self-reported measure of current mental well-being based on positive mood, vitality, and general interest. Respondents rate their degree of agreement with statements that describe how they felt in the last two weeks (e.g., “I have felt calmed and relaxed”), on a 6-point Likert scale, ranging from 0 (At no time) to 5 (All of the time). Aggregation across the five items would result in a total score ranging from 0 to 25, with 0 representing worst possible and 25 representing best possible quality of life.

Studies have demonstrated the WHO-5 to have high internal consistency, with Cronbach’s α coefficients above .80 [60, 61]. Cronbach’s α in the current study was .93 and .90 for the Singapore and Australia sample respectively, indicative of excellent internal consistency. In addition, studies have reported the WHO-5 to have good construct validity [62, 63], good predictive validity [64], and good convergent validity [65, 66].

Hospital anxiety and depression scale.

The HADS [67] is a 14-item brief self-reported questionnaire that measures symptoms of anxiety and depression, with seven items measuring each subscale, HADS-Anxiety and HADS-Depression. Respondents rate their degree of agreement with statements that describe how they felt in the last seven days (e.g., “I feel as if I am slowed down”), on a 4-point Likert scale. Aggregating the scores for each subscale would result in a subscale score ranging from 0 to 21, with 0 to 7 representing normal range, 8 to 10 representing borderline abnormal range, and 11 to 21 representing abnormal range.

HADS has demonstrated acceptable-to-high internal consistency across the literature; with Cronbach’s α ranging from .68 to .93 for the anxiety, and .67 to .90 for depression subscales [68]. Both subscales demonstrated good internal consistency for the current study; Cronbach’s α = .89 and .85 for the anxiety subscale for the Singapore and Australia samples, respectively; and Cronbach’s α = .87 and .77 for the depression subscale for the Singapore and Australia samples, respectively. Studies have also identified the HADS to have good factorial validity [69] and discriminant validity [68].

Self-rated mental health question.

A single self-rated question inquiring about the respondent’s evaluation of their own mental health was included (“Please rate your averaged level of mental health over the past month”). Participants responded on a 11-point Likert scale, ranging from 0 (Poor) to 10 (Excellent). This question was included as a mean of comparison to investigate the criterion validity of the MHC-SF.

Procedure

All procedures in the current study were approved and in accordance with the ethical standards of James Cook University’s Human Research Ethics Committee (Ethics number: #H7889).

The questionnaires were administered online via Qualtrics. The participants were free to complete the survey on their own time, it was expected that the survey took no longer than 15 minutes to complete. Participants were informed they were free to withdraw while responding, but they could not withdraw their data once submitted because the data were collected anonymously and could not be identified.

When the participants first clicked on the link to the study, they were presented with an Information Sheet briefly explaining the nature of the study and the questionnaires they will be required to complete. Informed consent was acknowledged when the participants continued past the Information Sheet. Subsequently, they were invited to complete the Background Information Form, the MHC-SF, the WHO-5, and the HADS. Finally, they were presented with a Feedback Form reiterating the nature of the study and thanking them for their participation.

Lastly, participants who were students of JCU and signed up via the research participation system would be granted one credit point towards their required research participation.

Statistical analysis

Analyses were conducted for each sample separately. Pearson’s correlations coefficients were conducted using the SPSS version 21, confirmatory factor analyses (CFA) were conducted using the AMOS version 21, and the ECV and MacDonald’s ω indices were calculated using the Bifactor Indices Calculator by Dueber. All statistical analyses were conducted with alpha level set at .05.

Confirmatory factor analyses (CFA) were employed to find the best fitting model and test structural validity of the MHC-SF. The models were identified by fixing the latent factor variances to 1 and freely estimating the factor loadings [70]. Knowing the limitations of the chi-square test of overall model fit [71], the following fit indices and their respective recommended cut-offs by Hu and Bentler [51] were utilised: χ²/df with a value of less than 3 being adequate; root mean square error of approximation (RMSEA) with a value of .06 or less being indicative of a good fit; standardised root mean square residual (SRMR) with a value of .08 or below as indications of good fit; and comparative fit index (CFI) with a value greater than .90 being acceptable fit and .95 and above being good fit.

MacDonald’s ω_H and ω_S coefficients were caluclated to identify any unique variance of well-being contributed by the GWB dimension and the various subscales [56]. The MacDonald’s ω_H coefficient will report the share of total variance contributed by the GWB factor, while the MacDonald’s ω_S coefficient will identify the unique share of variance contributed by each subscale, excluding the contribution of the GWB factor. According to Reise, Bonifay [72], when the MacDonald’s ω_H coefficient is > .80, the model be considered unidimensional and consequently, justifying the current aggregating scoring system of the MHC-SF employed by all researchers. ECV coefficient was also calculated to measure the relative strength of the GWB factor to the group factors. According to Rodriguez, Reise [73], an ECV value of .70 and above would indicate a strong general factor and that the common variance is essentially unidimensional. Therefore, a high ECV value, in tandem with a high MacDonald’s ω_H coefficient, would provide strong support to the unidimensionality of the model and the use of the continous scoring system.

Due to Cronbach’s α limitations with multidimensional measures [74], the current study included the use of MacDonald’s ω coefficient to test for the internal consistency reliability of MHC-SF. The MacDonald’s ω coefficient reflects the proportion of total item variance explained by the combined contribution of the GWB factor and the three group factors. As such, the MacDonald’s ω coefficient, akin to Cronbach’s α, allows the reliability of the GWB dimension and the three subscales to be evaluated. According to Reise, Bonifay [72], the MacDonald’s ω coefficient can be interpreted similiarly to other popular indices of reliabily, therefore, the current study employed the recommendations by Cortina [75]: .70 as acceptable, and .80 or higher as good. Similarly, Cronbach’s α was calculated in tandem and the same recommended cut-offs were employed.

Concurrent validity of the MHC-SF was investigated via its correlations with other validated measures of mental health. For the current study, a Pearson’s correlation between the MHC-SF and WHO-5, which measures the individual’s current mental well-being based on positive mood, vitality, and general interest was conducted, and a correlation coefficient of .30 or above would indicate good convergent validity [76].

Criterion validity of the MHC-SF was tested via a Pearson’s bivariate correlation of the MHC-SF with the self-rated mental health question (“Please rate your averaged level of mental health over the past month”). Similarly, a correlation coefficient of .30 or above would be indicative of good criterion validity [76].

Indication of discriminant validity was based on the current understanding and results of prior studies of the two-continua model of mental health and mental illness. According to Doré, O’Loughlin [17], a negative moderate to high correlation coefficient of the MHC-SF and measure of psychopathology (e.g., HADS) would be supportive of the two-continua model and indicative of discriminant validity. Thus, the current study conducted a Pearson’s bivariate correlation between MHC-SF and the two subscales of HADS which measures symptoms of anxiety and depression, and employed the cut-off of |.30| or above as indication of moderate correlation, and |.50| or greater as strong correlation [76].

Results

The descriptive statistics, continuous scores and categorical results of the MHC-SF for both the Singapore and Australia samples are presented in Table 3.

[Figure omitted. See PDF.]

Structural validity

Based on the theoretical underpinnings and results of prior studies (e.g., de Bruin & du Plessis, 2015; Hides et al., 2016; Jovanović, 2015; Żemojtel‐Piotrowska et al., 2018), the current study tested four different CFA models of the MHC-SF: (a) a single-factor model, where all 14 items loaded on one general dimension of well-being (Fig 1); (b) a two-factor model with two correlated dimensions of well-being (i.e., hedonic well-being, comprising of items 1 to 3, and eudaimonic well-being, comprising of items 4 to 14; Fig 2); (c) a three-factor model with three correlated dimensions of well-being (i.e., EWB consisting of items 1 to 3, SWB consisting of items 4 to 8, and PWB consisting of items 9 to 14; Fig 3); and (d) a bifactor model with a GWB factor and three uncorrelated group factors capturing the specific variance of EWB, SWB, and PWB (Fig 4). A single second-order factor model was not tested because a hierarchical model with three first-order factors is statistically equivalent to the correlated three-factor model.

[Figure omitted. See PDF.]

Note. GWB = general well-being.

[Figure omitted. See PDF.]

Note. HWB = hedonic well-being, EuWB = eudaimonic well-being.

[Figure omitted. See PDF.]

Note. EWB = emotional well-being, SWB = social well-being, PWB = psychological well-being.

[Figure omitted. See PDF.]

Note. GWB = general well-being, EWB = emotional well-being, SWB = social well-being, PWB = psychological well-being.

The fit indices across both samples for all four models are shown in Table 4. Across both samples, the single-factor model did not fit the data well. With the Singapore sample, only the SRMR fit index (.80) can be considered a good fit, while the Australia sample had all four fit indices lying outside of acceptable ranges.

[Figure omitted. See PDF.]

The two-factor model, albeit demonstrated improvements in fitting compared to the single-factor model across both samples, could not be considered acceptable. The Singapore sample reported two fit indices outside of their respective acceptable ranges, with only the SRMR (.07) and CFI (.92) showing indications of a good and acceptable fit respectively. The Australia sample fitter poorer comparatively, with three of the four fit indices lying outside of acceptable ranges, with only the CFI fit index of .91 being marginally acceptable.

Based on the combination of indices, the three-factor model for both samples, whilst demonstrating comparatively better fit than the prior two models, could not be confidently concluded to be within acceptable ranges. Both samples had only two of the four fit indices within the recommended ranges. Singapore sample reported SRMR of .07 and CFI of .94, indicative of good and acceptable fit respectively, while the Australia sample demonstrated adequate χ²/df value of 2.65 and good CFI value of .95.

Unlike the previous three models, the bifactor model for both Singapore and Australia samples can be conclusively concluded to have demonstrated good and excellent fit to the data respectively. The bifactor model for the Singapore sample reported the best fit to its data, with only the RMSEA fit index (.08) marginally outside of its recommended range. Similarly, the bifactor model for the Australia sample reported the best fit to its data, with all fit indices well within their recommended ranges.

Following the fit indices, the standardised factor loadings, MacDonald’s ω_H and ω_S coefficients, and ECV of the bifactor model for Singapore sample (Table 5) and Australia sample (Table 6) were investigated.

[Figure omitted. See PDF.]

[Figure omitted. See PDF.]

The bifactor model for the Singapore sample produced a strong GWB factor, with standardised factor loadings ranging from .69 to .88. Notably, all items had higher loadings on the GWB factor than on their respective specific factors. In addition, whilst all EWB items had significant loadings on both the GWB and EWB factors, two SWB items (items 4 [social contribution] and 5 [social integration]) and most of PWB items (excluding item 9 [self-acceptance]) had low loadings on its specific factors. The results provided a brief indication that the variances of all EWB items were likely spilt between GWB factor and its specific factor, while the variances of some SWB and PWB items were likely explained by the GWB factor.

In addition, the model highlighted the dominant strength of the GWB factor. The MacDonald’s ω_H coefficient, which reflects the proportion of total variance explained by the general factor was .92, indicating the significant contribution of the GWB factor to the model. Furthermore, the ECV index of the GWB factor was .85, which further supported the substantial contribution of the GWB factor in the true score variance. On the other hand, the MacDonald’s ω_S coefficients of the various group factors (EWB = .19, SWB = .17, PWB = .03), demonstrated their minimal contribution to the true score variance when excluding the contribution of the general factor. Notably, the PWB factor reported a comparatively smaller amount of unique variance (.03), which suggested that the variance it captures is better represented by the combination of all three subscales of MHC-SF, whereas the EWB (.19) and SWB (.17) factors demonstrated more distinct standalone contributions.

In combination, results of the MacDonald’s ω_H and ω_S coefficients as well as the ECV index strongly suggested that majority of the variance captured by the MHC-SF were shared by the three scales. Accordingly, the model can be identified as unidimensional, therefore suggesting that the GWB factor of MHC-SF to be most appropriate as an indicator of overall mental health in Singapore.

Similar to the model obtained in Singapore, the bifactor model of the Australia sample produced a strong GWB factor, with standardised factor loadings ranging from .60 to .80. All items except item 6 (SWB; [social actualisation]) and item 12 (PWB; [personal growth]) reported higher loadings on the GWB factor than on their respective specific factors. In addition, whilst all EWB items had significant loadings on both the GWB and EWB factors, two SWB items (items 4 [social contribution] and 5 [social integration]) and most of PWB items (excluding item 12 [personal growth]) had low loadings on its specific factors. This result, provided a brief indication that the variances of all EWB items were likely spilt between GWB factor and its specific factor, while the variances of some SWB and PWB items were likely explained by the GWB factor.

Results of the MacDonald’s ω_H and ω_S coefficients and ECV index for the Australia model were highly similar to that of the Singapore’s sample. The model highlighted the dominant strength of the GWB factor, with high MacDonald’s ω_H coefficient (.89) and high ECV index (.76). Likewise, the MacDonald’s ω_S coefficients of the various group factors were low, with the PWB factor reporting comparatively lower amount of unique variance (.05), while the EWB (.23) and SWB (.24) factors demonstrated more distinct standalone contributions.

Akin to the Singapore model, results of the Australia sample suggested that majority of the variance captured by the MHC-SF were shared by the three scales. Accordingly, the model can be identified as unidimensional, suggesting that the GWB factor of MHC-SF to be most appropriate as an indicator of overall mental health in Australia.

Internal consistency reliability

Table 7 reports the Cronbach’s α and MacDonald’s ω reliability coefficients of MHC-SF for both samples.

[Figure omitted. See PDF.]

Cronbach’s α coefficients was .95 (Singapore) and .94 (Australia) for the total MHC-SF score, and ranged from .87 to .92 for the three subscales. The MacDonald’s ω reliability coefficients, which demonstrates the proportion of true score variance while accounting for all factors, was .97 (Singapore) and .96 (Australia) for the GWB index, and ranged from .90 to .92 for the three subscales. Across both countries and assessments of reliability, the MHC-SF demonstrated high internal consistency across the total scale and with its various subscales.

Concurrent validity

The MHC-SF measuring an individual’s well-being across three dimensions (emotional, social, and psychological) was compared against the WHO-5 which measures an individual’s current mental well-being based on positive mood, vitality, and general interest. Pearson’s correlations of the MHC-SF total scale and subscales with the WHO-5 were examined (Table 8). The MHC-SF total scale and subscales across both countries demonstrated strong positive correlations with the WHO-5, ranging from r = .66 to .86, and are indicative of concurrent validity for the MHC-SF.

[Figure omitted. See PDF.]

Criterion validity

Pearson’s correlations of the MHC-SF total scale and subscales with the self-rated mental health question (“Please rate your averaged level of mental health over the past month”) were examined (Table 9). The MHC-SF total scale and subscales across both countries demonstrated moderate to strong positive correlations with the self-rated question, ranging from r = .52 to .72, and are indicative of criterion validity for the MHC-SF.

[Figure omitted. See PDF.]

Discriminant validity

Table 10 shows the Pearson’s correlations between MHC-SF total scale and subscales and both of HADS subscales for both samples. HADS anxiety and depression subscales negatively correlated with MHC-SF’s total scale and its three subscales, ranging from -.45 to -.71. Higher presence of psychopathology symptoms was moderately linked to lower levels of mental well-being across all dimensions. This result provided support to the two-continua model of mental health and mental illness, where both constructs are thought to be highly related yet distinct. In addition, the result is indicative of discriminant validity for the MHC-SF.

[Figure omitted. See PDF.]

Discussion

The current study aimed to evaluate the psychometric properties of the MHC-SF in Singapore and Australia, to clarify on the structural model of the MHC-SF, as well as to better inform the computation and interpretation of its scores.

Similar to prior studies that have included a bifactor model within their consideration [e.g., 47, 48, 52], results of the current study demonstrated that a bifactor model of the MHC-SF yielded the best fit to the data. The bifactor model with one GWB factor and three specific factors of EWB, SWB, and PWB, reported good and excellent fittings to the Singapore and Australia samples. Unlike the bifactor model, the fit indices of the single, two, and three-factor models of the MHC-SF could not be confidently concluded to be within acceptable ranges. The results of the current study do not support the three factor structure of the MHC-SF proposed by the original authors [25] nor the generally acknowledged three factor model results of prior CFA studies [e.g., 17, 18].

Further investigations into the common and specific variances captured by the MHC-SF total scale and the three subscales have revealed the overall strength of the GWB factor relative to the group factors. Like prior studies [e.g. 47, 48, 52], the GWB factor contributed to two-thirds or more of the total and common variance, whilst the contribution of the specific factors ranged from 3% to 24%. Notably, the PWB subscale captured lower contribution of unique variances when compared to its two counterparts (i.e., EWB and SWB subscale), suggesting that PWB is not as distinct of a standalone factor and that it largely overlaps with the entire scale. A possible explanation could be the broadness of the six theoretical dimensions of PWB proposed by Ryff [30]. For instance, the dimension of positive relations with others could be interpreted as part of the multidimensional model of SWB, while the dimension of purpose in life is highly related to the interest and life satisfaction dimensions found in EWB. However, due to the overlap, as suggested by Żemojtel‐Piotrowska, Piotrowski [48], if a shorter instrument is needed, utilising the PWB subscale alone may be adequately effective.

More importantly, while the results of the study suggested that EWB and SWB factors do uniquely contribute to well-being, and there is a qualitative distinction between the contribution of the GWB factor and the three specific factors. The low MacDonald’s ω_S coefficients and factor loadings strongly suggested that the three subscales do not yield precise and distinct enough measures of well-being to be utilised in research or clinical applications. Simultaneously, the high MacDonald’s ω_H coefficients and ECV values of the GWB factor across both samples indicated that the MHC-SF ought to be considered as unidimensional. Thus, the calculation of separate subscale scores is not theoretically justified, and the current continuous scoring approach of an aggregated score across all 14 items ought to be the default. This also highlighted the limitations within the categorical diagnosis scoring system proposed by Keyes [34]. To be identified as either languishing or flourishing mental health, the individual must obtain specific cut-off requirements within the various group factors (i.e., EWB, SWB, and PWB). Consequently, due to the requirement of needing individualised interpretation of each subscale score, the categorical scoring system heavily contradicts the unidimensionality of the MHC-SF model, and ought to be avoided. It would be more appropriate to score the MHC-SF via the continuous scoring system and interpret the results using the total MHC-SF score as an indicator of the respondent’s mental health, at least within the context of Singapore and Australia.

The unique trend of the common variance captured by the GWB factor, first reported by Żemojtel-Piotrowska, Piotrowski [48], was also identified in the current study. Across the results of 38 countries, Żemojtel-Piotrowska, Piotrowski [48] identified that in countries with high collectivism such as Nepal, Japan, and India, the strength of the GWB factor tended to be stronger than individualistic countries such as Germany and United Kingdom. Within the current study, Singapore, a comparatively collectivistic country (Tan & Goh, 2006), yielded an ECV value of .09 higher than the Australia sample. While the ECV value of both countries reported in the current study (.85 for Singapore and .76 for Australia) are equal or higher than all the 38 countries reported (.40 to .76), the ECV value gap between Singapore (collectivistic) and Australia (individualistic) was observable in the current study.

Żemojtel-Piotrowska, Piotrowski [48] proposed two potential explanations for the differences in common variance captured by the GWB factor between collectivistic and individualistic countries. The current study concurs with the possibility of the first, but was able to dismiss the second. Firstly, Żemojtel-Piotrowska, Piotrowski [48] drawing upon the information provided by Cross, Bacon [77], posited that due to the interdependent self-construal found within collectivistic countries, social and psychological well-being are likely to become less distinct domains of well-being, further strengthening the contribution of the GWB factor. This is a potential explanation, seeing the limited contributions by the SWB and PWB factors found in the current study. However, that possibility would require further investigation, because the contribution of the SWB factor was found to be similar to that of the EWB factor within the current study. This result suggested that the interdependent self-construal found within collectivistic countries only affected the psychological domain of well-being and not the social domain.

Żemojtel-Piotrowska, Piotrowski [48], drawing upon the understanding put forth by Harzing [78], also offered a second plausible explanation, which explains that the differences in the strength of the GWB factor could be contributed to the stronger effects of acquiescence (i.e., compliance) found in collectivistic countries. They noted that as there were no reverse-scored items for the MHC-SF, individuals from collectivistic countries were hypothesised to be more likely to pick similar responses for all the questions, thus, contributing to the common factor variance. However, this explanation was not supported by the results of the current study. If it were to be true, the total MHC-SF score of the Singapore sample should see a significantly smaller variance compared to the Australia sample, however the total MHC-SF score variance reported by both samples were largely similar (Singapore [SD = 1.30] and Australia [SD = 1.40]).

With regard to the other psychometric properties of the MHC-SF, both samples were found to have excellent internal consistency. Across both samples, Cronbach’s α was above .94 for the total scale and above .87 for the three subscales, while the MacDonald’s ω reliability was above .96 for the GWB factor and above .90 for the three specific factors. Compared to prior studies (e.g., de Bruin & du Plessis, 2015; Keyes et al., 2008; Lamers et al., 2011; Żemojtel-Piotrowska et al., 2018), the current study reported equal or greater internal consistency reliability. These findings provided support that the items of MHC-SF, especially within the Singapore and Australia context, were reliably measuring the construct that they were designed to (i.e., scores of all the items on each of the subscale reported similar responses). However, the high Cronbach’s α (.94) and MacDonald’s ω (.96) of the total MHC-SF scale may not be necessarily desirable, as it may be indicative that some of the items are redundant [79]. Drawing back to the low factor loadings and insignificant contributions of unique variances by the PWB factor across both samples, it is highly possible that the undesired high reliability coefficients obtained and some of the redundant items may be from the PWB factor.

Concurrent validity of the MHC-SF was demonstrated via strong correlations (r ≥ .66) between the MHC-SF total scale and the three subscales with the WHO-5, which measures an individual’s current mental well-being based on positive mood, vitality, and general interest. Notably, the correlations for the GWB, EWB, and PWB scale scores across both samples were similar, while the correlations for the SWB scale were uniformly lower. This result is expected, as the items on the WHO-5 matches very closely to EWB followed by PWB, while no items were related to SWB.

To the best of our knowledge, criterion validity in the form of a singular self-reported question on state of mental health has not been investigated by prior studies. Results of the current study provided support to the criterion validity of the MHC-SF, as the MHC-SF total scale and the three subscales correlated moderately to strongly with a self-rated question on their state of mental health (r ≥ .52) across both samples. In addition, the correlations observed were lower than that with the WHO-5, highlighting that a singular broad question was not able to fully capture the various dimensions of mental health and the ability of the MHC-SF to measure the nuances of mental health.

Discriminant validity of the MHC-SF was demonstrated by the moderate to high negative correlations between the MHC-SF total scale and the three subscales with the two subscales of the HADS (r = -.45 to -.71) across both samples. The current results were higher than the previously reported correlations (r = -.33 to -.56) by Doré et al. (2017). However, considering the predictiveness of mental health on mental illness and the intimate relationship between the two [20–22], results of the current study still demonstrated discriminant validity. In addition, it also provided support to the two-continua model; that mental health and mental illness are distinct but highly correlated constructs, and that symptoms of psychopathology do not imply the presence or absence of mental well-being.

Limitations and further research

The current study only conducted the psychometric evaluation of the MHC-SF’s structural, concurrent, criterion, and discriminant validity. Other forms of validity such as multigroup invariance (e.g., dividing the groups between students and workers or by gender), known-group validity (e.g., comparing stressed and non-stressed samples), or predictive validity (e.g., development of a mental disorder) could have been included to significantly add to the strength of the findings.

In addition, samples of the current study might have not been representative of the respective country’s population. Due to the use of non-random sampling methods, convenience and snowball sampling, more than 70% of the participants were engaged with the study for course credit. In addition, the sampling venues (i.e., social media sites) may have further homogenised the results across the two countries despite the assumed collectivistic and individualistic differences. Hence, future evaluation studies of the MHC-SF should prioritise larger and more heterogenous samples, accounting for variables such as exposure to stressful events, socioeconomic, employment status, physical and mental health status, among others.

Including the current study, there have been numerous reports of the PWB factor contributing significantly lesser unique variances to the overall measure when compared to its counterparts, and its six items demonstrating weak factor loadings [e.g. 47, 48, 52]. Future research could investigate the theoretical dimensions of the PWB factor and explore whether they significantly overlap with the dimensions of the EWB and SWB factors, which could lead to the removing of redundant items or utilising the PWB subscale as a standalone instrument.

Conclusion

Results of the current study are congruent with prior studies showing that the MHC-SF is a reliable and valid measure of mental health.

The results of the current study conclusively reported that a bifactor model of MHC-SF (a GWB factor and three specific factors of EWB, SWB, and PWB) demonstrated good to excellent fit to the data, and was significantly better than any of the previously proposed model (i.e., one-, two-, and three-factor models).

Further investigations also identified that the EWB and SWB factors of MHC-SF contributed uniquely to the measure, while the PWB factor was similar to the GWB factor and had little to no unique contribution to the overall construct. Importantly, the results highlighted the strength of the GWB factor, which further lends its support to the existing aggregated scoring system of the MHC-SF utilised by researchers. Consequently, the use of the categorical diagnosis scoring system ought to be conducted with caution, as this approach requires interpretations of the subscale scores according to the hedonic or eudaimonic representation.

In sum, the current study provided strong supportive evidence that the MHC-SF demonstrates good psychometric properties within the Singapore and Australia context. Results also lend support to the unidimensionality of the MHC-SF and advocate for the utilisation of an aggregated scoring system over the categorical diagnosis. The MHC-SF appears to be a valid standardised measure, reliable, theory-driven, and comprehensive measure of mental health. Consequently, and because the MHC-SF has cross-cultural validity, the current study strongly encourages researchers, clinicians, and mental health professionals to start utilising the instrument and promoting awareness of mental health.

Citation: Yeo ZZ, Suárez L (2022) Validation of the mental health continuum-short form: The bifactor model of emotional, social, and psychological well-being. PLoS ONE 17(5): e0268232. https://doi.org/10.1371/journal.pone.0268232

About the Authors:

Zack Zhishen Yeo

Contributed equally to this work with: Zack Zhishen Yeo, Lidia Suárez

Roles: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Software, Validation, Visualization, Writing – original draft, Writing – review & editing

E-mail: [email protected]

Affiliation: Department of Psychology, School of Social and Health Sciences, James Cook University, Singapore, Singapore

https://orcid.org/0000-0002-6321-3316

Lidia Suárez

Contributed equally to this work with: Zack Zhishen Yeo, Lidia Suárez

Roles: Conceptualization, Formal analysis, Supervision, Validation, Writing – review & editing

Affiliation: Department of Psychology, School of Social and Health Sciences, James Cook University, Singapore, Singapore

https://orcid.org/0000-0001-5614-6509

References

1. GBD 2017 DALYs and HALE Collaborators. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. The Lancet. 2018;392(10159):1789–858.

2. GBD 2017 DALYs and HALE Collaborators. Global, regional, and national disability-adjusted life-years (DALYs) for 359 diseases and injuries and healthy life expectancy (HALE) for 195 countries and territories, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. The Lancet. 2018;392(10159):1859–922.

3. Brundtland GH. Mental health in the 21st century. Bulletin of the World Health Organization. 2000;78:411–. pmid:10885158

4. Fuhrer R, Keyes KM. Population Mental Health in the 21st Century: Aspirations and Experiences. American Journal of Public Health. 2019;109:S150–S1. pmid:31242004

5. Fuhrer R, Keyes KM. Population Mental Health in the 21st Century: Time to Act. American Journal of Public Health. 2019;109:S152–S3. pmid:31242014

6. Kieling C, Baker-Henningham H, Belfer M, Conti G, Ertem I, Omigbodun O, et al. Child and adolescent mental health worldwide: evidence for action. The Lancet. 2011;378(9801):1515–25. pmid:22008427

7. Kessler RC, Angermeyer M, Anthony JC, De Graaf R, Demyttenaere K, Gasquet I, et al. Lifetime prevalence and age-of-onset distributions of mental disorders in the World Health Organization’s World Mental Health Survey Initiative. World Psychiatry. 2007;6(3):168. pmid:18188442

8. Kim-Cohen J, Caspi A, Moffitt TE, Harrington H, Milne BJ, Poulton R. Prior juvenile diagnoses in adults with mental disorder: developmental follow-back of a prospective-longitudinal cohort. Archives of General Psychiatry. 2003;60(7):709–17. pmid:12860775

9. Storrie K, Ahern K, Tuckett A. A systematic review: students with mental health problems—a growing problem. International Journal of Nursing Practice. 2010;16(1):1–6. pmid:20158541

10. Eisenberg D, Downs MF, Golberstein E, Zivin K. Stigma and help seeking for mental health among college students. Medical Care Research and Review. 2009;66(5):522–41. pmid:19454625

11. Kitzrow MA. The mental health needs of today’s college students: Challenges and recommendations. Journal of Student Affairs Research and Practice. 2003;41(1):167–81.

12. Arnett JJ. Emerging adulthood: The winding road from the late teens through the twenties: Oxford University Press; 2014.

13. Arnett JJ. Emerging adulthood: A theory of development from the late teens through the twenties. American Psychologist. 2000;55(5):469. pmid:10842426

14. Shim S, Xiao JJ, Barber BL, Lyons AC. Pathways to life success: A conceptual model of financial well-being for young adults. Journal of Applied Developmental Psychology. 2009;30(6):708–23.

15. Cornwell EY, Waite LJ. Social disconnectedness, perceived isolation, and health among older adults. Journal of Health and Social Behavior. 2009;50(1):31–48. pmid:19413133

16. Fiori KL, Antonucci TC, Cortina KS. Social network typologies and mental health among older adults. The Journals of Gerontology Series B: Psychological Sciences and Social Sciences. 2006;61(1):P25–P32. pmid:16399938

17. Doré I, O’Loughlin JL, Sabiston CM, Fournier L. Psychometric evaluation of the Mental Health Continuum–Short Form in French Canadian young adults. The Canadian Journal of Psychiatry. 2017;62(4):286–94. pmid:28363262

18. Lamers SMA, Westerhof GJ, Bohlmeijer ET, ten Klooster PM, Keyes CLM. Evaluating the psychometric properties of the Mental Health Continuum-Short Form (MHC-SF). Journal of Clinical Psychology. 2011;67(1):99–110. pmid:20973032

19. Keyes CLM. Mental illness and/or mental health? Investigating axioms of the complete state model of health. Journal of Consulting and Clinical Psychology. 2005;73(3):539–48. pmid:15982151

20. Keyes CLM, Dhingra SS, Simoes EJ. Change in level of positive mental health as a predictor of future risk of mental illness. American Journal of Public Health. 2010;100(12):2366–71. pmid:20966364

21. Keyes CLM, Eisenberg D, Perry GS, Dube SR, Kroenke K, Dhingra SS. The relationship of level of positive mental health with current mental disorders in predicting suicidal behavior and academic impairment in college students. Journal of American College Health. 2012;60(2):126–33. pmid:22316409

22. Westerhof GJ, Keyes CLM. Mental illness and mental health: The two continua model across the lifespan. Journal of Adult Development. 2010;17(2):110–9. pmid:20502508

23. Keyes CLM. Mental health in adolescence: Is America’s youth flourishing? American Journal of Orthopsychiatry. 2006;76(3):395–402. pmid:16981819

24. Westerhof G, Keyes C. Geestelijke gezondheid is meer dan de afwezigheid van geestelijke ziekte [Mental health is more than the absence of mental illness]. Maandblad Geestelijke Volksgezondheid. 2008;63:808–20.

25. Keyes CLM, Wissing M, Potgieter JP, Temane M, Kruger A, Van Rooy S. Evaluation of the mental health continuum–short form (MHC–SF) in setswana‐speaking South Africans. Clinical Psychology & Psychotherapy. 2008;15(3):181–92. pmid:19115439

26. Perugini MLL, de la Iglesia G, Solano AC, Keyes CLM. The mental health continuum–short form (MHC–SF) in the Argentinean context: Confirmatory factor analysis and measurement invariance. Europe’s Journal of Psychology. 2017;13(1):93. pmid:28344677

27. World Health Organisation. Promoting mental health: Concepts, emerging evidence, practice. Geneva; 2005.

28. Deci EL, Ryan RM. Hedonia, eudaimonia, and well-being: an introduction. Journal of Happiness Studies. 2008;9(1):1–11.

29. Keyes CLM. Social well-being. Social Psychology Quarterly. 1998;61(2):121–40.

30. Ryff CD. Happiness is everything, or is it? Explorations on the meaning of psychological well-being. Journal of Personality and Social Psychology. 1989;57(6):1069–81.

31. Waterman AS. Two conceptions of happiness: Contrasts of personal expressiveness (eudaimonia) and hedonic enjoyment. Journal of Personality and Social Psychology. 1993;64(4):678–91.

32. Diener E. Subjective well-being: The science of happiness and a proposal for a national index. American Psychologist. 2000;55(1):34–43. pmid:11392863

33. Diener E, Suh E, Lucas R, Smith H. Subjective Well-Being: Three Decades of Progress. Psychological Bulletin. 1999;125:276–302.

34. Keyes CLM. The Mental Health Continuum: From languishing to flourishing in life. Journal of Health and Social Behavior. 2002;43(2):207–22. pmid:12096700

35. Keyes CLM. Promoting and protecting mental health as flourishing: A complementary strategy for improving national mental health. American Psychologist. 2007;62(2):95–108. pmid:17324035

36. World Health Organisation Quality of Life Group. The World Health Organization quality of life assessment (WHOQOL): development and general psychometric properties. Social Science & Medicine. 1998;46(12):1569–85. pmid:9672396

37. World Health Organisation Quality of Life Group. Development of the World Health Organization WHOQOL-BREF quality of life assessment. Psychological medicine. 1998;28(3):551–8. pmid:9626712

38. World Health Organization, editor Wellbeing measures in primary health care/the DEPCARE project 1998; Copenhagen: WHO Regional Office for Europe.

39. Watson D, Clark LA, Tellegen A. Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology. 1988;54(6):1063–70. pmid:3397865

40. Hu Y, Stewart-Brown S, Twigg L, Weich S. Can the 12-item General Health Questionnaire be used to measure positive mental health? Psychological Medicine. 2007;37(7):1005. pmid:17274855

41. Pavot W, Diener E. Review of the Satisfaction With Life Scale. Psychological Assessment. 1993;5(2):164–72.

42. Hyde M, Wiggins RD, Higgs P, Blane DB. A measure of quality of life in early old age: The theory, development and properties of a needs satisfaction model (CASP-19). Aging & Mental Health. 2003;7(3):186–94.

43. Keyes CLM, Grzywacz JG. Health as a complete state: The added value in work performance and healthcare costs. Journal of Occupational and Environmental Medicine. 2005;47(5):523–32. pmid:15891532

44. King DL, Haagsma MC, Delfabbro PH, Gradisar M, Griffiths MD. Toward a consensus definition of pathological video-gaming: A systematic review of psychometric assessment tools. Clinical Psychology Review. 2013;33(3):331–42. pmid:23396015

45. Galesic M, Bosnjak M. Effects of questionnaire length on participation and indicators of response quality in a web survey. Public Opinion Quarterly. 2009;73(2):349–60.

46. Bonacchi A, Chiesi F, Lau C, Marunic G, Saklofske DH, Marra F, et al. (2021) Rapid and sound assessment of well-being within a multidimensional approach: The Well-being Numerical Rating Scales (WB-NRSs). PLoS ONE 16(6): e0252709. pmid:34125831

47. de Bruin GP, du Plessis GA. Bifactor analysis of the Mental Health Continuum—Short Form (MHC—SF). Psychological Reports. 2015;116(2):438–46. pmid:25730745

48. Żemojtel‐Piotrowska M, Piotrowski JP, Osin EN, Cieciuch J, Adams BG, Ardi R, et al. The Mental Health Continuum‐Short Form: The structure and application for cross‐cultural studies–A 38 nation study. Journal of Clinical Psychology. 2018;74(6):1034–52. pmid:29380877

49. Guo C, Tomson G, Guo J, Li X, Keller C, Söderqvist F. Psychometric evaluation of the Mental Health Continuum-Short Form (MHC-SF) in Chinese adolescents–a methodological study. Health and Quality of Life Outcomes. 2015;13(1):198. pmid:26651829

50. Singh K, Bassi M, Junnarkar M, Negri L. Mental health and psychosocial functioning in adolescence: An investigation among Indian students from Delhi. Journal of Adolescence. 2015;39:59–69. pmid:25588610

51. Lt Hu, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal. 1999;6(1):1–55.

52. Jovanović V. Structural validity of the Mental Health Continuum-Short Form: The bifactor model of emotional, social and psychological well-being. Personality and Individual Differences. 2015;75:154–9.

53. McCain JL, Jonason PK, Foster JD, Campbell WK. The bifactor structure and the “dark nomological network” of the State Self-Esteem Scale. Personality and Individual Differences. 2015;72:1–6.

54. Brouwer D, Meijer RR, Zevalkink J. On the factor structure of the Beck Depression Inventory–II: G is the key. Psychological Assessment. 2013;25(1):136. pmid:22800089

55. Gignac GE, Watkins MW. Bifactor modeling and the estimation of model-based reliability in the WAIS-IV. Multivariate Behavioral Research. 2013;48(5):639–62. pmid:26741057

56. Reise SP. The rediscovery of bifactor measurement models. Multivariate Behavioral Research. 2012;47(5):667–96. pmid:24049214

57. Hides L, Quinn C, Stoyanov S, Cockshaw W, Mitchell T, Kavanagh DJ. Is the mental wellbeing of young Australians best represented by a single, multidimensional or bifactor model? Psychiatry Research. 2016;241:1–7. pmid:27152903

58. Petrillo G, Capone V, Caso D, Keyes CLM. The Mental Health Continuum-Short Form (MHC-SF) as a measure of well-being in the Italian context. Social Indicators Research. 2015;121(1):291–312.

59. Karaś D, Cieciuch J, Keyes CLM. The Polish adaptation of the Mental Health Continuum-Short Form (MHC-SF). Personality and Individual Differences. 2014;69:104–9.

60. Awata S, Bech P, Yoshida S, Hirai M, Suzuki S, Yamashita M, et al. Reliability and validity of the Japanese version of the world health organization‐five well‐being index in the context of detecting depression in diabetic patients. Psychiatry and Clinical Neurosciences. 2007;61(1):112–9. pmid:17239048

61. Löve J, Andersson L, Moore CD, Hensing G. Psychometric analysis of the Swedish translation of the WHO well-being index. Quality of Life Research. 2014;23(1):293–7. pmid:23743856

62. Blom EH, Bech P, Högberg G, Larsson JO, Serlachius E. Screening for depressed mood in an adolescent psychiatric context by brief self-assessment scales—testing psychometric validity of WHO-5 and BDI-6 indices by latent trait analyses. Health Qual Life Outcomes. 2012;10:149. pmid:23227908

63. Lucas-Carrasco R, Allerup P, Bech P. The Validity of the WHO-5 as an Early Screening for Apathy in an Elderly Population. Curr Gerontol Geriatr Res. 2012;2012:171857. pmid:22991511

64. Birket-Smith M, Hansen BH, Hanash JA, Hansen JF, Rasmussen A. Mental disorders and general well-being in cardiology outpatients—6-year survival. J Psychosom Res. 2009;67(1):5–10. pmid:19539812

65. Hajós TR, Pouwer F, Skovlund S, Den Oudsten BL, Geelhoed‐Duijvestijn P, Tack C, et al. Psychometric and screening properties of the WHO‐5 well‐being index in adult outpatients with Type 1 or Type 2 diabetes mellitus. Diabetic Medicine. 2013;30(2):e63–e9. pmid:23072401

66. Omani-Samani R, Maroufizadeh S, Almasi-Hashiani A, Sepidarkish M, Amini P. The WHO-5 Well-Being Index: A Validation Study in People with Infertility. Iranian Journal of Public Health. 2019;48(11):2058. pmid:31970105

67. Zigmond AS, Snaith RP. The Hospital Anxiety and Depression Scale. Acta Psychiatrica Scandinavica. 1983;67(6):361–70. pmid:6880820

68. Bjelland I, Dahl AA, Haug TT, Neckelmann D. The validity of the Hospital Anxiety and Depression Scale: an updated literature review. Journal of Psychosomatic Research. 2002;52(2):69–77. pmid:11832252

69. Roberge P, Doré I, Menear M, Chartrand É, Ciampi A, Duhoux A, et al. A psychometric evaluation of the French Canadian version of the Hospital Anxiety and Depression Scale in a large primary care population. Journal of Affective Disorders. 2013;147(1–3):171–9. pmid:23218249

70. Brown TA. Confirmatory factor analysis for applied research: Guilford Publications; 2015.

71. West SG, Taylor AB, Wu W. Model fit and model selection in structural equation modeling. In: Hoyle RH, editor. Handbook of structural equation modeling. New York: Guilford Press; 2012. p. 209–31.

72. Reise SP, Bonifay WE, Haviland MG. Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment. 2013;95(2):129–40. pmid:23030794

73. Rodriguez A, Reise SP, Haviland MG. Applying bifactor statistical indices in the evaluation of psychological measures. Journal of Personality Assessment. 2016;98(3):223–37. pmid:26514921

74. Viladrich C, Angulo-Brunet A, Doval E. A journey around alpha and omega to estimate internal consistency reliability. Annals of Psychology. 2017;33(3):755–82.

75. Cortina JM. What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology. 1993;78(1):98.

76. Kaplan RM, Saccuzzo DP. Psychological testing: Principles, applications, and issues: Nelson Education; 2017.

77. Cross SE, Bacon PL, Morris ML. The relational-interdependent self-construal and relationships. Journal of Personality and Social Psychology. 2000;78(4):791. pmid:10794381

78. Harzing A-W. Response styles in cross-national survey research: A 26-country study. International Journal of Cross Cultural Management. 2006;6(2):243–66.

79. Streiner DL. Starting at the beginning: an introduction to coefficient alpha and internal consistency. Journal of Personality Assessment. 2003;80(1):99–103. pmid:12584072