Chinese version of Montreal Cognitive Assessment Basic for discrimination among different severities of Alzheimer’s disease

Abstract

Objectives: To find out whether the Chinese version of Montreal Cognitive Assessment Basic (MoCA-BC) and its subtests could be applied in discrimination among cognitively normal controls (NC), mild cognitive impairment (MCI), mild and moderate Alzheimer’s Disease (AD), and furthermore, to determine the optimal cutoffs most sensitive to distinguish between them.

Design: A cross-sectional validation study.

Setting: Huashan Hospital, Shanghai, China.

Participants: There was a total of 1,969 participants: individuals with MCI (n=663), mild (n=345), moderate (n=441) AD, and cognitively NC (n=520) were recruited from the Memory Clinic, Huashan Hospital, Shanghai, China.

Measurements: Baseline MoCA-BC scores were collected from firsthand data. Two subtests were calculated from MoCA-BC: the Memory Index Score of MoCA-BC (MoCA-BC-MIS) and the Non-memory Index Score of MoCA-BC (MoCA-BC-NM).

Results: MoCA-BC was an effective cognitive tool to discriminate among NC, MCI, mild and moderate AD in the Chinese elderly across all education groups, implying that it was efficient not only for detecting MCI, but for different severities of AD as well. For MCI screening, the total score of MoCA-BC (MoCA-BC-T) and MoCA-BC-MIS had similar high sensitivity and specificity. For discrimination among MCI, mild and moderate AD, the MoCA-BC-T and MoCA-BC-NM had similar performance.

Conclusion: MoCA-BC is an effective cognitive test to distinguish between NC, MCI, mild and moderate AD among the Chinese elderly with various levels of education.

Background

Montreal Cognitive Assessment (MoCA) has been widely used as a screening test for detection of mild cognitive impairment (MCI) in the highly educated elderly with an average education level of 13 years.^1–4 A revised MoCA test called the MoCA basic (MoCA-B) was devised to screen for MCI in a group of individuals with little education.⁵

However, the application of MoCA to discriminate patients with MCI and Alzheimer's Disease (AD) showed wide differences in thresholds due to cultural differences across regions. Comparisons between healthy controls and subjects with MCI or AD conducted in several international validation studies reported diverse cutoff points, sensitivity and specificity values. For instance, Bosco et al⁶ reported a minimum cutoff of 17 to detect MCI in Italy, while Ng et al⁷ as well as Tu et al⁸ suggested a maximum cutoff of 27 appropriate for Singaporean and Chinese. As for AD detection, cutoff points were also demonstrated differently ranging from a minimum of 14⁶ to a maximum of 26⁹ all over the world. The Chinese version of the MoCA-B (MOCA-BC), translated with cultural modifications from the original English version, has shown excellent validity and accuracy in screening for MCI in the Chinese elderly.^9–11 Since education level had been found to affect MoCA scores,^2,12 the optimal cutoff points of the MOCA-BC for screening for MCI were developed based on it as well.¹³ Our previous study determined the optimal cutoff scores for MCI screening among different education groups, and recognized the MoCA-BC as a more sensitive and accurate scale in screening for MCI compared to the mini-mental state examination (MMSE).¹⁰

In addition, epidemiological studies have found that MoCA was superior to MMSE as a brief and feasible assessment tool, particularly in discerning earlier stages of cognitive decline.¹⁴ To save time and avoid duplicate tests, the MOCA-BC had been observed to be frequently used in clinics in China as a replacement of MMSE due to convenience and higher screening efficiency.¹⁵ The MoCA-BC, given its sensitivity to non-memory cognitive domains such as executive function,^5,10 should be better suited for detecting more severe disease states. However, no relevant research has been conducted concerning MoCA-BC in staging dementia severity. The present study aimed to find out whether MoCA-BC or its subtests could be applied as a preliminary screening method to distinguish among outpatients with MCI, mild and moderate AD as a substitute for MMSE.

Many studies have shown that patients with MCI, who subsequently declined to AD, were first impaired in delayed recall memory.^16–18 In the early phase of MCI, executive functions were preserved as a compensation for hippocampal dysfunction.^19,20 However, with the progression of the disease, the frontal executive networks were also involved^1,20 and caused encoding memory deficit that did not improve with cueing.²⁰ The Memory Index Score of MoCA-BC (MoCA-BC-MIS) was calculated by adding the number of words remembered in free delayed recall, category-cued recall, and multiple choice-cued recall multiplied by 3, 2, and 1, respectively, with a score ranging from 0 to 15. This scoring method was designed to better detect the encoding memory deficit.²¹ The Non-memory Index Score of MoCA-BC (MoCA-BC-NM) was calculated by adding the score of orientation, calculation, animal naming, and verbal fluency (the number of fruits listed), with a score ranging from 0 to 15 likewise. The latter subtest was a novel creation for the present study and was devised to describe cognitive domains other than episodic memory.

Given the importance and popularity of MoCA-BC for clinical application, this study was aimed at exploring the capacity and examining the accuracy of MoCA-BC and its two subtests (MoCA-BC-MIS and MoCA-BC-NM) in discrimination of NC, MCI, mild and moderate AD.

Participants and methods

Participants

A total of 1,969 individuals were recruited: 520 cognitively normal controls (NC), 663 individuals with MCI, 345 individuals with mild AD, and 441 individuals with moderate AD. The participants with MCI or AD were recruited from the Memory Clinic, Huashan Hospital, Shanghai, China, from May 2015 to August 2017. NC were recruited from their spouses and friends. The descriptive statistics are reported in Table S1. Inclusion criteria for all were aged 50–90, nearly normal eyesight and hearing, no history of alcoholism, drug abuse, head trauma or other serious neuropsychiatric diseases that would affect their performance. Written consent was obtained from participants or their legally authorized caregivers. This study has been approved by the ethics committee of Huashan Hospital.

Materials and procedure

All subjects received complete neurological and neuropsychological assessments, brain imaging, and other necessary laboratory tests. The diagnosis decision was made independent of MoCA-BC scores.

MoCA-BC had been translated from the original version with subtle linguistic and cultural modifications, reviewed, and approved by the original author (Dr Ziad Nasreddine). MoCA-BC is free of charge, online for clinical use (www.mocatest.org, Basic section), assessing nine cognitive domains including language, attention, calculation, orientation, memory, concentration, executive function, conceptual thinking, and visual perception.

The total score of MoCA-BC (MoCA-BC-T), the MoCA-BC-MIS, and the MoCA-BC-NM scores were all calculated from MoCA-BC as aforesaid.

Diagnosis of MCI was made as follows:¹⁷ complaints of cognitive impairments; normal activities of daily living or slight impairment in instrumental activities of daily living; objective cognitive impairment (MMSE) score ≥ cutoff,²² Clinical Dementia Rating (CDR) score of 0.5,²³ and performance on a set of neuropsychological tests 1.5 SDs or more below the normative mean; no dementia.

Diagnosis of AD was based on the National Institute of Aging and Alzheimer's Association criteria:²⁴ insidious onset of symptoms, history of cognitive decline by observation, and the initial and most prominent cognitive deficits being typically amnestic or non-amnestic. Diagnosis of mild and moderate AD was based on CDR scores: a CDR score of 1 for mild AD, two for moderate AD.²³

Moreover, NC should have no memory complaints, no significant impairment in activities of daily living, MMSE score at or above the cutoff,²² a CDR score of 0,²³ a Modified Hachinski Ischemic Scale (MHIS) score of 4 or less,²⁵ and a Hamilton Depression Rating Scale (HDRS) score of 12 or less in the past 2 weeks.²⁶

Five examiners conducted all the assessments in the neuropsychological laboratory of Huashan Hospital, including MMSE,²² MoCA-BC, CDR,²³ MHIS,²⁵ and HDRS²⁶ as mentioned previously. Individuals were tested one at a time by blinded raters.

Statistical analysis

We used chi-square and one-way ANOVA to assess possible group differences between the four groups (NC, MCI, mild and moderate AD) in demographic characteristics and cognitive test performance. Pearson correlation coefficients (r) between MoCA-BC and MMSE were computed to measure criterion-related validity. Receiver operating characteristic (ROC)²⁷ curves were carried out to determine the ability of MoCA-BC, MoCA-BC-MIS, and MoCA-BC-NM to distinguish among NC, MCI, mild and moderate AD. The method of comparing the AUCs derived from the same sample of patients was from Hanley and Mcneil²⁷ (by Medcalc).²⁷ Analyses were all conducted using SPSS version 20.0 (IBM Corporation, Armonk, NY, USA) and Medcalc version 11.4.2.0 (Medcalc Software, bvba).

Results

Demographic information and performance of MoCA-BC and MMSE

The demographic characteristics of the four groups (NC, MCI, mild and moderate AD) are summarized in Table S1. The level of significance was set at α=0.05. Differences in significance were found in education, not in age. Correlation analysis revealed that MoCA-BC-T scores were strongly associated with education (r=0.484, P<0.001), not age (r=−0.069, P=0.116). In order to reduce education bias, subjects were divided into three education subgroups according to years of formal education: low-level education (≤6 years), mid-level education (7–12 years), and high-level education (>12 years). The demographic characteristics and scores of MMSE, MoCA-BC-T, MoCA-BC-MIS, and MoCA-BC-NM of the subgroups are shown in Table S2, with no significant difference in age or education.

Psychometric properties of MoCA-BC

Correlation coefficient between MoCA-BC and MMSE was 0.817 (P<0.001), implying high criterion-related validity. Among all the NC, only three subjects (0.58%) obtained the maximum MoCA-BC-T score, indicating no obvious ceiling effect. Among individuals with MCI or AD, only one subject obtained a MoCA-BC-T score of 0 (0.07%), indicating no obvious floor effect.

Comparison of MoCA-BC and its subtests for discrimination among NC, MCI, mild and moderate AD

For detecting MCI, shown by ROC analysis, MoCA-BC-T and MoCA-BC-MIS had similar high sensitivity and specificity with AUCs of more than 0.80 in the three education subgroups, suggesting that the predictive information captured by MoCA-BC-T and MoCA-BC-MIS was reasonably good regardless of education level, superior to MoCA-BC-NM. The optimal MoCA-BC-T cutoffs were 19 in the low-level education group, 22 in the mid-level education group, and 24 in the high-level education group respectively, consistent with our previous study.¹⁰ However, a MoCA-BC-MIS cutoff score of 8 was proposed for individuals across all levels of education (Table S3).

For discrimination among MCI, mild and moderate AD, MoCA-BC-T and MoCA-BC-NM had high sensitivity and specificity, with AUCs of around 0.80 in the three education subgroups, superior to MoCA-BC-MIS.

For mild AD, a MoCA-BC-T cutoff of 13 was suggested for individuals with low-level education, 15 for those with mid-level education, and 16 for high-level education. Comparatively, a MoCA-BC-NM cutoff of 9 was proposed for individuals in the low-level education group, and 10 for those in the mid- and high-level education groups.

As for moderate AD, a MoCA-BC-T cutoff score of 10 was proposed for individuals with 6 or fewer years of education, 11 for those with 7–12 years, and 13 for those with more than 12 years. Likewise, a MoCA-BC-NM cutoff score of 7 was proposed for individuals in the low-level education group, and 8 for those in the mid- and high-level education groups.

In general, the AUCs of MoCA-BC-MIS for detecting MCI were notably higher than that of MoCA-BC-NM (P<0.001), indicating that the ability of the memory-related subtest of MoCA-BC to screen for MCI was significantly better than the non-memory part of it. The results confirmed that delayed recall memory was the first domain to be impaired in individuals with MCI as mentioned previously. As to discrimination among MCI, mild and moderate AD, MoCA-BC-MIS performed worse than MoCA-BC-NM with lower AUCs (P<0.001), probably owing to floor effects.

Discussion

Our study was aimed at establishing the value of MoCA-BC for discrimination among different severities of AD and defining education cutoff scores of MoCA-BC and its subtests, among groups of NC, MCI, mild and moderate AD. The study revealed that MoCA-BC had the ability to determine AD severity with cutoffs based on different education levels.

Since education had been found to affect MoCA scores in previous literature^4,13,28 and has been considered the strongest non-cognitive factor influencing performance on MoCA,^3,22,28 the optimal cutoffs for MoCA-BC were calculated based on it. Thus, the best cutoffs of NC, MCI, mild and moderate AD were clarified in three different education level groups in this study to reduce the education bias. No significant difference of age or education was found per subgroup.

The corresponding education cutoffs of MoCA-BC and its subtests to best distinguish AD of different severities are summarized in Table 1. The optimal cutoff scores of MoCA-BC for screening for MCI and AD based on education level in the present study were similar to that found in Italy,⁶ lower than most other versions according to previous international studies.^7–9 The cutoffs of the aforementioned validation studies ranged from 17–27 for MCI detection and 14–26 for AD detection,^6–9 indicating huge regional differences in cognitive performance.

Table 1 Best MoCA-BC subtests and the corresponding cutoff scores recommended for discrimination between normal controls, mild cognitive impairment, mild and moderate Alzheimer's disease Abbreviations: MoCA-BC, Chinese version of Montreal Cognitive Assessment Basic; NC, cognitively normal controls; MCI, mild cognitive impairment; AD, Alzheimer's disease; MoCA-BC-T, total score of MoCA-BC; MoCA-BC-MIS, the Memory Index Score of MoCA-BC; MoCA-BC-NM, the Non-memory Index Score of MoCA-BC.

Our results presented large AUCs of MoCA-BC-T for detecting MCI and determining severity of AD in all education groups, indicating validity not only in MCI, but also in mild and moderate AD. In addition, MoCA-BC-T education cutoffs for MCI were in accordance with our previous study,¹⁰ which further confirmed our finding in a larger sample.

The best cutoff scores of MoCA-BC-MIS and MoCA-BC-NM in different education groups were almost the same in this study. This may be due to several reasons. Firstly, MoCA-BC-MIS was calculated by adding the number of words remembered, which might be, rarely, affected by education level. Secondly, MoCA-BC-NM was calculated by adding the scores of orientation, calculation, animal naming, and verbal fluency. The average years of education of the mid- and high-level education groups were all above 10 years, which might have led to small effects on the ability of calculation, orientation, and animal naming.

Since MoCA-BC is a quick, simple, and feasible assessment tool for outpatients, superior to the MMSE with less time needed and higher efficiency for screening for MCI,^1,5,19,29 it is of great importance for clinical practice to verify MoCA-BC as a capable test to help discriminate the severity of AD instead of MMSE. Our research concluded that MoCA-BC was a highly reliable and sensitive tool to not only screen for MCI, but also distinguish between mild and moderate AD, which had never been reported before. It also determined the optimal cutoffs most sensitive for discrimination. The results revealed that MoCA-BC could be applied in clinical work as a quick and sensitive screening tool to distinguish different severities of AD.

Some limitations of the present study should be noted when utilizing its results. This study was a cross-sectional observational study. Selection bias in our sample may have affected the results. The data of our research were mainly based on urban residents of Shanghai due to lack of participants from rural areas and other cities. As China is such a huge geographically and culturally pluralistic nation, specific regional characteristics are likely to have impacted the performance of these neuropsychological tests. Encouragingly, the sample size was very large, with 1,969 participants, with a minimum of 345 individuals in each group, and all individuals received the same tests over a timespan of 2 years. Future studies should be conducted recruiting more subjects from diverse regions of the country and ethnic minorities as well.

Conclusion

The study verified that MoCA-BC was a valid cognitive test to determine AD severity in the Chinese elderly and defined cutoff scores of NC, MCI, mild and moderate AD based on education levels. Regarding screening for MCI, MoCA-BC-T and MoCA-BC-MIS were found to have similar high sensitivity and specificity, while for discrimination among MCI, mild and moderate AD, MoCA-BC-T and MoCA-BC-NM would perform better. The results suggested that MoCA-BC could be applied in clinical work not only to screen for MCI, but also to distinguish different severities of AD.

Data sharing statement

The datasets analyzed during the current study are available from the corresponding author upon reasonable request.

Acknowledgments

This work was supported by a grant from National Key R&D Program of China to Qi-Hao Guo (2016YFC1306305). The sponsor had no role in the design, methods, subject recruitment, data collections, analyses, and preparation of the paper. Dr Ziad Nasreddine is the copyright owner of the MoCA test. Qi-Hao Guo obtained permission to translate the MoCA-B from Dr Nasreddine.

Author contributions

Qi-Hao Guo and Ying-Ru Lv were involved with the conception and design of the study. Lin Huang, Ke-Liang Chen, Bi-Ying Lin, Le Tang, and Qian-Hua Zhao contributed to the data collection. Lin Huang, Qi-Hao Guo, and Ying-Ru Lv were responsible for statistical analysis. Lin Huang was involved in drafting of the manuscript. Qi-Hao Guo and Ying-Ru Lv were responsible for critical revision and finalizing of the manuscript. All authors contributed toward data analysis, drafting and critically revising the paper and agree to be accountable for all aspects of the work.

Disclosure

The authors report no conflicts of interest in this work.

Supplementary materials

Table S1 Demographic information and mean scores of MoCA-BC and MMSE Note: *NC vs MCI, NC vs mild AD, NC vs moderate AD, MCI vs mild AD, MCI vs moderate AD, mild AD vs moderate AD. Abbreviations: MoCA-BC, Chinese version of Montreal Cognitive Assessment Basic; MMSE, mini-mental state examination; NC, cognitively normal controls; MCI, mild cognitive impairment; AD, Alzheimer's disease; MoCA-BC-T, total score of MoCA-BC; MoCA-BC-MIS, the Memory Index Score of MoCA-BC; MoCA-BC-NM, the Non-memory Index Score of MoCA-BC; sig, significant.

Table S2 Demographic characteristics and mean scores according to level of education Note: NC vs MCI, NC vs mild AD, NC vs moderate AD, MCI vs mild AD, MCI vs moderate AD, mild AD vs moderate AD. Abbreviations: MMSE, mini-mental state examination; MoCA-BC, Chinese version of Montreal Cognitive Assessment Basic; NC, cognitively normal controls; MCI, mild cognitive impairment; AD, Alzheimer's disease; MoCA-BC-T, total score of MoCA-BC; MoCA-BC-MIS, the Memory Index Score of MoCA-BC; MoCA-BC-NM, the Non-memory Index Score of MoCA-BC.

Table S3 MoCA-BC and its subtests to discriminate among NC, MCI, mild and moderate AD Abbreviations: MoCA-BC, Chinese version of Montreal Cognitive Assessment Basic; NC, cognitively normal controls; MCI, mild cognitive impairment; AD, Alzheimer's disease; MoCA-BC-T, total score of MoCA-BC; MoCA-BC-MIS, the Memory Index Score of MoCA-BC; MoCA-BC-NM, the Non-memory Index Score of MoCA-BC; AUC, area under the curve.

References

1. Freitas S, Simões MR, Alves L, Santana I. Montreal cognitive assessment: validation study for mild cognitive impairment and Alzheimer disease. Alzheimer Dis Assoc Disord. 2013;27(1):37–43.

2. Freitas S, Simões MR, Alves L, Santana I. Montreal Cognitive Assessment: influence of sociodemographic and health variables. Arch Clin Neuropsychol. 2012;27(2):165–175.

3. Lu J, Li D, Li F, et al. Montreal cognitive assessment in detecting cognitive impairment in Chinese elderly individuals: a population-based study. J Geriatr Psychiatry Neurol. 2011;24(4):184–190.

4. Nasreddine ZS, Phillips NA, Bédirian V, et al. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc. 2005;53(4):695–699.

5. Julayanont P, Tangwongchai S, Hemrungrojn S, et al. The Montreal Cognitive Assessment-Basic: A Screening Tool for Mild Cognitive Impairment in Illiterate and Low-Educated Elderly Adults. J Am Geriatr Soc. 2015;63(12):2550–2554.

6. Bosco A, Spano G, Caffò AO, et al. Italians do it worse. Montreal Cognitive Assessment (MoCA) optimal cut-off scores for people with probable Alzheimer’s disease and with probable cognitive impairment. Aging Clin Exp Res. 2017;29(6):1113–1120.

7. Ng A, Chew I, Narasimhalu K, Kandiah N. Effectiveness of Montreal Cognitive Assessment for the diagnosis of mild cognitive impairment and mild Alzheimer’s disease in Singapore. Singap Med J. 2013;54(11).

8. Tu QY, Jin H, Ding BR, et al. Reliability, validity, and optimal cutoff score of the Montreal Cognitive Assessment (Changsha version) in ischemic cerebrovascular disease patients of Hunan Province, China. Dement Geriatr Cogn Dis Extra. 2013;3(1):25–36.

9. Hu JB, Zhou WH, Hu SH, Jb H, Sh H, et al. Cross-cultural difference and validation of the Chinese version of Montreal Cognitive Assessment in older adults residing in Eastern China: preliminary findings. Arch Gerontol Geriatr. 2013;56(1):38–43.

10. Chen KL, Xu Y, Chu AQ, et al. Validation of the Chinese Version of Montreal Cognitive Assessment Basic for Screening Mild Cognitive Impairment. J Am Geriatr Soc. 2016;64(12):e285–e290.

11. Zhou S, Zhu J, Zhang N, et al. The influence of education on Chinese version of Montreal cognitive assessment in detecting amnesic mild cognitive impairment among older people in a Beijing rural community. ScientificWorldJournal. 2014;2014:689456–7.

12. Santangelo G, Siciliano M, Pedone R, et al. Normative data for the Montreal Cognitive Assessment in an Italian population sample. Neurol Sci. 2015;36(4):585–591.

13. Wong A, Law LS, Liu W, et al. Montreal Cognitive Assessment: One Cutoff Never Fits All. Stroke. 2015;46(12):3547–3550.

14. Roalf DR, Moberg PJ, Xie SX, Wolk DA, Moelter ST, Arnold SE. Comparative accuracies of two common screening instruments for classification of Alzheimer’s disease, mild cognitive impairment, and healthy aging. Alzheimers Dement. 2013;9(5):529–537.

15. Chu LW, Ng KH, Law AC, Lee AM, Kwan F. Validity of the Cantonese Chinese Montreal Cognitive Assessment in Southern Chinese. Geriatr Gerontol Int. 2015;15(1):96–103.

16. Cecato JF, Martinelli JE, Izbicki R, Yassuda MS, Aprahamian I. A subtest analysis of the Montreal cognitive assessment (MoCA): which subtests can best discriminate between healthy controls, mild cognitive impairment and Alzheimer’s disease? Int Psychogeriatr. 2016;28(5):825–832.

17. Petersen RC, Smith GE, Waring SC, Ivnik RJ, Tangalos EG, Kokmen E. Mild cognitive impairment: clinical characterization and outcome. Arch Neurol. 1999;56(3):303–308.

18. Zhao Q, Lv Y, Zhou Y, Hong Z, Guo Q. Short-term delayed recall of auditory verbal learning test is equivalent to long-term delayed recall for identifying amnestic mild cognitive impairment. PLoS One. 2012;7(12):e51157.

19. Belleville S, Fouquet C, Hudon C, Zomahoun HTV, Croteau J, Consortium for the Early Identification of Alzheimer’s disease-Quebec. Neuropsychological Measures that Predict Progression from Mild Cognitive Impairment to Alzheimer’s type dementia in Older Adults: a Systematic Review and Meta-Analysis. Neuropsychol Rev. 2017;27(4):328–353.

20. Zhao Q, Guo Q, Liang X, et al. Auditory Verbal Learning Test is Superior to Rey-Osterrieth Complex Figure Memory for Predicting Mild Cognitive Impairment to Alzheimer’s Disease. Curr Alzheimer Res. 2015;12(6):520–526.

21. Julayanont P, Brousseau M, Chertkow H, Phillips N, Nasreddine ZS. Montreal Cognitive Assessment Memory Index Score (MoCA-MIS) as a predictor of conversion from mild cognitive impairment to Alzheimer’s disease. J Am Geriatr Soc. 2014;62(4):679–684.

22. Zhang MY, Katzman R, Salmon D, et al. The prevalence of dementia and Alzheimer’s disease in Shanghai, China: impact of age, gender, and education. Ann Neurol. 1990;27(4):428–437.

23. Morris JC. The Clinical Dementia Rating (CDR): current version and scoring rules. Neurology. 1993;43(11):2412–2414.

24. Jack CR, Albert MS, Knopman DS, et al. Introduction to the recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 2011;7(3):257–262.

25. Hachinski V, Oveisgharan S, Romney AK, Shankle WR. Optimizing the Hachinski Ischemic Scale. Arch Neurol. 2012;69(2):169–175.

26. Worboys M. The Hamilton Rating Scale for Depression: The making of a “gold standard” and the unmaking of a chronic illness, 1960–1980. Chronic Illn. 2013;9(3):202–219.

27. Hanley JA, Mcneil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology. 1983;148(3):839–843.

28. Zheng L, Teng EL, Varma R, et al. Chinese-language montreal cognitive assessment for cantonese or mandarin speakers: age, education, and gender effects. Int J Alzheimers Dis. 2012;2012:204623–10.

29. Matallana D, de Santacruz C, Cano C, et al. The relationship between education level and mini-mental state examination domains among older Mexican Americans. J Geriatr Psychiatry Neurol. 2011;24(1):9–18.