Comparative diagnostic accuracy of ACE-III and MoCA for detecting mild cognitive impairment

Abstract

Objective: The aim of this study was to validate the reliability of the Chinese version of Addenbrooke’s Cognitive Examination III (ACE-III) for detecting mild cognitive impairment. Furthermore, the present study compares the diagnostic accuracy of ACE-III with that of Montreal Cognitive Assessment (MoCA).

Methods: One hundred and twenty patients with MCI and 136 healthy controls were included in the study. All patients were evaluated by the Chinese version of ACE-III, MoCA and MMSE.

Results: Subjects in the control group showed better performance in ACE-III total score and its subdomain scores than those in the MCI group. There was a significantly positive correlation between ACE-III total score and MoCA score. Meanwhile, there was also a significantly positive correlation between ACE-III total score and MMSE score. For ACE-III total score, a cut-off point of 85 yielded a sensitivity of 97.3% and a specificity of 90.7%. The AUC for ACE-III total score was 0.978. For MoCA, a cut-off point of 23 yielded a sensitivity of 86.5% and a specificity of 97.7%. The AUC for MoCA was 0.961. There were no significant differences in diagnostic accuracy between ACE-III and MoCA.

Conclusion: The present findings support that both ACE-III and MoCA are useful for detecting MCI in early stages.

Introduction

Mild cognitive impairment (MCI), which is considered as a transitional phase between healthy aging and dementia, refers to impairment in cognition above that which is seen in normal aging, but not severe enough to impair daily function.¹ There is an annual progression rate of 5–15% to dementia in patients with MCI than that in the general population.^2,3 On the other hand, studies have found a reversible rate of 30–50% to normal cognitive function in patients with MCI.⁴ The dichotomy between progression rate to dementia and reversible rate to normal suggests that early interventions play a pivotal role in preventing the progress of MCI. Hence, it is important to screen and detect MCI in early stage.

Cognitive assessment by appropriate screening test is central to the diagnosis of MCI. The Mini-Mental State Examination (MMSE) is the most commonly used test in clinical practice. However, MMSE has weaknesses, such as lack of episodic and semantic memory and executive functioning tasks, existence of ceiling and floor effects. Hence, MMSE has limitations in screening multiple cognitive domains and detecting MCI.

The Addenbrooke's Cognitive Examination III (ACE-III) was reported as a screening tool to detect cognition.⁵ ACE-III involves five cognitive domains: measuring attention/orientation (18 points), memory (26 points), fluency (14 points), language (26 points), and visuospatial function (16 points)–100 in total. Higher scores indicate higher levels of cognitive function. ACE-III has been translated into several languages.^6–8 The Chinese version of ACE-III has been proved to be a reliable assessment tool for dementia.⁹ The ACE-III total score and its subdomain scores make it useful for detecting MCI.¹⁰ Relatively few studies have assessed the utility of ACE-III for diagnosing MCI.^11–14 To the best of our knowledge, there has been no report about the utility of the Chinese version of ACE-III in the diagnosis of MCI.

The Montreal Cognitive Assessment (MoCA) are widely used for screening MCI in China and other countries.¹⁵ We suspected that the ACE-III may be effective to establish a cognitive profile and for the assessment of MCI. The purpose of this study was to compare the diagnostic ability of the Chinese version of ACE-III and MoCA in patients with MCI and controls. Furthermore, we assessed the correlation between ACE-III and MoCA.

Materials and methods Study design

This is a prospective, single-center, cross-sectional study. The present study was approved by the ethics committee of Geriatric Hospital of Nanjing Medical University. All the participants are governmental staffs. All participants gave their written informed consent to participate in the study. The present study was conducted in accordance with the Declaration of Helsinki.

Study sample

All participants were recruited from the memory disorders clinic and the physical examination center of Geriatric Hospital of Nanjing Medical University during the study period from August 2016 to July 2017. They were divided into two groups: MCI (n=120) and healthy control (n=136). All patients with MCI fulfilled the Pertersen criteria for MCI. Participants in control group were volunteers who fulfilled the following inclusion criteria: (a) no neurological or systemic diseases potentially affecting cognitive function; (b) no current psychiatric disorders; (c) no history of abusing alcohol or other substances; (d) absence of memory or other cognitive complaints; (e) absence of visual, motor, or auditory limitations impairing the administration of the test.

Neuropsychological assessment

In all participants, cognition was evaluated by the Chinese version of ACE-III, MoCA, and MMSE. In addition, the following instruments were used: Clinical Dementia Rating (CDR) and Activities of Daily Living Scale (ADL). CDR was used to determine the severity of cognition impairment. ADL was used to determine activities of daily living. Items overlapping across the different cognitive tests (for example, drawing a clock or specific questions regarding orientation) were only administered once. In order to prevent biases, all the rating scales were performed by two physicians of neurology, who are blinded to grouping. In the two physicians of neurology, one is in charge of MoCA, MMSE, CDR, and ADL. The other is in charge of ACE-III.

We have introduced the translation and adaptation of ACE-III from English to Chinese in previous study.⁹ Addenbrooke's Cognitive Examination III includes five domains, each reflecting a specific cognitive function and contributing equally to the total score. The total score is 100 points, which is allocated to the five cognitive domains separately as follows: 18 points for attention and orientation, 26 points for memory, 14 points for fluency, 26 points for language, and 16 points for visuospatial abilities. A higher score indicates better cognitive function. Some items of the original ACE-III were adapted for better understanding during administration within the Chinese population. We replaced "lemon" with "orange" in the item of three words for registration and recall. The name and address in anterograde memory, recall, and recognition was replaced by a Chinese name and address. The last question in retrograde memory was replaced by "who is the only female emperor in Chinese history?" We asked subjects to speak as many words as possible with the Chinese character "che" during 1 min instead of the letter "P" in the item for verbal fluency. All the words and sentences were replaced by Chinese phrases in the repetition item. The comprehension part of the domain "language" has four questions. The first one was replaced by "which thing is related to monarchy?" The second question was revised into "which animal lives in Australia?" The third question was revised into "which animal lives in south pole?" The fourth question was revised into "which thing is used for parking a boat?" Finally, we asked the patients to read out eight Chinese characters instead of five English words in the reading item.

Statistical analysis

The statistical analysis was conducted using SPSS Statistics 16.0. The chi-squared test and one-way ANOVA were carried out to determine the statistical significance in demographics, MMSE score, MoCA total score, MoCA delayed recall score, ACE-III total score, and its subdomain scores between the two groups. A two-tailed Pearson's correlation analysis was performed among multiple neuropsychological tests. Receiver operating characteristic (ROC) curves and areas under the curves (AUC) were estimated to evaluate the discriminating capacity of neuropsychological tests between the MCI and the control group. The method proposed by DeLong et al was used to compare the ROC curves of each test.¹⁶ Youden J index was used to estimate the optimal cutoff points.¹⁷

Results Clinical and demographic and profile

In total, 256 participants were included in the study. They were divided into two groups: MCI (n=120) and healthy control (n=136). Table 1 displays clinical and demographic characteristics in the MCI group and the control group. As shown in Table 1, there were no significant differences in sex between the MCI group and the MCI group. There were significant differences in age, education level, MMSE score, MoCA total score, MoCA delayed recall score, ACE-III total score, and ACE-III subdomain scores. The average age of patients in the MCI group is about 7.5 years old than that of subjects in the control group. The average education level of patients in the MCI group is about 4.0 years more than that of subjects in the control group. Patients in the MCI group had lower MMSE score, lower MoCA total score, lower MoCA delayed recall score than subjects in the control group. As expected, subjects in the control group showed better performance in ACE-III total score and its subdomain scores than those in the MCI group.

Table 1 Clinical and demographic characteristics (mean ± SD)

Correlation between ACE-III and MoCA

A two-tailed Pearson's correlation analysis was performed among ACE-III total, ACE-III subdomain scores, MoCA total score, MoCA delayed recall score, and MMSE score. There was a significantly positive correlation between ACE-III total score and MoCA score (rs=0.794, two-tailed, p<0.05). It suggested that ACE-III total score increased when MoCA total score increased (Table 2). Meanwhile, there was also a significantly positive correlation between ACE-III total score and MMSE score (rs=0.759, two-tailed, p<0.05). It suggested that ACE-III total score increased when MMSE score increased (Table 2).

Table 2 Correlations between tests

ACE-III and MoCA: discriminant ability between groups

In order to assess the discriminant ability of ACE-III between control group and MCI group, we estimated the ROC curves for ACE-III. Sensitivity and specificity were calculated with a ROC curve. As shown in Table 3, for ACE-III total score, a cut-off point of 85 yielded a sensitivity of 97.3% and a specificity of 90.7%. The AUC for ACE-III total score was 0.978. For MoCA, a cut-off point of 23 yielded a sensitivity of 86.5% and a specificity of 97.7%. The AUC for MoCA was 0.961. For MMSE, a cut-off point of 28 yielded a sensitivity of 78.4% and a specificity of 86.0%.

Table 3 Sensitivity, specificity, optimal cutoff point, and Area Under ROC Curve (AUC) of tests

Further, the diagnostic accuracy of ACE-III was compared with that of MoCA and MMSE by Area Under ROC Curve (AUC). As shown in Figure 1, the AUC for ACE-III (0.980) and MoCA (0.965) was superior than that for MMSE (0.891) (ACE-II vs MMSE, p<0.05；MoCA vs MMSE, p<0.05). It suggested that both ACE-III and MoCA had higher diagnostic accuracy in detecting MCI than MMSE. The AUC for ACE-III total score was larger than that for MoCA, but there were no significant differences between the two tests (p>0.05). Furthermore, there were no significant differences in pairwise comparison of ROC curves among ACE-III subdomain scores. There were no statistically significant differences in pairwise comparison of ROC curves between ACE-III memory score and MoCA delayed recall score (p>0.05).

Figure 1 Receiver operating characteristic (ROC) curve of tests.

Discussion

The aim of the present study was to compare the diagnostic ability of the Chinese version of ACE-III and MoCA in patients with MCI and controls. The results showed that the Chinese version of ACE-III had a good correlation with MoCA and excellent discriminant ability for MCI.

In the present study, the patients in the MCI group got higher score than healthy control in MMSE score, MoCA total score, MoCA delayed recall score, ACE-III total score, and its subdomain scores. It suggested that MMSE, MoCA, and ACE-III could differentiate MCI from healthy control.

The present study suggested that the accuracy of ACE-III was equivalent to that of MoCA. Our results showed that the AUC of ACE-III was larger than that of MoCA, but there were no significant differences between the two tests. Another study also demonstrated a high diagnostic accuracy of ACE-III for distinguishing MCI from controls.¹⁸ Matías-Guiu et al detected significant differences between the AUC of ACE-III and MoCA, which suggested that ACE-III was more accurate than MoCA in screening for AD.¹⁹ It may be ascribed to the different participants and sample size. Further study were needed to compare the diagnostic accuracy of ACE-III and MoCA in screening for MCI.

The present study suggested that both ACE-III and MoCA had acceptable sensitivity and specificity in differentiating MCI from control subjects. In other words, both ACE-III and MoCA were effective in screening for MCI, which confirmed the usefulness of ACE-III and MoCA in previous studies.^18,20,21 Compared to ACE-III, MoCA achieved higher sensitivity and lower specificity in the present study. In order to detect MCI in early stage, higher sensitivity was more important. Therefore, MoCA might be more suitable for screening for MCI than ACE-III. However, a recently small-size study suggested that the Portuguese version of ACE-III held sensitivity and specificity values higher than MoCA in all the domains.²² Hence, further studies were needed to compare sensitivity/specificity of MoCA and ACE-III.

In our study, the cut-off point of ACE-III for MCI was 85, which was higher than that (82) in previous study.²² It may be ascribed to the longer education years in the present study. On the other hand, the cut-off point of MoCA for MCI was 23. A meta-analysis in a sample of 9350 suggested that the best cut-off point of MoCA for MCI among people aged over 60 was 24/25, with the sensitivity of 80.48% and specificity of 81.19%.²³ In the present study, we failed to detect significant differences in diagnostic ability among ACE-III subdomain scores. Meanwhile, there were no significant differences between ACE-III memory score and MoCA delayed recall score.

Our results suggested that ACE-III total score and its subdomain scores were significantly positively correlated with MoCA. What is more, the correlation coefficient between ACE-III total score and MoCA total score was higher than that between ACE-III total score and MMSE score. Our results were in accordance with previous report.²¹

There are some limitations of the present study. First, all participants were assessed in a single session. It may produce interference between multiple scales, especially in the memory tasks. Second, the present study was carried out in an official hospital and the participants were all governmental staffs with a relatively high education level. Hence, further studies about Chinese version of ACE-III should be performed in other population with relatively lower education level.

Conclusion

In summary, the present study demonstrates that both ACE-III and MoCA are useful for accurate detection of MCI in elderly individuals. Furthermore, they support using Chinese version of ACE-III as both a screening tool and a means of describing cognitive profiles.

Acknowledgments

The study was funded by National Natural Science Foundation of China (81500916), Youth Medical Talents Program of "Science and Education Strong Health Project" of Jiangsu Province (QNRC2016079), Medical Innovation Team Program of "Science and Education Strong Health Project" of Jiangsu Province (CXTDA2017030) and Cardre's Health Care Project of Jiangsu Province (BJ18027).

Author contributions

BRW, HFZ, YDZ, and JQS participated in study design and data interpretation. YDZ and JQS played key roles in the development of all components of the manuscript. BRW and JQS drafted the manuscript. HFZ and YDZ revised the manuscript. CX and YS participated in data acquisition, data interpretation, and manuscript drafting. All authors gave final approval of the version to be published. All authors agreed to be accountable for all aspects of the work.

Disclosure

Dr Bian-Rong Wang reports grants from Cardre's Health Care Project of Jiangsu Province (BJ18027), during the conduct of the study. Dr Ying-Dong Zhang reports grants from Medical Innovation Team Program of "Science and Education Strong Health Project" of Jiangsu Province (CXTDA2017030), during the conduct of the study. Dr Jian-Quan Shi reports grants from National Natural Science Foundation of China (81500916), Youth Medical Talents Program of "Science and Education Strong Health Project" of Jiangsu Province (QNRC2016079) and Medical Innovation Team Program of "Science and Education Strong Health Project" of Jiangsu Province (CXTDA2017030), during the conduct of the study. The authors report no other conflicts of interest in this work.

References

1. Petersen RC, Doody R, Kurz A, et al. Current concepts in mild cognitive impairment. Arch Neurol. 2001;58(12):1985–1992. doi:10.1001/archneur.58.12.1985

2. Petersen RC. Mild cognitive impairment as a diagnostic entity. J Intern Med. 2004;256(3):183–194. doi:10.1111/j.1365-2796.2004.01388.x

3. Panza F, D’Introno A, Colacicco AW, et al. Current epidemiology of mild cognitive impairment and other predementia syndromes. Am J Geriatr Psychiatry. 2005;13(8):633–644. doi:10.1176/appi.ajgp.13.8.633

4. Sanford AM. Mild cognitive impairment. Clin Geriatr Med. 2017;33(3):325–337. doi:10.1016/j.cger.2017.02.005

5. Mathuranath PS, Nestor PJ, Berrios GE, Rakowicz W, Hodges JR. A brief cognitive test battery to differentiate Alzheimer’s™ disease and frontotemporal dementia. Neurology. 2000;55(11):1613–1620. doi:10.1212/01.wnl.0000434309.85312.19

6. Hsieh S, Schubert S, Hoon C, Mioshi E, Hodges JR. Validation of the Addenbrooke’s cognitive examination III in frontotemporal dementia and Alzheimer’s disease. Dement Geriatr Cogn Disord. 2013;36(3–4):242–250. doi:10.1159/000351671

7. Matias-Guiu JA, Fernández de Bobadilla R, Escudero G, et al. Validation of the Spanish version of Addenbrooke’s cognitive examination III for diagnosing dementia. Neurologia. 2015;30(9):545–551. doi:10.1016/j.nrl.2014.05.004

8. Matías-Guiu JA, Fernández-Bobadilla R, Cortés-Martínez A. Addenbrooke’s cognitive examination III: a neuropsychological test useful to screen and obtain a cognitive profile. Neurologia. 2018;33(2):140. doi:10.1016/j.nrl.2016.06.014

9. Wang BR, Ou Z, Gu XH, Wei CS, Xu J, Shi JQ. Validation of the Chinese version of Addenbrooke’s Cognitive Examination III for diagnosing dementia. Int J Geriatr Psychiatry. 2017;32(12):e173–e179. doi:10.1002/gps.4680

10. Matías-Guiu JA, Fernández-Bobadilla R, Fernández-Oliveira A, et al. Normative data for the Spanish version of the Addenbrooke’s cognitive examination III. Dement Geriatr Cogn Disord. 2016;41(5–6):243–250. doi:10.1159/000445799

11. Crawford S, Whitnall L, Robertson J, Evans JJ. A systematic review of the accuracy and clinical utility of the Addenbrooke’s cognitive examination and the Addenbrooke’s cognitive examination-revised in the diagnosis of dementia. Int J Geriatr Psychiatry. 2012;27(7):659–669. doi:10.1002/gps.2771

12. Wong L, Chan C, Leung J, et al. A validation study of the Chinese-Cantonese Addenbrooke’s cognitive examination revised (C-ACER). Neuropsychiatr Dis Treat. 2013;9:731–737. doi:10.2147/NDT.S45477

13. Larner A, Mitchell AJ. A meta-analysis of the accuracy of the Addenbrooke’s cognitive examination (ACE) and the Addenbrooke’s cognitive examination-revised (ACE-R) in the detection of dementia. Int Psychogeriatr. 2014;26(4):555–563. doi:10.1017/S1041610213002329

14. Menon R, Lekha V, Justus S, Sarma P, Mathuranath P. A pilot study on utility of Malayalam version of Addenbrooke’s cognitive examination in detection of amnestic mild cognitive impairment: a critical insight into utility of learning and recall measures. Ann Indian Acad Neurol. 2014;17(4):420–425. doi:10.4103/0972-2327.144018

15. Nasreddine ZS, Phillips NA, Bédirian V, et al. The montreal cognitive assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc. 2005;53(4):695–699. doi:10.1111/j.1532-5415.2005.53221.x

16. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1998;44(3):837–845. doi:10.2307/2531595

17. Larner AJ. Optimising the cutoffs of cognitive screening instruments in pragmatic diagnostic accuracy studies: maximizing accuracy or the Youden index? Dement Geriatr Cogn Disord. 2015;39(3–4):167–175. doi:10.1159/000369883

18. Matias-Guiu JA, Cortés-Martínez A, Valles-Salgado M, et al. Addenbrooke’s cognitive examination III: diagnostic utility for mild cognitive impairment and dementia and correlation with standardized neuropsychological tests. Int Psychogeriatr. 2017;29(1):105–113. doi:10.1017/S1041610216001496

19. Matías-Guiu JA, Valles-Salgado M, Rognoni T, Hamre-Gil F, Moreno-Ramos T, Matías-Guiu J. Comparative diagnostic accuracy of the ACE-III, MIS, MMSE, MoCA, and RUDAS for screening of Alzheimer disease. Dement Geriatr Cogn Disord. 2017;43(5–6):237–246. doi:10.1159/000469658

20. Roalf DR, Moberg PJ, Xie SX, Wolk DA, Moelter ST, Arnold SE. Comparative accuracies of two common screening instruments for the classification of Alzheimer’s disease, mild cognitive impairment and healthy aging. Alzheimers Dement. 2013;9(5):529–537. doi:10.1016/j.jalz.2012.10.001

21. Aguilar-Navarro SG, Mimenza-Alvarado AJ, Palacios-García AA, Samudio-Cruz A, Gutiérrez-Gutiérrez LA, Ávila-Funes JA. Validity and reliability of the Spanish version of the montreal cognitive assessment (MoCA) for the detection of cognitive impairment in Mexico. Rev Colomb Psiquiatr. 2018;47(4):237–243. doi:10.1016/j.rcp.2017.05.003

22. Peixoto B, Machado M, Rocha P, et al. Validation of the Portuguese version of Addenbrooke’s Cognitive Examination III in mild cognitive impairment and dementia. Adv Clin Exp Med. 2018;27(6):781–786. doi:10.17219/acem/68975

23. Ciesielska N, Sokołowski R, Mazur E, Podhorecka M, Polak-Szabela A, Kędziora-Kornatowska K. Is the montreal cognitive assessment (MoCA) test better suited than the mini-mental state examination (MMSE) in mild cognitive impairment (MCI) detection among people aged over 60? meta-analysis. Psychiatr Pol. 2016;50(5):1039–1052. doi:10.12740/PP/45368