Everyday functioning as a predictor of cognitive status in a group of community‐dwelling, predominantly Black adults

Abstract

INTRODUCTION

We examined whether the Performance Assessment of Self‐Care Skills (PASS) and Everyday Cognition Scale‐12 (ECog‐12) dichotomized cognitive groups in a sample of predominantly Black adults.

METHODS

Two hundred forty‐six community‐dwelling adults (95% Black, age 50+) completed cognitive testing, the PASS, and the ECog. Cognitive groups (probable vs unlikely cognitive impairment) were determined by performance on the Modified Mini‐Mental State Examination. We examined the predictive validity of the PASS shopping, medication management, and information retrieval subtests and the ECog‐12 to dichotomize cognitive groups.

RESULTS

Performance on all PASS subtests (all p’s < .05) differed between cognitive groups, but not ECog‐12 (p = 0.17). Only the PASS shopping and medication management had good reliability for determining cognitive group (areas under the curve (AUCs) of .74 each).

DISCUSSION

PASS shopping and medication management exhibited adequate predictive validity when distinguished between cognitive status groups, whereas the PASS information retrieval and ECog‐12 did not.

Highlights

Mild functional decline is a core diagnostic criterion for cognitive impairment. Performance‐based assessments are a valuable tool for assessing functional decline. Most performance‐based measures were developed using homogenous samples. Few studies have validated these measures in other racial and ethnic populations.

INTRODUCTION

Accurate assessment of cognitive disorders in older adults is essential for dementia prevention and treatment efforts. Clinical trials for Alzheimer's disease (AD) and related dementias (ADRD) have moved toward targeting people with mild cognitive impairment (MCI), as they are at higher risk for developing dementia and still at a potentially treatable stage of disease. However, diagnostic bias by race has been demonstrated in large cohort studies.¹ Although some studies have reported a lower percentage of Black Americans receiving MCI and dementia diagnoses despite equivalent or worse cognitive functioning, neuropsychiatric symptoms, and functional abilities compared with White Americans,² others have reported higher diagnostic incidence rates.³ Diagnostic rates can also differ based on the diagnostic classification system⁴ or individual cognitive measure⁵ used. The resulting classification errors represent a critical problem for current research and health care standards.

A core distinguishing feature between MCI and dementia is the capacity to complete activities of daily living (ADLs) such as self-care or hygiene activities. People with MCI can independently perform these activities, as well as complex instrumental activities of daily living (IADLs) such as financial management, medication management, and shopping, although possibly with the use of compensatory strategies.^6,7 Given the vital role that the measurement of everyday functioning plays in dementia diagnosis, understanding how functional assessments operate in minoritized people is essential. Measurement bias may contribute to both under- and overdiagnosis of dementia in minoritized racial/ethnic groups, which in turn can have multiple downstream negative effects.

RESEARCH IN CONTEXT

• Systematic review: The authors reviewed the literature using traditional (e.g., PubMed) sources. Of the existing assessments of everyday functioning, the vast majority were developed and normed on primarily White, homogenous samples. Few studies have examined the clinical utility of these measures in other populations.

• Interpretation: We examined the predictive utility of an existing measure of everyday functioning, the Performance Assessment of Self-Care Skills (PASS), and found that it can accurately differentiate between cognitive status groups in a sample of community dwelling, predominantly Black adults. We also provided updated clinical cutoff scores for select PASS subtests.

• Future directions: This study examined the clinical utility of a single performance-based measure of everyday functioning. Future work is needed to evaluate how other widely used performance-based measures perform across different racial, ethnic, and cultural populations. In addition, as new measures are developed, researchers should take into consideration the role of cultural and socioeconomic factors in the performance of everyday tasks.

IADL capacity is frequently measured using self- and informant-rated questionnaires. In individuals with cognitive impairment, self-reports are susceptible to judgment bias in one's own functioning.⁸ Informant reports are generally thought to be more reliable in people with MCI and dementia. However, informant reports can also be biased based on the nature and quality of the patient–informant relationship as well as informant demographic and neuropsychiatric characteristics.^9,10 In addition, many older adults lack a trusted confidant who can accurately report on daily functioning, who also fits eligibility criteria and has the availability to participate in the research process.¹¹

Performance-based measures of IADL capacity can assess aspects of capacity different from those assessed by questionnaires, particularly the ability to apply problem-solving skills.¹² However, a major limitation of many performance-based functional assessments is that they were developed and validated on homogenous samples of predominantly White, highly educated participants or that they lack information about the demographic or cultural makeup of the normative sample.^13,14 A notable exception to this are computerized functional skills assessments such as the functional skills assessment and training (FUNSAT) program,¹⁵ which assesses technology-based functional tasks such as online banking, prescription refilling, and online shopping. However, not all older adults use technology to complete IADLs. The Performance Assessment of Self-Care Skills (PASS) was developed and validated by occupational therapists to measure subtle and overt breakdowns in functional performance using non-computerized tasks.¹⁶ It has good construct and discriminative ability to discern MCI from dementia, but has been applied mainly to White, urban, highly educated older adults.¹⁷ The goal of the current study is to assess the ability of the PASS compared with self-report of functional change to differentiate middle aged to older Black adults with and without likely cognitive impairment.

METHOD

Participants

Study participants were community-dwelling residents 50 years of age or older who were recruited from the Pittsburgh Hill/Homewood Research on Neighborhood Change and Health (PHRESH) study. PHRESH is an ongoing longitudinal study (2011 to present) following residents living in two low-income, historically Black neighborhoods in Pittsburgh, PA (USA). The primary aims are to examine the impact of the built and social environment on residents’ overall health.¹⁸ The “Think PHRESH” supplement (2019–2020) expanded upon the parent study to include a comprehensive neuropsychological evaluation in a subsample (N = 256) of the original PHRESH cohort, including those who were 50 years or older at the time of assessment.¹⁹ Think PHRESH assessments were conducted at local community centers or in participant homes. Participants provided written informed consent for study procedures.

Cognitive performance

Global cognitive status was measured using the Modified Mini-Mental Status Examination (3MS), a screening measure designed to assess a range of cognitive domains including attention, memory, language, and orientation, with higher scores indicating better cognitive functioning.²⁰ Although research diagnoses of cognitive status were conferred for each participant using National Institute on Aging–Alzheimer's Association (NIA-AA) diagnostic criteria^6,7 during multidisciplinary consensus conferences, these diagnoses were based on information from neuropsychological test battery performance and the performance of daily activities. Given the need for a categorization source independent of the predictor variables (capacity in ADLs), participants were grouped for the current analysis based on 3MS performance using a cutoff of 84.²¹ We chose to dichotomize the participants rather than to examine 3MS scores continuously so as to be in line with clinical considerations. A supplemental analysis used a conservative actuarial approach as outlined by Jak/Bondi criteria²² to group participants (see supplementary materials). Literacy was assessed using the Wide Range Achievement Test 3 (WRAT-3) reading subtest.²³

Performance of daily activities/outcome measures

IADL capacity was measured with both performance-based and self-reported measures.

Performance-based assessment

The PASS is a standardized, criterion-referenced, performance-based measure of capacity to complete 26 distinct activities in four functional domains: functional mobility, personal self-care, IADLs with a cognitive emphasis (C-IADL), and IADLs with a physical emphasis.¹⁶ The PASS was originally designed to assess the capacity of psychiatric patients to live independently¹⁶. The C-IADL domain has been subsequently validated in distinguishing people with normal cognition from people with MCI.¹⁷ A trained assessor observes participants complete each functional task, providing cued assistance as needed. The assessor then rates the participant's level of preclinical disability using a standardized scoring system. A higher number of cues required for task completion corresponds with worse functioning.

The current study used three of the C-IADL tasks that were determined previously to distinguish between persons without a cognitive disorder from those with a mild cognitive disorder: shopping, medication management, and critical information retrieval.¹⁷ Shopping consists of participants selecting items from a shopping list and paying for those items using coupons and cash. Medication management requires that participants read directions from two prescription bottles, indicate the next time they would take the medication, and then place the medication for the next 2 days in the proper compartments of a pill organizer. Critical information retrieval requires that participants read a newspaper article, summarize what it said, and indicate one thing they learned from the article that they could apply to their own situation. All subtasks were analyzed using the number of cues required for independence to be met upon completion of the task.¹⁷

Self-report assessments

The Everyday Cognition Scale - 12 (ECog-12) is a 12-item self- and informant-rated questionnaire designed to measure global everyday functional ability in older adults.²⁴ Participants rate the amount of change in functioning on a scale of 1 (better/no change) to 4 (consistently much worse) for each item; a higher mean score indicates worse cognitive and functional ability. Although both the self- and informant-rated scales were used in Think PHRESH, informants were not required for participation, resulting in 46% of the sample missing an informant report. Due to the high amount of missing data in informant reports, the current study included only self-report scores in analyses. ECog-12 scores were analyzed using a mean score of the answered items. A higher score indicates greater decline in everyday functional ability.

Analysis

Of the 256 participants included in the THINK PHRESH sample, 10 were excluded due to missing 3MS scores (n = 3), all outcome data missing (n = 2), environmental interference during testing (n = 1), developmental disability that impacted ability to obtain accurate cognitive assessment (n = 1), or data validity being deemed invalid due to low effort at the time of testing as determined by research staff (n = 3). Effort was determined at the time of testing by trained assessors who provided both a qualitative assessment of participant effort as well as a categorical ranking of data quality of either confident to use, use with caution, or do not use. Data from the remaining participants were examined for individual missing scores prior to running analyses. Multiple imputation using the MICE²⁵ and Random Forest²⁶ packages in R were used to impute missing WRAT-3 (n = 14), ECog-12 (n = 2), PASS shopping (n = 17), PASS medication (n = 8), and PASS critical information retrieval (n = 7) scores. Following imputation, all data were examined for normality and outliers. No data transformation was required.

Participant groups were defined using a cutoff score of 84 on the 3MS. Participants who scored above the cutpoint (≥84) were classified as unlikely to have cognitive impairment (UCI). Participants with a score at or below the cutpoint (≤83) were classified as having probable cognitive impairment (PCI). Demographic characteristics, PASS scores, and ECog-12 scores were compared using independent sample t-tests for continuous variables and chi-square tests for categorical variables. We did not have traditional self-report questionnaires of functional capacity beyond the ECog-12 and, therefore, were not able to compare PASS performance to traditional functional measures used in research. However, self- and informant-based measures of everyday functioning tap into complementary but different constructs and, in general, do not correlate strongly with one another.¹² Results were reported as means (and SDs) for continuous variables and frequencies (and percentages) for categorical variables.

Only measures of functional performance (i.e., PASS subtasks, ECog-12) that differentiated between UCI and PCI groups were included in further analyses. Generalized linear regression models were conducted to compare remaining functional performance scores (PASS shopping, medication management) for UCI and PCI groups. Models controlled for age, education, and literacy, which have been identified as important confounders of cognitive status in older adults.²⁷ The PHRESH studies were designed to compare changing neighborhood-based conditions in two neighborhoods, one of which has experienced more changes (Hill District) than the other (Homewood), so models also adjusted for neighborhood to incorporate study design.¹⁸ Functional measures that were found to differentiate between groups after controlling for these variables (PASS shopping and medication management) were aggregated to determine if combining tests improved accuracy over administering an individual measure alone.

Receiver-operating characteristic (ROC) analyses were conducted to determine diagnostic utility of the individual PASS subtests and aggregated score.²⁸ The Youden index²⁹ was used to determine optimal cutoff scores on the PASS items to differentiate between the UCI and PCI groups. We report the area under the curve (AUC) of the ROC analyses. The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the optimal cutoff scores were also calculated for the individual and aggregate capacity measures to assess diagnostic accuracy. Sensitivity refers to the percentage of participants with the condition that (correctly) test positive, whereas specificity refers to the percentage of participants without the condition that (correctly) test negative. The PPV refers to the percentage of positive results that are true positives, whereas the NPV refers to the percentage of negative results that are true negatives.²⁸ Analyses were conducted in R version 4.1.0³⁰ The same analytic approach was applied in a follow-up analysis using an actuarial approach to cognitive categorization (supplementary materials).

RESULTS

A sample of 256 participants 50 years of age or older from the parent PHRESH cohort completed assessments for the Think PHRESH supplement; 246 were included in the current analyses. There were no significant differences in age, education, race, gender, or neighborhood between the participants who were included in the analyses and those who were excluded.

Compared to the UCI group, the PCI group was on average older, had less formal education, and had lower WRAT3-Reading Z-scores (Table 1). No differences in self-reported race or gender were found with more than 90% of both groups self-identifying as Black and more than 80% self-identifying as female. On average, both the PCI and UCI groups reported elevated levels of subjective cognitive complaints on the ECog-12.

TABLE 1 Participant demographics and functional outcome performance.

	Total sample	Unlikely cognitive impairment (3MS ≥84)	Possible cognitive impairment (3MS ≤83)
	N = 246 (M ± SD)	N = 188 (M ± SD)	N = 58 (M ± SD)	P-value	T	d
Demographics
Age (range 51–90)	66.47 ± 9.34	65.70 ± 9.19	68.97 ± 9.49	0.02	−2.31	−0.35
Education in years (range 5–20)	12.59 ± 2.23	12.96 ± 2.18	11.36 ± 1.96	> 0.01	5.29	0.75
WRAT-3 Reading Z-score (range −2.79 to 2.62)	−0.01 ± 0.97	0.17 ± 0.90	−0.59 ± 0.95	> 0.01	5.36	0.83
Modified Mini-Mental Status Examination (3MS) score, (range 54–100)	87 ± 8.7	90.88 ± 3.94	74.47 ± 8.00	> 0.01	15.08	3.17
						ω
Race, % Black, n (%)	234 (95.1%)	177 (94.1%)	57 (98.2%)	0.35	–	0.08
Gender, % female, n (%)	204 (82.9%)	156 (82.9%)	48 (82.8%)	1.00	–	0.002
Neighborhood, % Hill district, n (%)	166 (67.4%)	121 (64.3%)	45 (77.6%)	0.09	–	0.12
Married/partnered, % Partnered, n (%)	33 (13.4%)	29 (15.4%)	4 (7%)	0.15	–	0.11
Functional outcome measures
					Z	β
ECog-12, averaged score	1.49 ± 0.42	1.47 ± 0.41	1.56 ± 0.45	0.17	1.36	0.23
Shopping, no. cues	5.62 ± 5.42	4.51 ± 3.79	9.20 ± 7.87	0.04	2.09	0.49
Medication Management, no. cues	3.69 ± 3.92	2.84 ± 2.81	6.50 ± 5.47	> 0.01	2.77	0.57
Critical information retrieval, no. cues	0.37 ± 1.07	0.27 ± 0.84	0.72 ± 1.58	0.13	1.51	0.29
2 C-IADL, no. cues	9.32 ± 8.49	7.35 ± 5.58	15.71 ± 12.4	> 0.01	2.73	0.66

Group performance on PASS subtests was compared. Participants from both the UCI and PCI groups required relatively little assistance (i.e., few cues) to complete the PASS critical information retrieval subtask, with no significant difference between the UCI and PCI groups on the amount of assistance needed (Table 1). Compared to the UCI group, the PCI group required significantly more assistance on PASS shopping and medication management. The same pattern held when examining participants using the actuarial approach to cognitive grouping (Table S1).

The PASS shopping and medication management tasks were next examined, separately and aggregated, for differentiation between the UCI and PCI groups. The ECog-12 and PASS critical information task were not examined because scores on these measures did not differ significantly between groups. Table 2 shows the optimal cutoff scores, sensitivity, specificity, PPV, NPV, and AUC values for PASS shopping, medication management, and the 2 C-IADL model. Figure 1 shows the range of sensitivity and specificity for the two PASS tasks and the 2 C-IADL model in the form of ROC curves. The high NPV at the optimal cutoff (89%–98% of people classified as UCI are UCI) and low PPV (only ≈35% of people classified as PCI are PCI) indicate that the PASS shopping and medication management subtasks and the combined 2-C-IADL model are most effective at ruling-out cognitive impairment for participants whose cognitive performance is on the border between normal and mildly impaired, rather than ruling in cognitive impairment for participants with borderline scores. The same findings held when examining participants using the actuarial approach to cognitive grouping (Table S2; Figure S1).

TABLE 2 Performance Assessment of Self-care Skills (PASS) classification functions.

	Shopping	Medication management	2 C-IADL
Optimal cutoff, no. cues	4	3	6
Sensitivity	0.91	0.79	0.97
Specificity	0.47	0.56	0.44
PPV	0.35	0.36	0.35
NPV	0.95	0.89	0.98
AUC	0.74	0.74	0.76
AUC 95% CIs	0.67–0.81	0.67–0.81	0.69–0.83

[IMAGE OMITTED. SEE PDF]

DISCUSSION

This study examined the ability of the PASS and ECog-12 to distinguish between older adults with and without likely cognitive impairment in a sample of predominantly Black community-dwelling older adults. Unexpectedly, we found no significant differences between UCI and PCI groups in the level of self-reported functional decline or on PASS critical information retrieval performance. As expected, the PCI group performed worse on PASS shopping and medication management. AUC values were acceptable for the PASS shopping and medication management subtests³¹ and comparable to those reported previously, although the optimal cutoff scores between UCI and PCI were higher in the current sample than in a predominantly White, highly educated sample.¹⁷ When considering AUC confidence intervals (CIs), shopping, medication, and the combined 2 C-IADL model yielded borderline to acceptable sensitivity and NPV, making them most effective as rule-out tests for cognitive impairment rather than rule-in tests. The inability of the critical information subtask to differentiate between groups is likely due to floor effects, as all participants generally required no more than one cue to complete the task.

The inability of the self-report ECog-12 to differentiate between the UCI and PCI groups is likely due, at least in part, to both groups reporting mildly elevated levels of cognitive complaints.^24,32 When measuring subjective cognitive complaints in Black Americans, the associations with cognitive performance are not always as strong as the associations in White Americans.³³ This may reflect different lifestyle factors contributing to the salience of the ECog-12 questions. The question “Compared to 10 years ago, has there been any change in balancing the checkbook without error” is not relevant to an individual who does not use a checkbook or have a checking account. Similar findings were reported using the full ECog-39 in Black and Hispanic individuals.³² In addition, the current study sample comprises a much more socioeconomically disadvantaged population, including lower income, access to resources, and formal education, than participants in typical convenience or clinical samples. As such, the higher ECog-12 scores may be reflecting potentially lower cognitive reserve to compensate for normal cognitive aging effects as compared to typical clinical research participants.

Even when using informant ratings, distinguishing between cognitively normal participants and those with MCI is less robust than when differentiating between cognitively normal participants and those with dementia.²⁴ Many participants here (46%) did not have an informant they felt comfortable providing to the study team, likely due to the large number of unmarried/unpartnered participants. The requirement of an informant in many large AD clinical trials and dementia studies generally may be a barrier to efforts to diversify clinical trial enrollment and increase retention.^34–38 Black and Hispanic individuals are more likely to enroll in research with a non-spousal informant,³⁷ but non-spousal informants, such as adult children, often have additional occupational and childcare responsibilities and less availability for research activities.

Cognitive screening tools such as the Montreal Cognitive Assessment (MoCA) and 3MS are especially useful for detecting dementia, but struggle more with detecting mild impairments.³⁹ Although adjusting cutoff scores for demographic confounds can improve the sensitivity of cognitive screeners, many times these adjusted scores are determined in highly selective samples and are not broadly representative of community samples.⁴⁰ Both cognitive ability and functional ability are necessary to consider for diagnostic purposes, but there are varying degrees of correlation between cognitive screeners and functional assessments. Yu and colleagues found no association between MMSE or MoCA scores and functional performance in a group of older adults.⁴¹ In contrast, Lee and colleagues found that self-report of medication management and shopping ability was useful in discriminating between normal control participants and participants with MCI or dementia, arguing that functional assessment should be an integral part of cognitive assessment.⁴² However, cognitive screeners alone are not always sufficient for determining cognitive status, despite being regularly used in large-scale research studies,⁴³ and may only explain upwards of 20% of the variance in functional performance.⁴⁴

Performance-based measures have traditionally been created and validated on homogenous White samples.^13,14 Understanding the bias of a measure is imperative because single tools may not operate the same across different cultural or racial groups. This effort to understand psychometric properties across groups is emerging for performance-based measures, as seen by the Observed Test of Daily Living (R-OTDL) measure⁴⁵ and Everyday Problems Test (EPT),⁴⁶ as well as newly developed computerized-based measures that examine IADL via digital literacy.¹⁵ Our results add to this body of literature by demonstrating the utility of the PASS shopping and medication management subtasks in a primarily Black sample. Combined into a 2 C-IADL model, the PASS subtasks take approximately 15–25 min to administer, and our results indicate that the PASS shopping and medication subtasks can rule out cognitive impairment. This rule-out ability is especially important in people who may have lower average cognitive scores and may be susceptible to overdiagnosis.

AD research and clinical practice in the United States has a history of centering on White European-American culture despite Black Americans being disproportionately impacted by AD and other dementias.³ Black American adults have higher morbidity and mortality rates and show earlier brain aging effects than White adults do, likely resulting from the impact of systemic racism on overall health.⁴⁷ Many of the measures used in gold-standard assessment of dementia were developed with White, highly educated, Western participants as the normative basis. However, these measures may not only be inappropriate for a broader population, but harmful to minoritized groups generally and Black Americans specifically.^47,48 Calls to update AD research and clinical practices have been met with slow progress and often without sound theoretical approaches.^34,37 Given this historical context, it is imperative for dementia assessment to evolve beyond assuming that established normative tests operate similarly across all cultural groups.

There are several limitations in the current study. First, we used the 3MS to determine cognitive groups rather than adjudicated research diagnoses. This was because performance on the PASS and ECog-12 ratings were used in the adjudication process and, therefore, could not also be used as predictive variables of diagnosis. However, we did find the same pattern of results when using an actuarial approach to cognitive categorization based on neuropsychological tests. Second, although establishing the ability of the PASS to differentiate between UCI and PCI groups provides one piece of evidence for the validity of the PASS, we lacked the data needed to assess whether PASS performance correlates with additional IADL measures or predicts subsequent development of functional performance. Future work will examine the ability of the PASS to predict cognitive decline in this sample. Third, although a strength of this study lies in the examination of a socioeconomic group that is typically excluded from research, the restricted economic range of participants meant that we were unable to compare PASS performance across a range of socio-economic status. In addition, the current study does not address potential cultural relevance within the PASS subtests. Finally, most of this sample was female, due to the original PHRESH cohort recruitment methods focusing on the household primary food shopper.

Despite these limitations, the current study has several notable strengths. Many AD studies rely on convenience samples drawn from memory clinics that are not necessarily representative of the general population or applicable to minoritized people.^34,37 Participants in the present study were community dwelling older adults who were primarily tested in community centers and their homes. This allowed for the inclusion of participants who may be otherwise unable or unwilling to travel to an academic medical setting, thereby reducing sampling bias. The American Academy of Clinical Neuropsychology Relevance 2050 Initiative calls for a focus within the field on developing assessment methods for non-European Americans. The Think PHRESH sample consists of participants significantly under-represented in aging and dementia research, and a comprehensive assessment of the larger sample is ongoing.⁴⁹ In addition, our examination of self-report measures and performance-based measures of function improves generalizability beyond informant-rated metrics.

In summary, we found that the PASS medication management and shopping subtasks could distinguish between cognitive groups and be used effectively as cognitive impairment rule-out tests in a group of Black, community-dwelling adults. Future work is needed to evaluate how other widely used measures of capacity perform across racial, ethnic, and cultural populations. Most importantly, as new measures of functional capacity are developed, it is essential to ensure accurate and equitable measurement for those at risk of cognitive decline and disability.

ACKNOWLEDGMENTS

The authors would like to express their sincere appreciation to the THINK PHRESH participants for their contributions to this work. The authors would also like to thank La'Vette Wagner, Michelle Zmuda, and Isaac DeLozier for their coordination of this study as well as all study staff who assisted with the data collection. Funding was provided by the National Institute of Aging (R01AG072652), National Heart, Lung, and Blood Institute (HL131531-03S1), and the National Cancer Institute (CA149105-09S). A.W. is supported by the National Institute of Aging (K23AG076663). No funding sources had a role in study design, data collection, analysis, interpretation, in the writing of this article, or in the decision to submit the article for publication.

CONFLICT OF INTEREST STATEMENT

No authors have any conflicts of interest to report. Author disclosures are available in the supporting information.

CONSENT STATEMENT

Written informed consent was obtained from all participants or their authorized representatives for their participation in this study.

References

Barnes LL. Alzheimer disease in African American individuals: increased incidence or not enough data? Nat Rev Neurol. 2022;18(1):56‐62. doi: [DOI: https://dx.doi.org/10.1038/s41582-021-00589-3]