Building clinically relevant outcomes across the Alzheimer's disease spectrum

Abstract

Demonstrating that treatments are clinically meaningful across the Alzheimer's disease (AD) continuum is critical for meeting our goals of accelerating a cure by 2025. While this topic has been a focus of several Alzheimer's Association Research Roundtable (AARR) meetings, there remains no consensus as to what constitutes a “clinically meaningful outcome” in the eyes of patients, clinicians, care partners, policymakers, payers, and regulatory bodies. Furthermore, the field has not come to agreement as to what constitutes a clinically meaningful treatment effect at each stage of disease severity. The AARR meeting on November 19–20, 2019, reviewed current approaches to defining clinical meaningfulness from various perspectives including those of patients and care partners, clinicians, regulators, health economists, and public policymakers. Participants discussed approaches that may confer clinical relevance at each stage of the disease continuum and fostered discussion about what should guide us in the future.

INTRODUCTION

Defining what constitutes a clinically meaningful treatment response in Alzheimer's disease (AD) was the topic of a recent Alzheimer's Association's Research Roundtable (AARR). The AARR convened leaders from industry and academia as well as patients, care partners, clinicians, regulators, payers, and health economists to discuss the topic of “Building Clinically Relevant Outcomes.” Undoubtedly, all stakeholders agreed that a therapeutic response which produced a clear and sustainable benefit, while altering the disease trajectory, constitutes a clinically meaningful outcome. However, there has been little consensus as to how and when this is achieved across all stages of disease. With knowledge that the pathophysiological changes in AD start 15 to 20 years before the emergence of overt targeted symptoms,¹ a redefinition of what is a clinically meaningful treatment response is needed. Here we report on the proceedings of the AARR meeting that grappled with the definition of clinical meaningfulness across all stages of AD including asymptomatic, prodromal, and dementia. At each stage, we discussed: (1) To whom is the outcome relevant? (2) How is clinical meaningfulness demonstrated? and (3) When is clinical meaningfulness achieved?

PERSPECTIVES ON CLINICAL MEANINGFULNESS: TO WHOM IS THE OUTCOME RELEVANT The patient, care partner, and clinician

From the patient, care partner, and clinician perspective, a clinically meaningful treatment would vary across different stages of disease. For example, in early asymptomatic and prodromal stages, clinical meaningfulness would be achieved by disease-modifying treatments that delay or slow disease progression and by helping individuals maintain their current lifestyle. In dementia stages, a clinically meaningful outcome would be a treatment that targets symptoms of cognitive/functional decline, manages behavior, and delays institutionalization.²

Roundtable participants also discussed whether the result of a diagnostic test can be clinically meaningful. There is no question that providing a patient and family with a firm diagnosis of cognitive impairment would meet any definition for clinical meaningfulness, but what about information regarding the etiologic basis of that cognitive impairment? For example, would greater diagnostic certainty provided by amyloid positron emission tomography (PET) imaging³ or information about apolipoprotein E (APOE) ε4 genetic status have a clinically meaningful outcome in patients’ lives? Some clinicians argued that this knowledge is critical and could directly impact how individuals make life decisions. It would allow clinicians to help patients manage their illness, adopt lifestyle changes, and participate in clinical trials. Undoubtedly, an accurate diagnosis could support the clinical meaningfulness of a therapy. However, diagnostic accuracy alone is not sufficient to convey clinical meaningfulness for all stakeholders as there still needs to be evidence of a clear and sustainable therapeutic benefit. The Centers for Medicare and Medicaid Services (CMS) have not approved reimbursement of amyloid PET imaging as a diagnostic. However, many Roundtable participants agreed that an accurate diagnosis, particularly one that provides a clear-cut prognosis, remains a critical component in helping individuals, families, and policymakers plan for the future.

Regulators’ perspective on clinical meaningfulness

From the perspective of the U.S. Food and Drug Administration (FDA), a clinically meaningful treatment effect in AD is determined by whether the treatment has a positive and significant effect on how an individual feels, functions, or survives.^4–6 In other words, does the treatment make a difference in the patient's ability to think, care for themselves, and live independently? To enhance the incorporation of the patient's voice into product development, the FDA established the Patient-Focused Drug Development (PFDD) guidance.⁵ The role of the PFDD is to systematically identify what is important to patients by gathering information on patients’ experiences, perspectives, needs, and priorities and to meaningfully incorporate this input into drug development guidelines for regulatory decision-making.

The FDA also determines whether the potential risks of a treatment are outweighed by the potential benefits (i.e., the risk/benefit ratio). Are the therapeutic benefits of sufficient magnitude and the risks sufficiently low to justify approval of a drug? To achieve regulatory approval, regulators emphasize that there must be statistically robust evidence that a therapy provides a clinically relevant effect by producing a noticeable delay in deterioration or an improvement in daily activities that are durable.

Payers and health economists

Payers and health economists approach meaningfulness from a societal perspective. They concentrate on whether a novel treatment or diagnostic is worth the price compared to standard of care, that is, the cost-effectiveness of a treatment.^7–9 A meaningful outcome, from the perspective of CMS, is not the cost of the treatment per se, but whether the treatment will reduce the aggregate costs to health care (e.g., of emergency room visits, hospitalizations and institutionalization, etc.). Payer demand for economic value often guides access and reimbursement decisions. Does the direct cost of a treatment translate into cost-savings associated with delaying institutionalization or progressing to a more severe disease state? Thus, clinical meaningfulness, from the payer perspective, is defined by the cost/benefit ratio, that is, whether the cost of a treatment is balanced against its potential benefits to patients, care partners, and society.

From a health economist's perspective, the primary focus is evaluating how to achieve the greatest health benefit for society with the available budget. Health economists, in part, use mathematical models that incorporate projections regarding health benefits and the cost to health care and whether that cost can be alleviated by slowing disease progression or providing temporary symptom improvement. These cost/benefit evaluations use three basic methodological frameworks: a cost-benefit analysis, cost-effectiveness analysis, and cost-utility analysis. The differences among these economic evaluation frameworks is the way group ‘$Effect$’are valued; however, all share a common mathematical outcome, the incremental cost-effectiveness ratio (ICER), which is calculated for a new intervention, versus a comparator (e.g., usual care), as the incremental cost of the new alternative ($Cos{t}_{new}-Cos{t}_{usualcare}$) divided by incremental benefit of the new alternative ($Effec{t}_{new}-Effec{t}_{usualcare}$). Formally, the ICER is expressed as: $$ICER=\frac{Cos{t}_{new}-Cos{t}_{usualcare}}{Effec{t}_{new}-Effec{t}_{usualcare}}$$

In other words, the value of a new intervention is expressed in terms of the outcomes achieved relative to the costs, and, as an economic outcome, provides information to decision-makers to balance economic consequences to health care and to society as a whole against the benefit of intervention.¹⁰

The most commonly used health economic measure of effectiveness is to express Effect in terms of a preference-weighted generic health outcome, known as is the quality-adjusted life-year, or QALY. The QALY is based on two major components, length of life and quality of life, which are combined into a single index.^11,12 This cost-utility analysis is often preferred over cost-benefit or cost-effectiveness because the QALY avoids the pitfall of using either monetary or natural units as a value of outcome and gives a summary indicator of the health that can be produced by the health system. The QALY aggregates multiple dimensions of health, and then weights the change in health using population preferences that also reflect patient choice. These individualized QALYs promote more fair and equal access to scarce resources. A variety of instruments are used to estimate QALYs in dementia¹³ and promising new approaches are emerging.¹⁴

While economic evaluation provides a very powerful framework for informing policy decisions, compared to application in other competing areas of health priorities (e.g., stroke, heart disease, cancer), there are important methodological challenges to consider for dementia. For example, one recent study found that most dementia cost-of-illness studies have substantially underestimated the wider societal costs by not fully capturing indirect costs across the disease continuum, for example, the costs incurred by the entire community of people who contribute to the care of a person with dementia, and the full spectrum of how this financial burden impacts households.¹⁵ According to a report from the Alzheimer's Association, family and unpaid caregivers in the United States provided 18.5 billion hours of care valued at $234 billion in 2018 alone;¹⁶ yet, <20% of published dementia cost-effectiveness analyses include caregiver costs and caregiver QALYs.¹⁷ Also, caregiver quality of life is rarely addressed and^18,19 health-care payers are at risk of considering benefits only to patients when assessing the clinical meaningfulness of a treatment despite data that caregivers’ health and QOL are adversely affected resulting in added health-care costs. Furthermore, how informal care is valued as the disease progresses is an important consideration to ensure relevance to health budgets.²⁰ Therefore, in stating the perspective of an economic evaluation (i.e., to whom outcomes will inform),²¹ it has been recommended that implications to care partner and family be incorporated into these economic analyses so that AD interventions will not be deprioritized over other disease areas.²²

MEASURING CLINICAL MEANINGFULNESS IN ALZHEIMER'S DISEASE: HOW IS IT DEMONSTRATED?

How a clinically meaningful treatment response is demonstrated in AD drug trials often involves the use of clinical outcome assessments (COAs). COAs are usually validated instruments that generally have adequate psychometric properties such as validity, reliability, and responsiveness to change required to detect a therapeutic effect. Most COAs are questionnaires or assessments that include information from patients, care partners, and clinicians about the effects of the treatment over the course of the clinical trial. The most commonly used COAs include patient-reported outcomes (PROs), clinician/caregiver–reported outcomes (ClinROs), and composite performance outcomes (PerfOs).^5,6 These outcome scales frequently produce a continuous or categorical numeric that can be statistically analyzed to support evidence of benefit. A discussion of how clinical meaningfulness has been approached during various stages of disease severity was presented to Roundtable participants.

Measuring clinical meaningfulness in AD dementia

Since 1996, the FDA has approved two classes of medications to symptomatically treat AD; cholinesterase inhibitors (Cognex®—currently discontinued in the United States, Aricept®, Exelon®, and Razadyne®) and an N-methyl-D-aspartate (NMDA) antagonist, memantine (Namenda®). These clinical trials were conducted at mild to moderate stages of AD dementia and efficacy was measured using co-primary outcome assessments, that is, a combination of a performance-based measure (PerfO) such as the Alzheimer's Disease Cooperative Study Alzheimer's Disease Assessment Scale-Cognition (ADCS-ADAS-Cog), and a ClinRO, such as the ADCS Clinician Global Impression of Change plus Caregiver scale (CGIC+). These currently marketed pharmacological agents for the treatment of AD obtained approval, not only because of a statistically significant effect on outcomes of cognition and global/functional change, but also on evidence of clinical relevance, that is, demonstrating a 6-month delay in disease progression.²³

Measuring clinical meaningfulness in prodromal AD

For a number of years, the FDA draft guidance⁵ has required that AD clinical trials show efficacy on co-primary outcome measures: cognition and function. Since the ADAS-Cog was useful for obtaining FDA approval at later stages of disease severity, this COA became the pre-specified cognitive endpoint for numerous AD clinical trials at earlier stages of disease.²³ Unfortunately, this outcome measure failed to detect a therapeutic effect in numerous clinical trials, and no new FDA treatments for AD have been approved since 2003. It has been argued that the ADAS-Cog-11 lacked sensitivity at earlier disease stages,^24,25 which led to the development of variations of the ADAS-Cog. However, even with the inclusion of more sensitive items, numerous clinical trials at prodromal stages of AD failed to show a therapeutic effect; not only due to the therapeutic intervention but also to the outcome measure as well.²⁶ Consequently, new approaches and COAs were needed to measure a therapeutic benefit. While there has been extensive effort in the development of COAs to measure efficacy in cognition and function, not all of these outcomes have qualified for use in registration trials at prodromal and early stages of AD. Roundtable participants heard about these efforts; some of which may hold promise for measuring a clinically meaningful treatment effect.

The Critical Path for Alzheimer's Disease (CPAD), a part of the C-Path program, has been involved in the development and qualification of various COAs that might qualify as “fit for purpose” efficacy endpoints in AD clinical trials.^27–29 To address the need for COAs that capture the patient's voice and can be used across the globe, these PROs and PerfOs have been translated into multiple languages.³⁰ This has required the interviewing of patients and care partners in multiple countries and cultures and identification of a pool of clinically relevant items that can be culturally adapted across groups. These COAs are specifically designed to have conceptual equivalence so data can be used in pooled analyses. While promising, these COAs have not yet been fully validated or qualified as efficacy outcome measures for registration trials. Work on these COAs is ongoing, as culturally appropriate outcomes for use in global AD clinical trials are desperately needed.

An example of a PerfO scale undergoing development as an efficacy endpoint for AD is the University of California San Diego Performance-Based Skills Assessment (UPSA).^31,32 The UPSA was initially designed to capture a patient's performance on day-to-day activities that were deemed most important to patients, that is, communication skills, financial skills, and overall comprehension/planning. The items were adapted for different languages, cultures, and experiences so they could be used globally. However, the UPSA continues to undergo modifications. Nevertheless, its value in assessing real-world abilities that are important to patients is closely aligned with FDA guidance as representing a critical component of a clinically meaningful outcome.

The Goal Attainment Scale (GAS) represents another approach at capturing clinically meaningful outcomes by assessing whether a treatment meets the goals and objectives of the patient, care partner, and clinician.^33,34 The GAS is a ClinRO used by clinicians to judge individual treatment effects. However, unlike other ClinROs, the GAS is individualized according to the patient and care-partner needs. This enables clinicians, patients, and care partners to work together to set goals for treatment efficacy. A standardized interview is used to identify the symptoms and behaviors that are relevant and then rated as improved or declined from baseline. The most commonly expressed goals for treatment were related to cognition, function, leisure, behavior, and social interaction. The GAS was successfully used as a primary outcome measure in the Video-Imaging Synthesis of Treated Alzheimer's Disease (VISTA) study of the FDA-approved cholinesterase inhibitor galantamine.³⁵ The systematic incorporation of patient and care-partner goals about the effectiveness of a treatment, alongside the treating physician's expectations, is a strength of this ClinRO.

Measuring clinical meaningfulness in asymptomatic AD

In its latest draft guidance,⁵ the FDA endorsed the expanding knowledge gained from biomarker evidence of AD and recognized that treatment interventions may be more effective if they are initiated earlier along the pathophysiological cascade, prior to overt symptoms. In the absence of symptoms, how will clinical meaningfulness be demonstrated in asymptomatic stages of AD? Will primary endpoints involve quantifying amyloid beta (Aβ) or tau biomarkers and will lower biomarker levels translate into a clinically meaningful delay of disease? More importantly, will there be an impact on preventing future cognitive and functional decline as currently required by FDA guidance?

Disease-modifying secondary prevention trials in asymptomatic individuals have commenced and the field is tackling the development of outcome measures that can identify the earliest cognitive and functional changes associated with biomarker evidence of AD. Detection of these subtle changes may require the development of novel measurement tools rather than a total accuracy score or time to completion. What remains unknown is how these outcome measures will translate into demonstrating a clinically meaningful therapeutic response in AD prevention trials

One such approach to measuring efficacy in asymptomatic AD comes from the Alzheimer's Prevention Initiative (API). The API used a statistical modeling technique, similar to the development of the Alzheimer's Disease Composite Score (ADCOMS),^36,37 which uses partial least squares (PLS) regression to determine a combination of measures predicted to be sensitive to tracking cognitive and functional decline while controlling for practice effects and changes due to normal aging.³⁸ They developed this model using longitudinal data from three observational cohorts; the Religious Orders Study, the Memory and Aging Project, and the Minority Aging Research Study and created the API Preclinical Cognitive Composite (APCC). They demonstrated that the APCC was able to detect subtle cognitive changes more than a decade before the onset of overt dementia.³⁹ To establish that the composite could predict real-world meaningful changes, Novartis, Amgen, and API initiated the Insights to Model Alzheimer's Progression in Real Life Study (iMAP). They incorporated risk factors and determinants of transition into their statistical models and explored the predictive ability of change on composite endpoints such as the APCC, the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS), the Clinical Dementia Rating Scale-Sum of Boxes (CDR-SB), amyloid burden, and APOE genotype. The iMAP was to be a 5-year, multinational, prospective observational study of participants at-risk for developing AD or who had mild cognitive impairment (MCI) or dementia due to AD.⁴⁰ The aim was to contribute to our understanding of different disease stages and define models of individual trajectories over the course of the disease. Intended to run in parallel to the API Generation Program, iMAP was unfortunately discontinued when the Generation Program was terminated. However, the use of advanced statistical modeling to develop composite endpoints to track future decline and to assess treatment efficacy is being explored by other researchers in an attempt to discover a clinically meaningful endpoint in AD prevention trials.

A similar approach was used to develop the Preclinical Alzheimer Cognitive Composite (PACC) currently used in the Anti-Amyloid in Asymptomatic Alzheimer's Disease study (A4). A composite endpoint was derived from existing data across three large observational cohorts, the ADCS Prevention Instruments Project (ADCS-PI), the Australian Imaging Biomarkers and Lifestyle Study (AIBL), and the Alzheimer's Disease Neuroimaging Initiative (ADNI).⁴¹ Additional validation of the PACC using the Harvard Aging Brain Study (HABS), AIBL, and ADNI demonstrated subtle cognitive decline in clinically normal older adults with elevated Aβ biomarkers over 3 years. A 4- to 5-fold increase in risk for progression either to a clinical diagnosis of MCI, or a decline in Clinical Dementia Rating (CDR) from 0 to 0.5 was detected.⁴² These findings suggest that a cognitive composite, such as the PACC, may predict future cognitive decline in individuals in asymptomatic stages of AD. Nevertheless, it remains to be seen whether changes on a cognitive composite associated with elevated Aβ in observational studies will have adequate sensitivity to detect a treatment effect during an AD prevention trial and whether preserving cognition, by itself, will be clinically meaningful to regulators.

Additional efforts are being directed at optimizing COA composites based on biomarker-informed research tracking cognitive decline in those with positive biomarkers.⁴³ A revised version of the PACC, called the PACC5, now includes a measure of semantic fluency. The PACC5 showed a larger effect size of decline between high and low Aβ groups compared to the original PACC.⁴⁴ The PACC5 has been deployed in the U.S. POINTER Study, a lifestyle intervention, and is scheduled to be implemented into the AHEAD study, a future secondary prevention trial in AD. Another modified PACC derived from ADNI, AIBL, and the Swedish Biomarkers for Identifying Neurodegenerative Disorders Early and Reliably study (BioFINDER) showed that the elevated Aβ group declined consistently on a modified PACC composite but half did not reach a diagnosis of MCI until 6 years later.⁴⁵ The investigators of this study warned that the timeframe for a clinically meaningful change in cognition related to Aβ status may require larger and longer trials than is currently being considered, particularly if cognition is the only domain being measured.

Finally, COAs that were developed to measure real-world abilities and goals meaningful to patients at the stage of prodromal AD, may have value when combined with cognition in asymptomatic stages. For example, the UPSA, a COA that assesses more complex ADL activities and the GAS, which assesses social interaction and other higher order functions, may be useful for measuring maintenance of function or exposing subtle functional vulnerabilities, particularly at these earlier stages of disease. Future work will be needed to determine whether these primary endpoints in AD prevention trials are associated with the lowering of Aβ or tau biomarkers and whether they will result in a clinically meaningful delay of disease that could support a claim of therapeutic benefit for regulatory purposes.

MEASURING CLINICAL MEANINGFULNESS IN ALZHEIMER'S DISEASE: WHEN IS IT DEMONSTRATED?

In addition to establishing how clinical meaningfulness is achieved; determining when and at what magnitude a treatment response expresses clinical meaningfulness is also important. A clinically meaningful treatment effect is based on a combination of factors including the effect size, the durability of the effect, and the consistency and reliability of the effect in primarily slowing or preventing disease and having a durable impact on an individual's life. Roundtable participants discussed various methods for determining when benefit is achieved and how magnitude of change is measured during the course of an AD clinical trial.

Measuring a minimal clinically important difference

Establishing clinical meaningfulness is critical for attaining regulatory approval of a therapeutic. In AD, measuring a salient change in symptoms or disease course is complicated by progressive stages of disease severity as well as heterogeneity of symptoms. The concept of a minimal clinically important difference (MCID) was developed to define the smallest treatment change or difference that patients would recognize as important and that clinicians would concur merited a change in the patient's management. There are a number of different methods to obtain an MCID and a number of different factors that can influence the MCID value. Two common techniques, usually used together, are anchor-based and distribution-based approaches.⁴⁶

The anchor-based method compares scores or ratings on a COA with a clinically meaningful external “anchor” to precisely measure the smallest but most meaningful degree of change in a clinical treatment. For example, the anchor establishes whether a patient is better or worse after the treatment compared to baseline, either corresponding to the patient's own perception or the care partner's and clinician's opinion. The most commonly used anchors are global impressions of change such as the Patient or Clinician Global Impression.⁴⁷ These are 7-point Likert scales that measure if the intervention has succeeded in making the patient better (1 = markedly improved) or worse (7 = markedly worse) compared to baseline. These commonly used anchors are designed to denote a noticeable but meaningful change in the patient's overall condition and measure whether the time to decline signifies an increase or delay in symptomatic progression.⁴⁸ In earlier stages of disease, patients can participate in reporting symptomatic change. In later stages of disease, these anchor-based measurements rely more heavily on the report of the care partner and clinician because the patient's lack of awareness interferes with their ability to accurately report their symptoms.

Another approach to estimating MCID is the distribution-based method, which is a statistical technique that examines the ratio of change on a COA score to its variability. This measure of variability could be in the sample (effect size), change on a COA (standardized response mean), or the measurement precision of the COA (standard error of the mean).⁴⁶ The advantage of distribution-based approaches is that they are statistical methods that do not require an external anchor, thus making them easier to implement compared to anchor-based approaches. They are also independent of the sample size and not limited by statistical significance. While distribution-based approaches may not provide direct information regarding clinical relevance, when used in conjunction with anchor-based approaches, distribution-based methods and the magnitude of an effect size can provide very useful estimates of MCID.⁴⁶

Both the anchor-based and distribution-based approaches have been used in observational and clinical trials for AD dementia to account for important factors such as improvement versus worsening of disease severity. To illustrate, a study using observational data from the National Alzheimer's Coordinating Centers (NACC) Uniform Data Set (UDS) sought to validate the definition of MCID across the AD spectrum.⁴⁹ The NACC data include a derived indicator of whether the clinician has observed a meaningful decline in a patient's cognition, behavior, ADLs, and motor movements relative to previously attained abilities. Using this clinician's assessment of meaningful decline as their anchor, they found an average of 1- to 2-point increase in CDR-SB, a 1- to 3-point decrease on the Mini-Mental State Examination (MMSE), and a 3- to 5-point increase on the Functional Activities Questionnaire (FAQ) as clinically meaningful. However, they also found that the scores on these three tests used to indicate a clinically meaningful effect changed with disease severity. The MCID estimate of the distribution-based approach was slightly larger but consistent with the anchor-based approach. However, they cautioned that MCID measures of clinical significance in AD trial designs must take into account disease severity.

Another example of anchor-based and distribution-based methodologies is the secondary data analysis from the vitamin E and donepezil trial for the treatment of MCI.⁵⁰ The anchor-based method used a clinician-rated measure of change (CDR-SB) and a distribution-based method of disease severity (MMSE, ADAS-Cog). Based on a triangulation of these results, a change of at least one point on the CDR-SB was proposed as a threshold for a minimal deterioration in a patient at the MCI stage, while a change of 2.5 points or more was suggested as moderate deterioration. They found that treatment with vitamin E delayed the clinical diagnosis of AD only in the initial 12 months among patients already being treated with donepezil.

In sum, both anchor-based and distribution-based estimates of MCID may provide an important index of disease progression and be useful for assessing a clinically significant magnitude of a therapeutic response. However, the assessment and quantification of change can be quite challenging in AD trials across the disease trajectory, as the disease presents with a variety of symptoms and progresses at varying rates. However, MCID estimates can be a promising approach for determining when clinical meaningfulness is achieved.

Study design and statistical approaches for exploring clinical meaningfulness

Another important tool for exploring the meaningfulness of a therapeutic response is time-to-event (TTE). TTE is a study design methodology that uses statistical techniques of survival analysis such as Kaplan-Meier or Cox proportional hazard models to measure the length of time until the incidence of a relevant event (e.g., diagnosis of MCI/dementia, institutional placement, death).⁵¹ In TTE, events such as being diagnosed with MCI or dementia is meaningful to patients, care partners, and clinicians but methodologically problematic in that the diagnostic definitions contain subjective decisions based on clinical diagnosis at each visit.⁵² The power of TTE is tied to its variability. This variability can be very high when the event is frequent, or very rare when the event is delayed, as in asymptomatic AD. Interval censoring also limits power to see a meaningful effect when information is not available because of loss to follow-up or the event did not occur prior to the end of the study. TTE designs in AD clinical trials may be problematic for other reasons including the less defined transitions that occur along the AD disease trajectory that may also be meaningful events but not captured by the clinical rating.⁵²

Other statistical approaches that are purported to be more powerful than TTE are those that use continuous outcomes of disease progression and a common close design. In this type of approach, there is no censoring. Every patient is included and contributes to the average outcome score. In the common close design, follow-up continues until the last enrolled subject reaches the last visit. As a result, this approach produces a great deal more information and mitigates the problem of forcing subjects out of the trial before all the relevant information has been collected.^51,52 However, these designs are also problematic in that some subjects will be exposed to placebo for very long periods of time. These study designs may also end up having substantial missing data that can bias results. Nonetheless, a continuous measure design, in contrast to TTE, has the potential of capturing relative slowing of disease progression and less defined transitions, information that could be both meaningful and understandable to patients and care partners. In essence, use of continuous measurements provides some reassurance that even small changes will be captured that may indicate when clinically meaningful effects occur.

Survival analyses using categorical events versus continuous variables have strengths and weaknesses when applied to AD clinical trials. Categorical events in TTE seem more easily related to a clinically meaningful and tangible therapeutic response but at the risk of requiring larger sample sizes due to censoring and loss to follow-up. On the other hand, continuous measure designs are less prone to bias introduced by subjective diagnostic impressions but are dependent on the sensitivity of a COA to disease progression. It may be that both tools are needed because they are more easily related to clinical meaningfulness. Newer statistical tools are being explored that more robustly establish responder thresholds of clinical change according to FDA guidance. Further work is needed to define the relevance and magnitude of a therapeutic response to which all stakeholders can agree.

CONCLUSIONS

During the Fall 2019 AARR, it became clear that determining whether an AD intervention is clinically meaningful remains a challenge. Regulators provide guidance that a disease-modifying intervention must establish clinical benefit by slowing decline or preventing future impairment. It must also include the patient's voice and show that the intervention has a positive effect on how individuals feel, function, or survive. A clinically meaningful outcome takes into account various perspectives (i.e., patients, care partners, clinicians, regulators, payers, and health economists). How cognitive and functional change should be measured is complicated by a syndrome that has considerable heterogeneity in pathologies, phenotype, and rates of progression. Roundtable participants considered clinical meaningfulness at each stage of disease and from three vantage points: (1) To whom is the outcome relevant? (2) How is clinical meaningfulness demonstrated? and (3) When is clinical meaningfulness achieved?

Essentially, all stakeholders agree that a clinically meaningful outcome must produce a clear and sustainable benefit, while altering the disease trajectory. Whether and how that could be achieved across all stages of disease is challenging. Early in the AD process, cognitive deficits are not significant enough to impact everyday activities and progression tends to be slow, making detection of a clinically meaningful therapeutic effect very challenging. Cognitive endpoints might lack the ability to detect decline in a timely fashion due to item difficulty and ceiling effects, as well as the inherent within-day variability that can overwhelm small effect sizes and subtle changes. While cognitive composites attempt to capture disease heterogeneity, questions remain about their psychometric properties, internal consistency, how they were derived, and whether every item equally contributes to change over time.⁵³ Another key consideration is whether composite scales can disambiguate normal fluctuations in cognition from treatment effects, particularly at asymptomatic stages, when cognitive changes are subtle and within measurement error. In addition, clinical outcomes must also demonstrate a meaningful effect across diverse populations, cultures, and multiple countries, to harmonize clinical trial data worldwide.

While regulators consider cognition to be sufficient as a meaningful endpoint in prevention trials, they emphasize the importance of incorporating the patient's voice into clinically relevant outcomes. This raises questions as to whether a cognitive outcome captures what is most essential to the patient. Perhaps functional activities of daily living such as maintaining the ability to drive, prepare taxes, or work may be more relevant than changes on a cognitive composite. It may be that a combination of measures that comprise aspects of cognition, motor, function, and biomarkers might convey clinical relevance. Future work is needed to determine whether combining multiple composites that encompass a broad range of cognitive, functional, genetic, and biological outcomes will be useful for this purpose.

PROs have gained prominence for regulators because they address the patient and care partner perspective on clinical relevance. Those scales that incorporate patient and care-partner goals along with the clinician's expectations, bring a perspective to clinical meaningfulness not otherwise captured. However, PROs are also problematic. Patient responses to these PROs can be susceptible to anosognosia, that is, a lack of insight into the severity of their cognitive and functional changes. Furthermore, it is unclear when patients begin to experience diminished insight resulting in the unreliability of their responses. Care partner reports are also susceptible to many factors that might obscure the reliability of PROs including cultural and religious influences, how long they have known the patient, the nature of their relationship, whether they live together, their own medical issues, and whether the care partner also has early symptoms of an affective illness (i.e., depression) or neurodegenerative disease.^54,55

Statistical approaches to clinical meaningfulness are exciting and lend an objectivity to the measurement of treatment efficacy that is not accomplished by other COAs. However, the interpretation of meaningfulness using either TTE or a continuous measure (or a joint model that combines the two approaches) will still depend on the development and validation of the outcome measure, the size of the effect, and the expense of the treatment. Even if TTE fails to reach statistical significance, delay of onset to further disease progression remains meaningful to regulators, particularly if it directly impacts how a patient feels, functions, and survives.

Newer digital technologies should also be considered as to whether they might add value in determining treatment efficacy in a clinical trial. Digital tools, such as the Computerized Cognitive Composite (C3)^56,57 is a secondary outcome measure in the A4 study that measures other aspects of cognition including reaction time, working memory, and executive functions that may capture atypical changes earlier in the disease. Perhaps, even more refined digital analytic outcomes would be meaningful, that is, those that expose the process by which someone completes a task, rather than merely a total accuracy score.^58,59 Passive monitoring with digital tools that measure gait, sleep, tremor, word cadence, and location might also be explored as novel endpoints. Digital tools, unlike other COAs, more intimately adhere to the patient's experience and could reveal meaningful and relevant treatment effects that go unobserved in everyday life.⁶⁰

Finally, it remains to be seen whether biomarkers of AD pathology will determine clinical meaningfulness because they provide biological proof of disease modification.⁶¹ Even so, if a disease-modifying intervention successfully lowers Aβ or tau, how much lowering will be necessary to be clinically meaningful? Will other biomarkers, such as those that target neuroinflammation or additional physiologic processes, be required to realize a clinically meaningful delay in AD progression? Nevertheless, there still needs to be a closer link as to whether the disease-modifying treatment slows cognitive and functional decline; prevents future impairment; and has a meaningful impact on how individuals feel, function, or survive.

While we did not reach consensus as to what constitutes a “clinically meaningful outcome,” it became clear that a clinically relevant treatment effect needs to incorporate the voices and expectations of all stakeholders. AARR participants had the privilege of hearing about the innovative work that is occurring across the globe to build clinically relevant outcomes. Combined with the intensive work being done on developing novel disease-modifying interventions, these efforts bring us another step closer to reaching our goal of accelerating a cure by 2025.

ACKNOWLEDGMENTS

The authors thank our contributing speakers, panelists, and moderators: David Knopman, MD; Dan Gibbs, MD; Katie Brandt, MM; John Hsiao; Megan Canniere Zoschg, PharmD; Michelle Campbell, PhD; James Rollins, MD; Pei Jung Lin, PhD; Dominic Trepel, PhD; Eric Bastings; Caroline Petit-Turcotte; Ananth Annapragada, PhD; Eric Siemers, MD; Ira Shoulson, MD; Ralph Benedict, PhD; Adam Staffaroni, PhD; J. Michael Ryan, MD; Dorene Rentz, PsyD; Alette Wessels, PhD; Jessica Langbaum, PhD; Glenn Stebbins, PhD; Sonya Eremenco, MA; Kenneth Rockwood, MD; Fiona McDougall, PhD; Kate Papp, PhD; Sam Dickson, PhD; Michael Donohue, PhD.

CONFLICTS OF INTEREST

AW is a full-time employee and shareholder of Eli Lilly and Company. AA declares stock ownership and consulting fees from Alzeca Inc, stock ownership in Sensulin LLC. CJE is a full-time employee of Cogstate Ltd. MG is a full-time employee of Abbvie, Inc. DSM is a full-time employee of Signant Health. CR is a full-time employee of MedAvante-ProPhase. JMR, previously with Rodin Therapeutics, at time of publication is a full-time employee of Carnot Pharma, LLC dba RA Ventures. GW is a full-time employee of Boehringer Ingelheim. DSK served on a Data Safety Monitoring Board for the DIAN study. He serves on a Data Safety monitoring Board for a tau therapeutic for Biogen, but receives no personal compensation. He is an investigator in clinical trials sponsored by Biogen, Lilly Pharmaceuticals, and the University of Southern California. He serves as a consultant for Samus Therapeutics, Third Rock, Roche, and Alzeca Biosciences but receives no personal compensation. He receives research support from the NIH. AMS receives support from the NIH/NIA, the Larry L. Hillblom Foundation, and the Bluefield Project to Cure FTD. MCC, CW are full-time employees of the Alzheimer's Association.