Highly sensitive measurements of disease progression in rare disorders: Developing and validating a multimodal model of retinal degeneration in Stargardt disease

Abstract

Background

Each inherited retinal disorder is rare, but together, they affect millions of people worldwide. No treatment is currently available for these blinding diseases, but promising new options—including gene therapy—are emerging. Arguably, the most prevalent retinal dystrophy is Stargardt disease. In each case, the specific combination of ABCA4 variants (> 900 identified to date) and modifying factors is virtually unique. It accounts for the vast phenotypic heterogeneity including variable rates of functional and structural progression, thereby potentially limiting the ability of phase I/II clinical trials to assess efficacy of novel therapies with few patients. To accommodate this problem, we developed and validated a sensitive and reliable composite clinical trial endpoint for disease progression based on structural measurements of retinal degeneration.

Methods and findings

We used longitudinal data from early-onset Stargardt patients from the Netherlands (development cohort, n = 14) and the United Kingdom (external validation cohort, n = 18). The composite endpoint was derived from best-corrected visual acuity, fundus autofluorescence, and spectral-domain optical coherence tomography. Weighting optimization techniques excluded visual acuity from the composite endpoint. After optimization, the endpoint outperformed each univariable outcome, and showed an average progression of 0.41° retinal eccentricity per year (95% confidence interval, 0.30–0.52). Comparing with actual longitudinal values, the model accurately predicted progression (R2, 0.904). These properties were largely preserved in the validation cohort (0.43°/year [0.33–0.53]; prediction: R2, 0.872). We subsequently ran a two-year trial simulation with the composite endpoint, which detected a 25% decrease in disease progression with 80% statistical power using only 14 patients.

Conclusions

These results suggest that a multimodal endpoint, reflecting structural macular changes, provides a sensitive measurement of disease progression in Stargardt disease. It can be very useful in the evaluation of novel therapeutic modalities in rare disorders.

About the Authors:

Stanley Lambertus

Affiliation: Department of Ophthalmology, Donders Institute for Brain, Cognition and Behaviour, Radboud university medical center, Nijmegen, The Netherlands

ORCID http://orcid.org/0000-0001-9600-9592

Nathalie M. Bax

Affiliation: Department of Ophthalmology, Donders Institute for Brain, Cognition and Behaviour, Radboud university medical center, Nijmegen, The Netherlands

Ana Fakin

Affiliation: Moorfields Eye Hospital and UCL Institute of Ophthalmology, London, United Kingdom

Joannes M. M. Groenewoud

Affiliation: Department for Health Evidence, Donders Institute for Brain, Cognition and Behaviour, Radboud university medical center, Nijmegen, The Netherlands

B. Jeroen Klevering

Affiliation: Department of Ophthalmology, Donders Institute for Brain, Cognition and Behaviour, Radboud university medical center, Nijmegen, The Netherlands

Anthony T. Moore

Affiliation: Moorfields Eye Hospital and UCL Institute of Ophthalmology, London, United Kingdom

Michel Michaelides

Affiliation: Moorfields Eye Hospital and UCL Institute of Ophthalmology, London, United Kingdom

Andrew R. Webster

Affiliation: Moorfields Eye Hospital and UCL Institute of Ophthalmology, London, United Kingdom

Gert Jan van der Wilt

Affiliation: Department for Health Evidence, Donders Institute for Brain, Cognition and Behaviour, Radboud university medical center, Nijmegen, The Netherlands

Carel B. Hoyng

* E-mail: [email protected]

Affiliation: Department of Ophthalmology, Donders Institute for Brain, Cognition and Behaviour, Radboud university medical center, Nijmegen, The Netherlands

Introduction

Inherited blindness affects millions of people worldwide—the majority suffering from retinal disease [1]. Inherited retinal disorders now represent the primary cause of blindness in the working age population in the UK, and secondary in childhood [2]. They are clinically and genetically heterogeneous, caused by sequence variants in more than 300 distinct genes (RetNet) http://www.sph.uth.tmc.edu)/. Mutations in the ATP-binding cassette, subfamily A, member 4 (ABCA4) gene are linked to arguably the most common retinal dystrophy: autosomal recessive Stargardt disease (STGD1) [3]. Each case of STGD1 is, in a sense, unique by specific combinations of pathogenic ABCA4 variants (> 900 variants identified to date) and modifying factors. Consequently, the natural course is highly variable, ranging from severe early-onset rapid degeneration [4, 5] to relatively mild late-onset disease [6, 7]. The eventual vision loss results from progressive impairment and degeneration of photoreceptors and their supporting retinal pigment epithelium (RPE) [8].

Recently, significant advancement has been made in the development of therapies that aim to slow disease progression, or even to restore lost photoreceptors in STGD1. These include replacement of ABCA4 by gene therapy [9], cell-based therapies [10], and pharmacological strategies including slowing the visual cycle or inhibition of vitamin A dimerization [11, 12]. Clinical trials are currently recruiting patients to assess safety and efficacy. However, successful evaluation of these therapies critically hinges on sensitive and reliable measures for disease progression.

To monitor efficacy of a treatment, current trials generally use functional endpoints (S1 Table). One of the most widely used U.S. Food and Drug Administration-approved endpoints is best-corrected visual acuity (BCVA) [13]. However, the main disadvantage of BCVA lies in its extremely variable deterioration rate in patients with retinal dystrophies. Moreover, visual acuity decline can be a late phenomenon following a long period of pathophysiological changes [14]. As a result, the endpoint has an unfavorable signal-to-noise ratio, which leads to a need for longer follow-up and large cohorts. This setup is impossible to achieve given the unacceptable long time frame and the rarity of these disorders.

However, studies suggest that structural abnormalities gradually expand centrifugally [5, 15], starting from the foveal center towards the periphery. Although loss of foveal function is highly important from a patient’s perspective, it is only one step in the overall pattern of retinal degeneration. This pattern is thought to initiate with melanization abnormalities in the RPE [16, 17], and is trailed by changes in lipofuscin fluorophores [18–20], degeneration of the RPE [21–23], and loss of the ellipsoid zone (EZ) and external limiting membrane (ELM) [24–27]. However, these transition zones are not present in every case, and do not always start at the center of the macula; many are still not well understood.

A composite outcome measure is likely to outperform single candidate outcome measures [28] in accurately measuring short-term progression, and can therefore increase statistical power of pivotal clinical trials. In this study, we chose to measure four structural metrics of expanding transition zones and one psychophysical metric over time in a cohort of patients with early-onset STGD1. These included questionably and definitely decreased autofluorescence (QDAF and DDAF), and loss of the EZ, ELM and BCVA. Next, we assessed intra- and inter-grader differences. Having standardized all metrics, we then calculated an optimal weighted composite. The same measurements were made in a second patient cohort, and, using the composite, we predicted and compared progression with real longitudinal values. We subsequently ran a simulation to examine the power of the composite endpoint and alternatives to detect a difference in outcome given a theoretical intervention with a significant impact on progression rate.

Results

Characteristics of the initial patient population

A cohort of 14 patients with early-onset STGD1 ascertained at the Radboud university medical center in Nijmegen had a median age at disease onset of 9 years (range, 4–11). Seventy-four eye-visits of 28 eyes were included in this study (range, 2–4 visits per eye). At the time of inclusion, all eyes had abnormal fundus autofluorescence (FAF) imaging and evidence of loss of photoreceptors on spectral-domain optical coherence tomography (SD-OCT), primarily limited to the macula at their baseline visit. Due to their early disease onset and rapidly progressive macular changes, a significant proportion of patients were expected to progress to retina-wide disease over time [4, 5, 29]. The follow-up time ranged from 1.1 to 9.7 years (median, 4.7).

Visual acuity measurements indicate extra-macular dysfunction

At the first visit, the Nijmegen cohort had a median BCVA of 20/205 Snellen in both eyes. The highest BCVA can be obtained at the fovea and diminishes rapidly by retinal eccentricity (ε) as shown in Fig 1 [30]. According to Fig 1, 20/205 Snellen corresponds to 12.76° of ε by BCVA (εBCVA), the equivalent diameter being 7.7 mm. Assuming a 5.5 mm diameter of the macula, the high degree of eccentricity indicated that the patients’ visual function corresponded to extra-macular disease at baseline.

[Figure omitted. See PDF.]

Fig 1. Visual acuity as a function of degrees of retinal eccentricity.

https://doi.org/10.1371/journal.pone.0174020.g001

Structural parameters are measured with high reproducibility

To assess the accuracy and reliability of measurements on FAF imaging and SD-OCT, we analyzed inter- and intra-grader agreements. These included transition zones of QDAF and DDAF areas, and transverse loss of the EZ and ELM. The absolute mean (± standard deviation) differences within one grader for εQDAF, εDDAF, εEZ and εELM were 0.19 ± 0.18°, 0.20 ± 0.23°, 0.10 ± 0.12°, and 0.20 ± 0.22°, respectively. The intra-grader measurements were highly correlated with intraclass correlation coefficients (95% confidence interval) of 0.995 (0.989–0.997), 0.994 (0.986–0.997), 0.998 (0.993–0.999), and 0.998 (0.994–0.999), respectively. The absolute mean differences between graders were 0.26 ± 0.23°, 0.22 ± 0.21°, 0.26 ± 0.23°, and 0.23 ± 0.18°, respectively. The measurements from both graders were highly correlated with intraclass correlation coefficients of 0.992 (0.986–0.995), 0.994 (0.990–0.996), 0.992 (0.971–0.998), and 0.997 (0.992–0.999), respectively.

Initial transition zones on retinal imaging

We measured the abnormalities of lipofuscin fluorophores in the RPE by determining both the transition zones of DDAF and QDAF on FAF imaging as shown in Fig 2A [20, 31]. At the first visit, the median εDDAF and εQDAF for both eyes were 1.80° (range, 0–5.60) and 4.22° (range, 1.68–7.67), equivalent to an area of 0.92 mm2 and 5.03 mm2, respectively. Additionally, we measured the horizontal loss of the photoreceptor-related ELM and EZ through the foveal center on SD-OCT (Fig 2B). At the first visit, the median εELM and εEZ for both eyes were 5.09° (range, 2.50–10.71) and 3.54° (range, 2.99–7.83), equivalent to a loss of 3.05 mm and 2.12 mm, respectively.

[Figure omitted. See PDF.]

Fig 2. Schematic and representative images of measurements in fundus autofluorescence imaging and spectral-domain optical coherence tomography.

Patient 9: area of questionably decreased autofluorescence (QDAF, blue), 1.37 mm2; area of definitely decreased autofluorescence (black), 0.33 mm2; transverse loss of external limiting membrane (ELM-loss, red), 1.75 mm, transverse loss of ellipsoid zone (EZ-loss, blue), 2.24 mm; best-corrected visual acuity, 20/100; ABCA4 variants, c.1622T>C;3113C>T:p.[Leu541Pro;Ala1038Val] and c.6316C>T:p.(Arg2106Cys).

https://doi.org/10.1371/journal.pone.0174020.g002

A structural composite measure of expanding transition zones outperforms univariable measures

We assessed the change of each individual parameter over time by linear mixed-effects models. The models accounted for between-patients and between-eyes effects. We then calculated the sensitivity of each parameter by the ratio of the population mean slope, i.e., overall disease progression, and the residual standard deviation (mean-to-standard-deviation ratio, MSDR). εQDAF had the highest sensitivity (MSDR, 2.32), whereas εBCVA had the lowest (MSDR, 0.08). The MSDR for εEZ could not be obtained as there were not sufficient measurements available. Sensitivities of all individual parameters are shown in Table 1. We constructed a composite variable from changes in ε as measured by BCVA, QDAF, DDAF, EZ, and ELM. Results from MSDR calculations of every potential weighting combination indicated that the most sensitive composite consisted of a weighted mean of changes in εBCVA (0%), εQDAF (25%), εDDAF (5%), εEZ (55%), and εELM (15%). We observed an overall progression rate of 0.41°/year (95% confidence interval, 0.30–0.52). Potential MSDRs for all measures with different weighting scores are shown in Fig 3. Based on the weighted composite score we predicted changes in ε of six patients with a third or fourth visit. The predicted values correlated strongly with the measured weighted composite score (R2, 0.904; slope, 0.90 [0.70–1.11]; intercept, 0.14 [-0.55–0.83]; Fig 4A).

[Figure omitted. See PDF.]

Fig 3. Highest potential mean-to-standard deviation ratio (MSDR) for each single outcome measure at different weightings with all possible weight combinations of the other metrics.

MSDRs for best-corrected visual acuity (grey) decrease at increasing weight. MSDRs for transition zones of questionably decreased autofluorescence (blue) increase until 25% weight, but gradually decrease at higher weights. MSDRs for transition zones of definitely decreased autofluorescence decrease at weights higher than 5%. MSDRs for loss of the ellipsoid zone (green) are constant, but decrease substantially at weights higher than approximately 70%. MSDRs for loss of the external limiting membrane (black) decrease at weights higher than approximately 80%.

https://doi.org/10.1371/journal.pone.0174020.g003

[Figure omitted. See PDF.]

Fig 4. Weighted composite score and predicted outcome.

Matching colors represent the right and left eye of the same patient. (A) Results from six early-onset Stargardt patients. (B) The predicted outcome in the replication cohort showed comparable results.

https://doi.org/10.1371/journal.pone.0174020.g004

[Figure omitted. See PDF.]

Table 1. Yearly progression rate of changes in retinal eccentricity (ε) by visual function, fundus autofluorescence and optical coherence tomography.

https://doi.org/10.1371/journal.pone.0174020.t001

Accurate prediction of disease progression is validated in a replication cohort

Using the identical weighted score in the mixed-effects model, we predicted the progression in a separate replication cohort of 18 Stargardt patients, ascertained at Moorfields Eye Hospital in London. One hundred and thirty-eight eye-visits were included (range, 2–6 visits per patient). The London cohort was not significantly different to the Nijmegen cohort: with a median age at disease onset of 8 years (range, 5–11; Mann-Whitney U, P = 0.419), and follow-up from 1.0 to 11.0 years (median, 4.5; Mann-Whitney U, P = 0.722). We observed an overall progression of 0.43°/year (95% confidence interval, 0.33–0.53). Cohort characteristics compared to the Nijmegen cohort are further described in Table 2. Predicted values correlated with the measured weighted composite score (R2, 0.872; slope, 1.17 [0.99–1.34]; intercept, -0.29 [-0.77–0.18]; Fig 4B).

[Figure omitted. See PDF.]

Table 2. Characteristics of early-onset Stargardt cohorts from Radboud university medical center (Radboudumc) and Moorfields Eye Hospital (MEH).

https://doi.org/10.1371/journal.pone.0174020.t002

Simulation reveals high statistical power despite small numbers of patients and short follow-up

Finally, to assess the value of the composite biomarker in an interventional trial, we simulated a randomized-controlled paired trial in 14 patients, with a two-year follow-up period, and different treatment effects (Fig 5). Using the optimized weighted composite score as the primary endpoint, we obtained a statistical power of > 80% (with a significance level of 0.05) in the case of a 25% treatment effect. Unweighted structural scores decreased the power by approximately 50%.

[Figure omitted. See PDF.]

Fig 5. Power calculations of a simulated therapeutic trial based on fourteen early-onset Stargardt patients.

Dark blue line: a power of 80% is reached with a 25% overall treatment effect and a two-year follow-up period. Purple line: the power will drastically decrease when best-corrected visual acuity is included in the outcome measure. Turquoise line: worse eye treated. Green line: one-year follow-up. Ocher line: unweighted structural composite. Red line: non-paired trial design. Yellow line: best-corrected visual acuity as a single outcome measure.

https://doi.org/10.1371/journal.pone.0174020.g005

Discussion

The genotypic and phenotypic heterogeneity of rare diseases are a challenge for designing therapeutic clinical trials using conventional parameters. This affects many patients; current estimates are between 6.5 and 9.9 million inhabitants of the EU28 countries (1.3–2.0%). Jointly, these diseases represent a relevant public health issue [32], and to evaluate novel treatments, better strategies are needed. Current strategies use biomarkers, which are often insufficient to provide appropriate sample size calculations, or require long-term follow-up (S1 Table). However, an integrated approach of these individual biomarkers can result in a reliable and sensitive marker for disease progression. In this paper, we showed that such markers can be developed using composite endpoints and weighting optimization techniques.

The composite endpoint that we developed to sensitively measure disease progression in patients with early-onset STGD1 was based on its spatiotemporal disease course. Although the exact pathophysiological mechanisms of the disease pattern are still not completely understood, natural history studies suggest a centrifugal expansion from macula-only to potentially retina-wide disease [5, 15, 16, 29, 33]. Visual acuity failed to detect this gradual expansion due to its low signal-to-noise ratio in the composite model. In the final model, the expansion of different transition zones is analyzed by two widely available imaging techniques. SD-OCT can visualize photoreceptor damage, represented by loss of the ellipsoid zone, followed by loss of the external limiting membrane [34–36], and FAF imaging can detect RPE atrophy associated with photoreceptor dysfunction/loss [26]. In the future, the composite model may be further optimized by incorporating more sophisticated retinal imaging techniques such as adaptive optics scanning light ophthalmoscopy which affords in-vivo cellular imaging. Parameters derived from electrophysiological assessment could also potentially strengthen the composite model, although it is limited by significantly greater test-retest variability than aforementioned structural testing [37–39].

We are aware that, ultimately, the value of novel therapeutic modalities should be inferred from their impact on outcomes that matter to patients. In the context of eye disease, these would certainly include vision and the impact that visual impairments have on daily life (patient-reported outcome measures). In addition, the long-term safety of such novel treatments should be safeguarded. For these purposes, the metric that we have developed in this study is unlikely to be appropriate in isolation, since its focus lies on structural abnormalities alone. We believe its value mainly derives from the ability to rationally select promising novel treatment modalities and also potentially facilitate patient selection for recruitment to clinical trials. Consequently, it will need to be demonstrated that the employed structural parameters correlate with functional outcome in the long term.

Multimodal analysis is a powerful technique, potentially reducing costs and duration of clinical trials and also likely reducing beta errors in data analysis, thereby hopefully facilitating effective treatments being identified more readily and rapidly for patients with rare diseases. There is also the possibility of further improvements with the inclusion of other biomarkers in the future, and the potential to be extrapolated to other disorders.

Materials and methods

Patient selection

We selected patients from the Stargardt databases of the Departments of Ophthalmology at Radboud university medical center (Nijmegen, The Netherlands) and Moorfields Eye Hospital (London, United Kingdom). We used the patient data from Radboud university medical center to develop the progression model, and the data from Moorfields Eye Hospital to replicate the study. We included patients with the faster progressive early-onset form of STGD1, harboring two or more likely disease-causing sequence variants in ABCA4 (Table 3), and with at least one year follow-up with FAF imaging and/or SD-OCT The early-onset phenotype typically presents with foveal atrophy that may precede the development of yellow-white fundus flecks. Early-onset STGD1 is associated with the most rapid deterioration of all patients with STGD1 [5]. We only included patients with a reported disease onset <12 years of age [4, 5]. We excluded (1) patients with very early disease in which only thickening of the external limiting membrane was present, because this would preclude the OCT measurements, (2) patients with advanced disease characterized by RPE atrophy beyond the vascular arcades at first presentation, and (3) patients who participated in an interventional trial [40]. This study was approved by the local ethics committee on Research Involving Human Subjects of the Radboud university medical center “Commissie Mensgebonden Onderzoek Regio Arnhem-Nijmegen” and the National Research Ethics Service (NRES) Committee London—Camden & Islington, and was performed in accordance with the Declaration of Helsinki. All patients provided informed consent prior to receiving additional ophthalmologic examinations. Written informed consent was obtained from parents, caretakers, or guardians on behalf of the children below 18 years (at Radboud university medical center) or below 16 years (at Moorfields Eye Hospital). Additional written informed consent was obtained from children between 12 and 17 years old (at Radboud university medical center) or between 6 and 15 years old (not necessary, but preferable at Moorfields Eye Hospital).

[Figure omitted. See PDF.]

Table 3. ABCA4 variants in early-onset Stargardt patients from Radboud university medical center (Radboudumc) and Moorfields Eye Hospital (MEH).

https://doi.org/10.1371/journal.pone.0174020.t003

Clinical examinations

We reviewed the records and imaging databases to extract information including BCVA, FAF imaging, and SD-OCT. Best-corrected visual acuity was measured using a Snellen or Early Treatment Diabetic Retinopathy Study chart. Short-wave FAF imaging (λ = 488 nm, emission 500–700 nm) was performed using a confocal scanning laser ophthalmoscope (Spectralis HRA+OCT or HRA2, Heidelberg Engineering, Heidelberg, Germany). The field of view was set at 30°×30° or 55°×55° and was centered at the macula. Cross-sectional images were obtained using SD-OCT (Spectralis HRA+OCT, Heidelberg Engineering, Heidelberg, Germany) centered at the macula (S1 Dataset).

Functional measurements

We analyzed data with SAS Statistical Analysis Software Version 9.2 (SAS Institute, Cary, NC). Best-corrected visual acuity can be expressed in retinal eccentricity (ε) to estimate the spatial extent of retinal dysfunction spatially. It has been calculated previously that an object must grow by 0.2° in size to maintain BCVA for each degree of eccentricity [30]. The BCVA is therefore reduced by a factor of 1/1.2 for each degree. This results in a transformed BCVA to the estimated equivalent of ε:(1)which can be written as(2)

Quantitative measurements on imaging

Abnormalities that were expected to consistently increase over time were included for quantification. Two independent authors (S.L. and N.M.B.), blinded to the each other’s findings, manually delineated areas of abnormal autofluorescence signals based on darkness levels on FAF imaging, and the loss of retinal layers on SD-OCT. These included areas of questionably decreased autofluorescence (QDAF), definitely decreased autofluorescence (DDAF), transverse loss of the external limiting membrane (ELM) and ellipsoid zone (EZ) on the OCT scan through the fovea. All measurements were standardized to retinal eccentricity. One degree of eccentricity corresponds to approximately 0.3 mm on the retina [41]. Therefore, the eccentricity can be calculated as the radius of the circular equivalents of the sum of QDAF and DDAF areas using the previously reported conversion factor:(3)and(4)

As the transverse horizontal loss of retinal layers represents the diameter of a circular equivalent, the eccentricities of ELM loss and EZ loss could be calculated as follows:(5)and(6)

Reproducibility of measurements

If the discrepancy between graders exceeded 1°, the graders were asked to reach consensus on the location and extent of the transition zone. To assess the intra-grader reproducibility, one of the two graders (S.L.) measured each image by each method twice, with a two-month interval between gradings of the same image. We calculated the absolute inter- and intrarater agreement of εQDAF, εDDAF (5%), εELM, and εEZ by intraclass correlation coefficients with 95% confidence intervals. Averaged values of the graders were used for final analyses.

Modelling disease progression

The composite ΔC was constructed from changes in ε as measured in all univariable biomarkers:(7)

* a, b, c, d, and e are weighting scores for each biomarker.

The composite biomarker was used to detect disease progression in a linear two-level random effects mixed model, which can describe expansion rates of a transition zone quite well within a short period. It accounts for variations between patients and between the eyes of each patient, and can thus incorporate a potential non-linear process in the variance components of these random effect [42]:(8)

* ΔCij(Δt) is the change of the composite score from baseline for the ith patient in eye j at time since baseline Δt,

* Δtik is the kth follow-up time for patient i,

* s is the mean population slope (first level fixed effect),

* si are the deviations of the ith patient’s slope from the population value (independent second level random effects),

* sij are the deviations of the slope of both eyes in patient i from his mean regression line (independent third level random effects),

* Eijk is the residual error.

The intercept of the model was set to zero, because differences from baseline were used.

Weighting scores optimization

Weighting scores were subsequently chosen by an optimization criterion [28], which was constructed by the ratio of the population mean slope and the residual standard deviation (MSDR). The criterion was empirically evaluated for different combinations of weighting scores of parameters:(9)

* s is the mean population slope,

* RMSE (root-mean-square error) is the residual standard deviation, a scale-dependent measure for accuracy.

The total number of unique combinations follows a binomial coefficient , where n is the number of intervals, and a is the number of biomarkers. Five biomarkers with 5% intervals (20 steps from 0 to 100%) resulted in combinations, which could be tested within reasonable computational time. A weighting of zero resulted in exclusion of that particular biomarker. When a certain biomarker was missing or not measurable, the ΔC was calculated by the changes in the other biomarkers with their respective weighting scores. The combination with the highest MSDR was eventually used in the final model (S2 Dataset and S1 Appendix).

Validation and replication of the model

In a subset of patients (six), in which three or more visits were available, we calculated disease progression of the last visit. These final visits were excluded in the construction of the weighted composite. We compared these scores with the predicted scores based on the mixed model. Using identical weighting scores, we replicated the study in another cohort.

Trial simulations

Based on these calculations, the optimal composite score difference was used to perform simulations of a two-year interventional trial with the change in retinal eccentricity as primary outcome measured by the composite biomarker. By introducing a potential treatment effect, we calculated the expected power of a trial with these patients. The treated eye was randomly assigned by a Bernoulli distribution. The progression was then simulated as follows:(10)where(11)

* difference of ΔCi,t(Δti,l+x) is the difference of the change of the composite score from baseline for the ith patient in the jth eye t at x years after the last visit l, compared to the last visit.

* the adjusted RMSE was calculated for residual errors of follow-up measurements without baseline measurements (Δti,0), where Eij is zero in all patients,

* the treatment effect δ was simulated from 0 to 1 by steps of 0.05 in treated eyes.

As the degree of abnormalities are highly correlated between left and right eyes [21, 23] the power can be increased considerably when the fellow eye serves as the paired control. A paired-samples T-test was therefore performed to assess the differences between eyes in progression of changes in retinal eccentricity. Power calculations were performed by 10000 simulations for each data point (S2 Appendix).

Supporting information

[Figure omitted. See PDF.]

S1 Table. Current gene therapy trials for retinal dystrophies, including expected number of patients enrolled, timeframe and endpoints that assess efficacy.

https://doi.org/10.1371/journal.pone.0174020.s001

(PDF)

S1 Dataset. Functional and structural measurements from the development (Radboud university medical center) and external validation (Moorfields Eye Hospital) cohort.

https://doi.org/10.1371/journal.pone.0174020.s002

(XLSX)

S2 Dataset. Outcome of all combinations of weighting scores.

https://doi.org/10.1371/journal.pone.0174020.s003

(XLSX)

S1 Appendix. Syntax for the weighting scores optimization.

https://doi.org/10.1371/journal.pone.0174020.s004

(PDF)

S2 Appendix. Syntax for the trial simulations.

https://doi.org/10.1371/journal.pone.0174020.s005

(PDF)

Acknowledgments

We thank Frans P.M. Cremers (Department of Human Genetics, Radboud university medical center, Nijmegen, The Netherlands) and John (Pei-Wen) Chiang (Casey Eye Institute, Portland, OR, U.S.A.) for performing the genetic analysis, and Willem Woertman (former affiliation: Department for Health Evidence, Radboud university medical center, Nijmegen) and Jelle J. Goeman (Department of Medical Statistics and Bioinformatics, Leiden University Medical Center, Leiden, The Netherlands) for their fruitful discussions that helped to develop the methods used in this study.

Author Contributions

1. Conceptualization: SL NMB BJK CBH GJVDW.

2. Data curation: SL NMB AF JMMG.

3. Formal analysis: SL JMMG GJVDW.

4. Funding acquisition: BJK ATM MM ARW GJVDW CBH.

5. Investigation: SL NMB AF.

6. Methodology: SL JMMG GJVDW.

7. Project administration: SL NMB CBH.

8. Resources: SL NMB BJK ATM MM ARW CBH.

9. Software: JMMG.

10. Supervision: ARW GJVDW CBH.

11. Validation: AF ATM MM ARW.

12. Visualization: SL JMMG.

13. Writing – original draft: SL.

14. Writing – review & editing: NMB AF JMMG BJK MM ARW GJVDW CBH.

Citation: Lambertus S, Bax NM, Fakin A, Groenewoud JMM, Klevering BJ, Moore AT, et al. (2017) Highly sensitive measurements of disease progression in rare disorders: Developing and validating a multimodal model of retinal degeneration in Stargardt disease. PLoS ONE 12(3): e0174020. https://doi.org/10.1371/journal.pone.0174020

References

1. Berger W, Kloeckener-Gruissem B, Neidhardt J. The molecular basis of human retinal and vitreoretinal diseases. Prog Retin Eye Res. 2010;29(5):335–375. pmid:20362068

2. Liew G, Michaelides M, Bunce C. A comparison of the causes of blindness certifications in England and Wales in working age adults (16–64 years), 1999–2000 with 2009–2010. BMJ Open. 2014;4(2):e004015. pmid:24525390

3. Kaplan J, Gerber S, Larget-Piet D, Rozet JM, Dollfus H, Dufier JL, et al. A gene for Stargardt's disease (fundus flavimaculatus) maps to the short arm of chromosome 1. Nat Genet. 1993;5(3):308–311. Epub 1993/11/01. pmid:8275096

4. Fujinami K, Zernant J, Chana RK, Wright GA, Tsunoda K, Ozawa Y, et al. Clinical and molecular characteristics of childhood-onset Stargardt disease. Ophthalmology. 2015;122(2):326–334. Epub 2014/10/15. pmid:25312043

5. Lambertus S, van Huet RA, Bax NM, Hoefsloot LH, Cremers FP, Boon CJ, et al. Early-onset Stargardt disease: phenotypic and genotypic characteristics. Ophthalmology. 2015;122(2):335–344. Epub 2014/12/03. pmid:25444351

6. Westeneng-van Haaften SC, Boon CJ, Cremers FP, Hoefsloot LH, den Hollander AI, Hoyng CB. Clinical and genetic characteristics of late-onset Stargardt's disease. Ophthalmology. 2012;119(6):1199–1210. Epub 2012/03/28. pmid:22449572

7. Fujinami K, Sergouniotis PI, Davidson AE, Wright G, Chana RK, Tsunoda K, et al. Clinical and molecular analysis of Stargardt disease with preserved foveal structure and function. Am J Ophthalmol. 2013;156(3):487–501.e481. Epub 2013/08/21. pmid:23953153

8. Armstrong JD, Meyer D, Xu S, Elfervig JL. Long-term follow-up of Stargardt's disease and fundus flavimaculatus. Ophthalmology. 1998;105(3):448–457; discussion 457–448. Epub 1998/03/21. pmid:9499775

9. Binley K, Widdowson P, Loader J, Kelleher M, Iqball S, Ferrige G, et al. Transduction of photoreceptors with equine infectious anemia virus lentiviral vectors: safety and biodistribution of StarGen for Stargardt disease. Invest Ophthalmol Vis Sci. 2013;54(6):4061–4071. Epub 2013/04/27. pmid:23620430

10. Schwartz SD, Regillo CD, Lam BL, Eliott D, Rosenfeld PJ, Gregori NZ, et al. Human embryonic stem cell-derived retinal pigment epithelium in patients with age-related macular degeneration and Stargardt's macular dystrophy: follow-up of two open-label phase 1/2 studies. Lancet. 2015;385(9967):509–516. Epub 2014/12/03. pmid:25458728

11. Radu RA, Mata NL, Nusinowitz S, Liu X, Sieving PA, Travis GH. Treatment with isotretinoin inhibits lipofuscin accumulation in a mouse model of recessive Stargardt's macular degeneration. Proc Natl Acad Sci U S A. 2003;100(8):4742–4747. Epub 2003/04/03. pmid:12671074

12. Charbel Issa P, Barnard AR, Herrmann P, Washington I, MacLaren RE. Rescue of the Stargardt phenotype in Abca4 knockout mice through inhibition of vitamin A dimerization. Proc Natl Acad Sci U S A. 2015.

13. Csaky KG, Richman EA, Ferris FL 3rd. Report from the NEI/FDA Ophthalmic Clinical Trial Design and Endpoints Symposium. Invest Ophthalmol Vis Sci. 2008;49(2):479–489. pmid:18234989

14. Rotenstreich Y, Fishman GA, Anderson RJ. Visual acuity loss and clinical observations in a large series of patients with Stargardt disease. Ophthalmology. 2003;110(6):1151–1158. Epub 2003/06/12. pmid:12799240

15. Cukras CA, Wong WT, Caruso R, Cunningham D, Zein W, Sieving PA. Centrifugal expansion of fundus autofluorescence patterns in Stargardt disease over time. Arch Ophthalmol. 2012;130(2):171–179. Epub 2011/10/12. pmid:21987580

16. Cideciyan AV, Swider M, Schwartz SB, Stone EM, Jacobson SG. Predicting Progression of ABCA4-Associated Retinal Degenerations Based on Longitudinal Measurements of the Leading Disease Front. Invest Ophthalmol Vis Sci. 2015;56(10):5946–5955. Epub 2015/09/18. pmid:26377081

17. Kellner S, Kellner U, Weber BH, Fiebig B, Weinitz S, Ruether K. Lipofuscin- and melanin-related fundus autofluorescence in patients with ABCA4-associated retinal dystrophies. Am J Ophthalmol. 2009;147(5):895–902, 902.e891. Epub 2009/02/27. pmid:19243736

18. Smith RT, Gomes NL, Barile G, Busuioc M, Lee N, Laine A. Lipofuscin and autofluorescence metrics in progressive STGD. Invest Ophthalmol Vis Sci. 2009;50(8):3907–3914. Epub 2009/04/24. pmid:19387078

19. Lois N, Halfyard AS, Bird AC, Holder GE, Fitzke FW. Fundus autofluorescence in Stargardt macular dystrophy-fundus flavimaculatus. Am J Ophthalmol. 2004;138(1):55–63. Epub 2004/07/06. pmid:15234282

20. Kuehlewein L, Hariri AH, Ho A, Dustin L, Wolfson Y, Strauss RW, et al. Comparison of manual and semiautomated fundus autofluorescence analysis of macular atrophy in Stargardt disease phenotype. Retina. 2016;36(6):1216–1221. Epub 2015/11/20. pmid:26583307

21. McBain VA, Townend J, Lois N. Progression of retinal pigment epithelial atrophy in Stargardt disease. Am J Ophthalmol. 2012;154(1):146–154. Epub 2012/04/03. pmid:22464366

22. Fujinami K, Lois N, Mukherjee R, McBain VA, Tsunoda K, Tsubota K, et al. A longitudinal study of Stargardt disease: quantitative assessment of fundus autofluorescence, progression, and genotype correlations. Invest Ophthalmol Vis Sci. 2013;54(13):8181–8190. Epub 2013/11/23. pmid:24265018

23. Chen B, Tosha C, Gorin MB, Nusinowitz S. Analysis of autofluorescent retinal images and measurement of atrophic lesion growth in Stargardt disease. Exp Eye Res. 2010;91(2):143–152. Epub 2010/04/20. pmid:20398653

24. Ritter M, Zotter S, Schmidt WM, Bittner RE, Deak GG, Pircher M, et al. Characterization of Stargardt disease using polarization-sensitive optical coherence tomography and fundus autofluorescence imaging. Invest Ophthalmol Vis Sci. 2013;54(9):6416–6425. Epub 2013/07/25. pmid:23882696

25. Ergun E, Hermann B, Wirtitsch M, Unterhuber A, Ko TH, Sattmann H, et al. Assessment of central visual function in Stargardt's disease/fundus flavimaculatus with ultrahigh-resolution optical coherence tomography. Invest Ophthalmol Vis Sci. 2005;46(1):310–316. Epub 2004/12/30. pmid:15623790

26. Gomes NL, Greenstein VC, Carlson JN, Tsang SH, Smith RT, Carr RE, et al. A comparison of fundus autofluorescence and retinal structure in patients with Stargardt disease. Invest Ophthalmol Vis Sci. 2009;50(8):3953–3959. Epub 2009/03/28. pmid:19324865

27. Anastasakis A, Fishman GA, Lindeman M, Genead MA, Zhou W. Infrared scanning laser ophthalmoscope imaging of the macula and its correlation with functional loss and structural changes in patients with Stargardt disease. Retina. 2011;31(5):949–958. Epub 2011/02/05. pmid:21293320

28. Langbaum JB, Hendrix SB, Ayutyanont N, Chen K, Fleisher AS, Shah RC, et al. An empirically derived composite cognitive test score with improved power to track and evaluate treatments for preclinical Alzheimer's disease. Alzheimers Dement. 2014;10(6):666–674. pmid:24751827

29. Cideciyan AV, Swider M, Aleman TS, Tsybovsky Y, Schwartz SB, Windsor EA, et al. ABCA4 disease progression and a proposed strategy for gene therapy. Hum Mol Genet. 2009;18(5):931–941. Epub 2008/12/17. pmid:19074458

30. Anstis SM. Letter: A chart demonstrating variations in acuity with retinal position. Vision Res. 1974;14(7):589–592. pmid:4419807

31. Strauss RW, Ho A, Munoz B, Cideciyan AV, Sahel JA, Sunness JS, et al. The Natural History of the Progression of Atrophy Secondary to Stargardt Disease (ProgStar) Studies: Design and Baseline Characteristics: ProgStar Report No. 1. Ophthalmology. 2016;123(4):817–828. Epub 2016/01/21. pmid:26786511

32. Mazzucato M, Visona Dalla Pozza L, Manea S, Minichiello C, Facchin P. A population-based registry as a source of health indicators for rare diseases: the ten-year experience of the Veneto Region's rare diseases registry. Orphanet J Rare Dis. 2014;9:37. pmid:24646171

33. Walia S, Fishman GA. Natural history of phenotypic changes in Stargardt macular dystrophy. Ophthalmic Genet. 2009;30(2):63–68. Epub 2009/04/18. pmid:19373676

34. Lazow MA, Hood DC, Ramachandran R, Burke TR, Wang YZ, Greenstein VC, et al. Transition zones between healthy and diseased retina in choroideremia (CHM) and Stargardt disease (STGD) as compared to retinitis pigmentosa (RP). Invest Ophthalmol Vis Sci. 2011;52(13):9581–9590. Epub 2011/11/15. pmid:22076985

35. Lee W, Noupuu K, Oll M, Duncker T, Burke TR, Zernant J, et al. The External Limiting Membrane in Early-Onset Stargardt Disease. Invest Ophthalmol Vis Sci. 2014. Epub 2014/08/21.

36. Staurenghi G, Sadda S, Chakravarthy U, Spaide RF, International Nomenclature for Optical Coherence Tomography P. Proposed lexicon for anatomic landmarks in normal posterior segment spectral-domain optical coherence tomography: the IN*OCT consensus. Ophthalmology. 2014;121(8):1572–1578. pmid:24755005

37. Kretschmann U, Seeliger MW, Ruether K, Usui T, Apfelstedt-Sylla E, Zrenner E. Multifocal electroretinography in patients with Stargardt's macular dystrophy. Br J Ophthalmol. 1998;82(3):267–275. Epub 1998/05/29. pmid:9602623

38. Kuniyoshi K, Terasaki H, Arai M, Hirose T. Multifocal Electroretinograms in Stargardt's Disease/Fundus Flavimaculatus. Ophthalmologica. 2014. Epub 2014/06/28.

39. Praidou A, Hagan R, Newman W, Chandna A. Early diagnosis of Stargardt disease with multifocal electroretinogram in children. Int Ophthalmol. 2014;34(3):613–621. Epub 2013/06/14. pmid:23760584

40. Teussink MM, Lee MD, Smith RT, van Huet RA, Klaver CC, Klevering BJ, et al. The effect of light deprivation in patients with Stargardt disease. Am J Ophthalmol. 2015;159(5):964–972.e962. Epub 2015/02/15. pmid:25681002

41. Drasdo N, Fowler CW. Non-linear projection of the retinal image in a wide-angle schematic eye. Br J Ophthalmol. 1974;58(8):709–714. pmid:4433482

42. Dreyhaupt J, Mansmann U, Pritsch M, Dolar-Szczasny J, Bindewald A, Holz FG. Modelling the natural history of geographic atrophy in patients with age-related macular degeneration. Ophthalmic Epidemiol. 2005;12(6):353–362. pmid:16283987