Is the Concept of Quality of Life Relevant for Multiple Sclerosis Patients with Cognitive Impairment? Preliminary Results of a Cross-Sectional Study

Abstract

Background

Cognitive impairment occurs in about 50% of multiple sclerosis (MS) patients, and the use of self-reported outcomes for evaluating treatment and managing care among subjects with cognitive dysfunction has been questioned. The aim of this study was to provide new evidence about the suitability of self-reported outcomes for use in this specific population by exploring the internal structure, reliability and external validity of a specific quality of life (QoL) instrument, the Multiple Sclerosis International Quality of Life questionnaire (MusiQoL).

Methods

Design: cross-sectional study. Inclusion criteria: MS patients of any disease subtype. Data collection: sociodemographic (age, gender, marital status, education level, and occupational activity) and clinical data (MS subtype, Expanded Disability Status Scale, disease duration); QoL (MusiQoL and SF36); and neuropsychological performance (Stroop color-word test). Statistical analysis: confirmatory factor analysis, item-dimension correlations, Cronbach's alpha coefficients, Rasch statistics, relationships between MusiQoL dimensions and other parameters.

Principal Findings

One hundred and twenty-four consecutive patients were enrolled. QoL scores did not differ between the 69 cognitively non-impaired patients and the 55 cognitively impaired patients, except for the symptoms dimension. The confirmatory factor analysis performed among the impaired subjects showed that the structure of the questionnaire matched with the initial structure of the MusiQoL. The unidimensionality of the MusiQoL dimensions was preserved, and the internal validity indices were satisfactory and close to those of the reference population.

Conclusions/Significance

Our study suggests that executive dysfunction did not compromise the reliability and the validity of the self-reported QoL questionnaires.

About the Authors:

Karine Baumstarck

* E-mail: [email protected]

Affiliation: EA3279 Self-Perceived Health Assessment Research Unit and Department of Public Health, Nord University Hospital, APHM, Marseille, France

Jean Pelletier

Affiliation: Departments of Neurology and CRMBM CNRS6612, Timone University Hospital, APHM, Marseille, France

Valérie Aghababian

Affiliation: EA 3273 Psychology of Cognition, Language, and Emotion Research Centre, Aix-Marseille University, Aix-en-Provence, France

Françoise Reuter

Affiliation: Departments of Neurology and CRMBM CNRS6612, Timone University Hospital, APHM, Marseille, France

Irina Klemina

Affiliation: Departments of Neurology and CRMBM CNRS6612, Timone University Hospital, APHM, Marseille, France

Julie Berbis

Affiliation: EA3279 Self-Perceived Health Assessment Research Unit and Department of Public Health, Nord University Hospital, APHM, Marseille, France

Anderson Loundou

Affiliation: EA3279 Self-Perceived Health Assessment Research Unit and Department of Public Health, Nord University Hospital, APHM, Marseille, France

Pascal Auquier

Affiliation: EA3279 Self-Perceived Health Assessment Research Unit and Department of Public Health, Nord University Hospital, APHM, Marseille, France

Introduction

Measures of health-related quality of life (HRQoL) are being used with increasing frequency in the treatment of multiple sclerosis (MS) as an outcome measure for assessing disease progression, evaluating treatment and managing care [1], [2]. While regulatory authorities and clinicians request this type of information, HRQoL remains rarely used in clinical practice to adjust the management of the patient care because assessment of HRQoL is suspected of containing some limitations [3].

The use of self-reported outcomes among subjects with cognitive dysfunction is of particular concern [3]. While cognitive impairment occurs in about 50% of MS patients [4], [5], even during the early stages of the disease [6], [7], the extent to which MS patients with cognitive dysfunction can validly self-report their quality of life (QoL) is a crucial issue that remains insufficiently examined. The main argument against using self-reported QoL information from patients with cognitive dysfunction was based on the fact that the QoL instruments were not developed among these specific individuals. Although there is a little evidence concerning the reliability and validity of health status measures in cognitively impaired patients [3], two perspectives have been presented. While some authors have argued that individuals with cognitive impairment are not able to produce valid QoL measures [4], [8], others reported some empirical evidence suggesting that individuals with a moderate degree of cognitive impairment can perform reliable HRQoL assessments [9]–[11]. Most of the studies provided information about patients with severe mental disorders [12]–[15] or older populations [16] presenting with dementia or other severe cognitive impairment [16]–[18]. To our knowledge, only two main studies have reported data from MS patients [10], [11]. These results suggested that cognitive decline does not compromise the reliable and valid assessment of self-reported health measures. These studies did not report how the factorial structure described in the impaired samples fit with the initial structure of the tested instrument, which is a key point when considering validity in these specific populations.

To provide new evidence about the suitability for using self-reported QoL information in this specific population, we propose to explore the internal structure, reliability and external validity of a specific QoL instrument, the Multiple Sclerosis International Quality of Life questionnaire (MusiQoL), exclusively developed from the patients' point-of-view [19]. The study sample includes MS subjects with or without cognitive impairment. The MusiQoL is a self-administered, disease-specific QoL instrument that is available in 14 languages [20]–[23].

Methods

This study relied on a cross-sectional design and was performed in the neurology department of a French public academic teaching hospital (Marseille, France). The inclusion criteria were as follows: patient with MS diagnosis according to McDonald criteria [24], any disease subtype, no history of psychiatric or neurological disease (other than MS), no history of alcohol/drug abuse, and native French speaker. The French Ethics Committee (Comité de Protection des Personnes Marseille II) approved the study, and patients gave their informed consent to participate. Sociodemographic (age, gender, marital status, education level, and occupational activity) and clinical (MS subtype and disease duration) data for each patient were recorded. The MS disability was assessed using the Expanded Disability Status Scale (EDSS).

HRQoL was assessed by means of the MusiQoL. The MusiQoL is a well-validated questionnaire that describes the following nine dimensions and yields a global index score: activity of daily living (ADL), psychological well-being (PWB), symptoms (SPT), relationships with friends (RFr), relationships with family (RFa), relationships with health care system (RHCS), sentimental and sexual life (SSL), coping (COP), and rejection (REJ). HRQoL assessment was completed using the Short Form 36 (SF36), which is a generic questionnaire [25] describing eight subscales (physical function, social functioning, role physical, role emotional, mental health, vitality, bodily pain, and general health). Two composite scores (physical and mental, PCS-SF36 and MCS-SF36) were also calculated.

Neuropsychological performance was assessed using the carded-based version of the Stroop color-word test [26]. We used the more widespread version including 3 subtests: 1) the color naming subtest where the subject was instructed to name the color of a string of dots; 2) the word naming subtest where the subject was instructed to read a list of words indicating colors printed in black letters; and 3) the color-word naming subtest where the subject had to name the color of the letters of color words printed in different colors. Performance was assessed by calculating the time required to name 100 items in each trial (higher scores indicate worse performance). The test was administered in a standardized manner by the same psychologist (FR) who was intensively trained in test administration. The same instructions were given to the subjects prior to each trial.

For each subtest, the subject was defined as impaired or non-impaired by applying French normative values [27] according to age and educational level. Patients were categorized into the following categories according to cognitive function as measured by the Stroop test: a) cognitively non-impaired group (3 normal subtests); b) cognitively impaired group (one or more abnormal subtests).

Statistical analyses

Statistical analyses were performed to explore the internal structure, reliability and external validity of the MusiQoL. The exploration of the psychometric properties of a questionnaire was largely described in the specific literature. The definitions of the main psychometric properties were summarized in the Figure 1. Statistical analyses were performed on the two groups defined above using the same procedure reported in the initial validation publication (reference population), except for factor analysis (confirmatory instead exploratory).

[Figure omitted. See PDF.]

Figure 1. Psychometric properties of a QoL questionnaire: definitions.

https://doi.org/10.1371/journal.pone.0030627.g001

The structures of the MusiQoL, both in the non-impaired and impaired groups, were explored using confirmatory factor analysis to determine how these structures matched with the initial structure of the MusiQoL issued of a principal component factor analyses with varimax rotation. Kaiser-Meyer-Olkin (KMO) measures of sample adequacy of the model for the residual matrices were computed: if the KMO index was higher than 0.50, then a factor analysis of the residual matrix was performed.

The multidimensional structure (construct validity) of the version was checked using the multi-trait/multi-item analysis program [28]. Internal structural validity was assessed by investigating item-dimension correlations. Item internal consistency (IIC) was assessed by correlating each item with its scale, and item discriminant validity (IDV) was assessed by determining the extent to which items correlated with the dimension they were hypothesized to represent as compared to correlations with other dimensions. Floor and ceiling effects were reported to assess the homogeneous repartition of the response distribution (effects lower than 10% are expected). For each dimension, internal consistency reliability was evaluated by Cronbach's alpha coefficient [29]; the values of which were compared between the non-impaired and impaired groups using the alpha test program [30].

The unidimensionality of each scale was explored by computation of item goodness-of-fit statistics (INFIT) issued from Rasch analyses [31]. INFIT values ranging from 0.7 to 1.2 ensure that all the items of the scale tend to measure the same concept. Differential item functioning (DIF) analyses were performed, comparing the item difficulties between the two groups according to the cognitive status (non-impaired, impaired) to check whether all the items behave the same way [31]. DIF means that an item performs and measures differently for one subgroup of a population than for another.

To explore external validity, Spearman's correlation coefficients were used to investigate relationships between dimensions of the MusiQoL and SF36 in each group, and the associations between the MusiQoL dimension scores and sociodemographic and clinical features were reported. For qualitative variables, mean dimension scores of the MusiQoL were compared across patient groups that were expected to differ (e.g., gender, educational level, marital status, and occupational status) using one-way analysis of variance. Quantitative variables (e.g., age, EDSS score, and MS duration) were analyzed using Spearman's correlation coefficients. The underlying assumption was that the strength of the relationships would be similar for both groups (non-impaired and impaired) and the reference population. Comparisons of correlation coefficients were performed [32].

Acceptability was assessed by calculating the percentage of missing data per dimension.

Data analyses were performed using SPSS 11.0, MAP-R, LISREL and WINSTEP software.

Results

One hundred and twenty-four consecutive patients were enrolled. The mean age was 45 years (SD 11), 57.3% of the patients were women, and 47.2% had more than 12 years of education. The MS subtypes included 61 secondary progressive, 36 relapsing remitting, 20 primary progressive, and 7 clinically isolated syndromes. From the French normative values [27], performances on Stroop subtests varied from 24 to 28% (24.1% impaired for the color naming subtest, 26.5% for the word naming subtest, and 28.0% for the color-word naming subtest). The definition of cognitive status classified 69 patients as cognitively non-impaired and 55 (44.3%, 95% confidence interval 35.6–53.0) as cognitively impaired.

MusiQoL scores

The mean dimension scores and indices did not differ between the non-impaired and impaired subjects except for the symptoms dimension, with higher scores among the non-impaired subjects (Figure 2). Missing values were higher in the impaired group but never exceeded 10% (range from 4.8 to 10.0%). Details are presented in table 1.

[Figure omitted. See PDF.]

Figure 2. Means of dimension/index scores of MusiQoL according to the cognitive status.

https://doi.org/10.1371/journal.pone.0030627.g002

[Figure omitted. See PDF.]

Table 1. Internal structural validity/reliability/unidimensionality.

https://doi.org/10.1371/journal.pone.0030627.t001

Construct validity

The 9-factor structure of the MusiQoL accounted for 73.4% of the total variance among the non-impaired patients and for 77.3% among the impaired patients.

In the non-impaired group, the 9-factor structure was clearly retrieved. Only 3 of the 31 items contributed to a second factor without being major contributors. In the impaired group, only 8 of the 9 initial factors were identified. Two items (numbers 28 and 29) that belonged to the rejection dimension in the initial structure contributed to another factor: the psychological well-being dimension, which is close to the rejection dimension. All other items mainly contributed to their initial dimension, except item number 15, which was initially caught by the symptoms dimension. The content analysis of the new isolated factor (factor 4) did not identify a specific meaning, grouping both psychological well-being and rejection dimensions. These structures appear acceptable and are presented in the table S1.

Internal structural validity was satisfactory for all dimensions in the two groups; each item achieved the 0.40 standard for IIC. The correlation for each item with its contributive dimension was higher than with the others (IDV), except for two dimensions (i.e., activity of daily living and psychological well-being) in the impaired group. Floor effects were less than 10%, except in the sentimental and sexual life dimension (18.8% among non-impaired subjects and 22.0% among impaired, respectively). The wrong ceiling effects were produced for the rejection dimension, 25.0 and 44.0% respectively. Cronbach's alpha coefficients ranged from 0.63 to 0.88 in the non-impaired group, and from 0.64 to 0.90 in the impaired group, indicating satisfactory internal consistency. No statistical differences were found between the non-impaired and impaired groups using Cronbach's alpha. For 6 of the 9 dimensions, no items showed an INFIT statistic outside the acceptable range; items were outside the acceptable range for activity of daily living in both groups, for relationships with friends in the non-impaired group, and for psychological well-being in the impaired group. All results are detailed in table 1. According to the definition of DIF, there should be no association between the item and the cognitive status, showing that MusiQoL dimensions are relevant whatever the cognitive status (this was the case only for item number 15 with p<0.05, data not shown).

External and discriminant validity

Spearman's correlation coefficients between MusiQoL and SF36 scores are provided in the table S2. The concepts covered by the MusiQoL and the SF36 are not strictly overlapping. The social functioning domain did not correlate with ‘relationships-like’ dimensions of MusiQoL. As expected, the mental health dimension and mental composite score of the SF36 were mainly statistically associated with the psychological well-being dimension of the MusiQoL, while physical functioning, vitality, bodily pain, general health and physical composite score of SF36 correlated more strongly with the activity of daily living dimension of the MusiQoL, across all cognitive groups. Among the 100 tested correlations, only 80% of them were not statistically different between the non-impaired and impaired groups (results not reported). As expected, few significant correlations were found between MusiQoL scores and MS duration or EDSS, except for EDSS, which highly correlated with activity of daily living in the non-impaired group. Contrarily, the age of patients was not linked to activity of daily living. These results are detailed in table 2. As expected, the women in this study reported lower psychological well-being scores than the men, and single subjects reported lower sentimental and sexual life and index scores than subjects having a partner among both non-impaired and impaired individuals (table 3).

[Figure omitted. See PDF.]

Table 2. Correlations between MusiQoL, age and clinical features according to the cognitive status.

https://doi.org/10.1371/journal.pone.0030627.t002

[Figure omitted. See PDF.]

Table 3. Associations between MusiQoL dimension scores and sociodemographic characteristics according to the cognitive status.

https://doi.org/10.1371/journal.pone.0030627.t003

Discussion

While the assessment of quality of life in MS has received increasing recognition as an outcome parameter in MS research, one should consider whether self-reported information remains reliable when patients experience cognitive problems and to what extent HRQoL measurement remains valid in such a context. Therefore, it seems absolutely necessary to check if the initial internal structure of the self-reported measure is well adapted when HRQoL measures will be used for cognitively impaired individuals and to confirm if the psychometric properties are satisfactory in these populations [3].

Our results provide strong arguments to support the conclusion that cognitively impaired MS patients, as defined from an executive dysfunction, are reliable and consistent when answering the MusiQoL questionnaire. First, the confirmatory factor analysis showed that the structure performed among the impaired subjects almost matched with the initial structure of the MusiQoL. Overall, 8 of the 9 dimensions were clearly identified. Items describing the predefined rejection dimension mainly contributed to the psychological well-being dimension. The limitation regarding the relative small size of the sample and the meaning of the items describing this rejection dimension which are not so fairly distant to the items constituting the psychological well-being dimension should be noted. This last point can be supported by the examination of the moderate correlation between the 2 dimensions issued of the initial validation (r = 0.39, p<0.001, data not shown in the initial publication) [19]. However, the unidimensionality of each of these dimensions seemed ensured by the satisfactory INFIT statistics. Moreover, IIC and IDV values reported in the impaired group were very close to those of the reference population, and similar to those of the non-impaired sample. Internal consistency coefficients, despite the patient's cognitive status, were near to the initial reference population, except for the coping dimension (which presented a less satisfactory coefficient). Floor and ceiling effects were similar to those reported in the initial validation publication, except for the floor effect which was higher for sentimental and sexual life in both the impaired and non-impaired groups. This was probably due to the specificity of the French sample, whereas reference values were issued from patients from 14 countries including North-American subjects. Indeed, it is now well-known that the populations of south Europe more easily accept sexuality as a normal part of life than do North-American populations [33]. Lastly, no difference was found for item functioning, whatever the cognitive status, indicating the relevance of the structure.

Otherwise, the MusiQoL scores of both groups were consistent with those of the SF36 as compared to the reference population. As expected, activity of daily living was strongly linked to the ‘physical-like’ dimensions of SF36 (including the physical composite score), and psychological well-being was highly correlated to the ‘psychological-like’ dimensions of SF36 (including the mental composite score). These findings support the validity of the MusiQoL in altered and non-altered patients adding information not covered by the generic questionnaires [34].

However, some limitations should be considered. The sample size was small but similar to other studies [10], [11]. The representativeness of our sample should also be noted. Our patients had a more severe disability profile and a higher proportion of secondary progressive disease as compared to international and European MS populations [19], [35]. However, the proportion of cognitively impaired subjects, 44%, was in accordance with the literature [4], [5] and was similar to other studies with like objectives [10], [11]. Nevertheless, the present study did not focus on the most severe cases because patients with dementia or those unable to be assessed using neuropsychological tests were not included.

Another important aspect of this study regards our definition of cognitive dysfunction. Indeed, cognition can be defined as a mental process of knowing, including aspects such as awareness, perception, reasoning, and judgment. Several theoretical approaches to the definition of cognitive impairment can be implemented [36]. The ‘all or none’ approach is based on existence or absence of an abnormality. Another approach is an epidemiologically (or ‘categorically’) based approach [37], which determines that cognitive domains affected in MS patients may be similar between individuals. We arbitrarily restricted cognitive function to its composite executive function to produce additional insight as compared to the two main studies reporting similar data, which focused on memory assessment. Cognitive impairment was defined from the Symbol Digit Modalities Test (SDMT) [11] and from both the Wechsler Adult Intelligence Scale (WAIS-III) and the Wechsler Memory Scale (WMS-III) [10]. Considering just one composite would not have been a perfect reflection of a global cognitive function. It would have been misleading to assume that our patients were not suffering from other neuropsychological deficits [38]. It has been well documented in previous studies that it would be unusual to observe executive deficits in isolation [39], [40] and that HRQoL measurement may be altered differently depending on the kind of cognitive impairment in patients [41]. Executive dysfunction is a frequent finding in MS patients, even at the early stage of the disease. On a psychometric point of view, a recent meta-analysis reported than Stroop word and color test was a more sensitive task to detect executive dysfunction in MS [42]. Future studies could provide further information according to other definitions of cognitive dysfunction integrating combination of different composites (i.e., memory, attention, and concentration).

In the same way as defining executive dysfunction from one test, the Stroop test can also be biased. Because the test requires the use of different cognitive functions such as memory, concentration and executive functions, individuals with different incapacities can be categorized in the same group although they do not present the same deficit [43]. First, while this test is recognized as a good performance tool to assess inhibition ability, general speed of processing [27], and attention performance [44], executive function can include other components such as working memory, initiation and inhibition of responses, strategy planning and conceptual activity, which are insufficiently assessed by the Stroop test. Second, while this test is considered as a standardized neuropsychological instrument, several procedural variations and performance reports are available leading to various interpretations [45], [46]. Our choice to implement this test relied on the following points: i) the high sensitivity of the test [47]; ii) the recent availability of French norms, taking account age and educational level effects [48], eliminating the need for a control group [49]; iii) the existing relationships between Stroop performance and QoL [43]; iv) the frequency of impairment of the Stroop performance in MS population [50]; and v) the cultural robustness, including a French language version [51].

Our study confirms preliminary results reported from two similar previous studies using different QoL measurements and suggesting that executive dysfunction did not compromise the reliable and valid assessment of self-reported health measures. These robust results will be confirmed by performing other cognitive composites, such as memory or attention, among more severely affected individuals. If these findings will be confirmed, assessment of QoL in MS patients could be more widely used without fear of inadequacy of this approach in those patients with cognitive impairment.

Supporting Information

[Figure omitted. See PDF.]

Table S1.

KMO Kaiser-Meyer-Olkin index. ADL activity of daily living, PWB psychological well-being, RFr relationships with friends, SPT symptoms, RFa relationships with family, RHCS relationships with health care system, SSL sentimental and sexual life, COP coping, REJ rejection. Factor loadings lower than 0.30 are not reported.

https://doi.org/10.1371/journal.pone.0030627.s001

(DOC)

Table S2.

ADL activity of daily living, PWB psychological well-being, RFr relationships with friends, SPT symptoms, RFa relationships with family, RHCS relationships with health care system, SSL sentimental and sexual life, COP coping, REJ rejection. MCS mental composite score, PCS physical composite score. NI non-impaired, I impaired, Ref reference population. Spearman rank correlation coefficients were presented. Bold values: p<0,05, *p-value <0,05, **p-value <0,01. Italic characters: reference population values.

https://doi.org/10.1371/journal.pone.0030627.s002

(DOC)

Acknowledgments

The authors are grateful to all the patients for their participation in the study. The authors thank Mohamed Boucekine for his consequent contribution for the statistical analyses.

Author Contributions

Conceived and designed the experiments: JP PA. Performed the experiments: VA FR IK JP. Analyzed the data: KB AL PA. Contributed reagents/materials/analysis tools: KB JP VA FR JB. Wrote the paper: KB PA.

Citation: Baumstarck K, Pelletier J, Aghababian V, Reuter F, Klemina I, Berbis J, et al. (2012) Is the Concept of Quality of Life Relevant for Multiple Sclerosis Patients with Cognitive Impairment? Preliminary Results of a Cross-Sectional Study. PLoS ONE 7(1): e30627. https://doi.org/10.1371/journal.pone.0030627

References

1. Mitchell AJ, Benito-Leon J, Gonzalez JM, Rivera-Navarro J (2005) Quality of life and its assessment in multiple sclerosis: integrating physical and psychological components of wellbeing. Lancet Neurol 4: 556–566.AJ MitchellJ. Benito-LeonJM GonzalezJ. Rivera-Navarro2005Quality of life and its assessment in multiple sclerosis: integrating physical and psychological components of wellbeing.Lancet Neurol4556566

2. Solari A (2005) Role of health-related quality of life measures in the routine care of people with multiple sclerosis. Health Qual Life Outcomes 3: 16.A. Solari2005Role of health-related quality of life measures in the routine care of people with multiple sclerosis.Health Qual Life Outcomes316

3. Riemsma RP, Forbes CA, Glanville JM, Eastwood AJ, Kleijnen J (2001) General health status measures for people with cognitive impairment: learning disability and acquired brain injury. Health Technol Assess 5: 1–100.RP RiemsmaCA ForbesJM GlanvilleAJ EastwoodJ. Kleijnen2001General health status measures for people with cognitive impairment: learning disability and acquired brain injury.Health Technol Assess51100

4. Goverover Y, Chiaravalloti N, DeLuca J (2005) The relationship between self-awareness of neurobehavioral symptoms, cognitive functioning, and emotional symptoms in multiple sclerosis. Mult Scler 11: 203–212.Y. GoveroverN. ChiaravallotiJ. DeLuca2005The relationship between self-awareness of neurobehavioral symptoms, cognitive functioning, and emotional symptoms in multiple sclerosis.Mult Scler11203212

5. Amato MP, Zipoli V, Portaccio E (2006) Multiple sclerosis-related cognitive changes: a review of cross-sectional and longitudinal studies. J Neurol Sci 245: 41–46.MP AmatoV. ZipoliE. Portaccio2006Multiple sclerosis-related cognitive changes: a review of cross-sectional and longitudinal studies.J Neurol Sci2454146

6. Feuillet L, Reuter F, Audoin B, Malikova I, Barrau K, et al. (2007) Early cognitive impairment in patients with clinically isolated syndrome suggestive of multiple sclerosis. Mult Scler 13: 124–127.L. FeuilletF. ReuterB. AudoinI. MalikovaK. Barrau2007Early cognitive impairment in patients with clinically isolated syndrome suggestive of multiple sclerosis.Mult Scler13124127

7. Achiron A, Barak Y (2003) Cognitive impairment in probable multiple sclerosis. J Neurol Neurosurg Psychiatry 74: 443–446.A. AchironY. Barak2003Cognitive impairment in probable multiple sclerosis.J Neurol Neurosurg Psychiatry74443446

8. Benedict RH, Cox D, Thompson LL, Foley F, Weinstock-Guttman B, et al. (2004) Reliable screening for neuropsychological impairment in multiple sclerosis. Mult Scler 10: 675–678.RH BenedictD. CoxLL ThompsonF. FoleyB. Weinstock-Guttman2004Reliable screening for neuropsychological impairment in multiple sclerosis.Mult Scler10675678

9. Auquier P, Simeoni MC, Sapin C, Reine G, Aghababian V, et al. (2003) Development and validation of a patient-based health-related quality of life questionnaire in schizophrenia: the S-QoL. Schizophr Res 63: 137–149.P. AuquierMC SimeoniC. SapinG. ReineV. Aghababian2003Development and validation of a patient-based health-related quality of life questionnaire in schizophrenia: the S-QoL.Schizophr Res63137149

10. Marrie RA, Miller DM, Chelune GJ, Cohen JA (2003) Validity and reliability of the MSQLI in cognitively impaired patients with multiple sclerosis. Mult Scler 9: 621–626.RA MarrieDM MillerGJ CheluneJA Cohen2003Validity and reliability of the MSQLI in cognitively impaired patients with multiple sclerosis.Mult Scler9621626

11. Gold SM, Schulz H, Monch A, Schulz KH, Heesen C (2003) Cognitive impairment in multiple sclerosis does not affect reliability and validity of self-report health measures. Mult Scler 9: 404–410.SM GoldH. SchulzA. MonchKH SchulzC. Heesen2003Cognitive impairment in multiple sclerosis does not affect reliability and validity of self-report health measures.Mult Scler9404410

12. Orley J, Saxena S, Herrman H (1998) Quality of life and mental illness. Reflections from the perspective of the WHOQOL. Br J Psychiatry 172: 291–293.J. OrleyS. SaxenaH. Herrman1998Quality of life and mental illness. Reflections from the perspective of the WHOQOL.Br J Psychiatry172291293

13. Nishiyama T, Ozaki N (2010) Measurement limit of quality-of-life questionnaires in psychiatric settings. Qual Life Res 19: 25–30.T. NishiyamaN. Ozaki2010Measurement limit of quality-of-life questionnaires in psychiatric settings.Qual Life Res192530

14. Voruganti L, Heslegrave R, Awad AG, Seeman MV (1998) Quality of life measurement in schizophrenia: reconciling the quest for subjectivity with the question of reliability. Psychol Med 28: 165–172.L. VorugantiR. HeslegraveAG AwadMV Seeman1998Quality of life measurement in schizophrenia: reconciling the quest for subjectivity with the question of reliability.Psychol Med28165172

15. Wong JG, Cheung EP, Chen EY, Chan RC, Law CW, et al. (2005) An instrument to assess mental patients' capacity to appraise and report subjective quality of life. Qual Life Res 14: 687–694.JG WongEP CheungEY ChenRC ChanCW Law2005An instrument to assess mental patients' capacity to appraise and report subjective quality of life.Qual Life Res14687694

16. Baro E, Ferrer M, Vazquez O, Miralles R, Pont A, et al. (2006) Using the Nottingham Health Profile (NHP) among older adult inpatients with varying cognitive function. Qual Life Res 15: 575–585.E. BaroM. FerrerO. VazquezR. MirallesA. Pont2006Using the Nottingham Health Profile (NHP) among older adult inpatients with varying cognitive function.Qual Life Res15575585

17. Bureau-Chalot F, Novella JL, Jolly D, Ankri J, Guillemin F, et al. (2002) Feasibility, acceptability and internal consistency reliability of the nottingham health profile in dementia patients. Gerontology 48: 220–225.F. Bureau-ChalotJL NovellaD. JollyJ. AnkriF. Guillemin2002Feasibility, acceptability and internal consistency reliability of the nottingham health profile in dementia patients.Gerontology48220225

18. Hoe J, Katona C, Roch B, Livingston G (2005) Use of the QOL-AD for measuring quality of life in people with severe dementia–the LASER-AD study. Age Ageing 34: 130–135.J. HoeC. KatonaB. RochG. Livingston2005Use of the QOL-AD for measuring quality of life in people with severe dementia–the LASER-AD study.Age Ageing34130135

19. Simeoni M, Auquier P, Fernandez O, Flachenecker P, Stecchi S, et al. (2008) Validation of the Multiple Sclerosis International Quality of Life questionnaire. Mult Scler 14: 219–230.M. SimeoniP. AuquierO. FernandezP. FlacheneckerS. Stecchi2008Validation of the Multiple Sclerosis International Quality of Life questionnaire.Mult Scler14219230

20. Flachenecker P, Vogel U, Simeoni MC, Auquier P, Rieckmann P (2011) [MusiQol: international questionnaire investigating quality of life in multiple sclerosis: Validation results for the German subpopulation in an international comparison.]. Nervenarzt. P. FlacheneckerU. VogelMC SimeoniP. AuquierP. Rieckmann2011[MusiQol: international questionnaire investigating quality of life in multiple sclerosis: Validation results for the German subpopulation in an international comparison.].Nervenarzt

21. Beiske AG, Baumstarck K, Nilsen RM, Simeoni MC (2011) Validation of the multiple sclerosis international quality of life (MusiQoL) questionnaire in Norwegian patients. Acta Neurol Scand. AG BeiskeK. BaumstarckRM NilsenMC Simeoni2011Validation of the multiple sclerosis international quality of life (MusiQoL) questionnaire in Norwegian patients.Acta Neurol Scand

22. Baumstarck-Barrau K, Pelletier J, Simeoni MC, Auquier P (2011) [French validation of the Multiple Sclerosis International Quality of Life Questionnaire.]. Rev Neurol (Paris). K. Baumstarck-BarrauJ. PelletierMC SimeoniP. Auquier2011[French validation of the Multiple Sclerosis International Quality of Life Questionnaire.].Rev Neurol (Paris)

23. Triantafyllou N, Triantafillou A, Tsivgoulis G (2009) Validity and Reliability of the Greek Version of the Multiple Sclerosis International Quality-of-Life Questionnaire. J Clin Neurol 5: 173–177.N. TriantafyllouA. TriantafillouG. Tsivgoulis2009Validity and Reliability of the Greek Version of the Multiple Sclerosis International Quality-of-Life Questionnaire.J Clin Neurol5173177

24. Polman CH, Wolinsky JS, Reingold SC (2005) Multiple sclerosis diagnostic criteria: three years later. Mult Scler 11: 5–12.CH PolmanJS WolinskySC Reingold2005Multiple sclerosis diagnostic criteria: three years later.Mult Scler11512

25. Leplege A, Ecosse E, Verdier A, Perneger TV (1998) The French SF-36 Health Survey: translation, cultural adaptation and preliminary psychometric evaluation. J Clin Epidemiol 51: 1013–1023.A. LeplegeE. EcosseA. VerdierTV Perneger1998The French SF-36 Health Survey: translation, cultural adaptation and preliminary psychometric evaluation.J Clin Epidemiol5110131023

26. Jensen AR (1965) Scoring the Stroop test. Acta Psychol (Amst) 24: 398–408.AR Jensen1965Scoring the Stroop test.Acta Psychol (Amst)24398408

27. Godefroy O, et le Groupe de Reflexion pour l'Evaluation des Fonctions EXécutives (GREFEX) (2008) Fonctions exécutives et pathologies neurologiques et psychiatriques. Evaluation en pratique clinique. Marseille, France: Collection Neuropsychologique. O. Godefroyet le Groupe de Reflexion pour l'Evaluation des Fonctions EXécutives (GREFEX)2008Fonctions exécutives et pathologies neurologiques et psychiatriques. Evaluation en pratique cliniqueMarseille, FranceCollection Neuropsychologique

28. Ware JE, Harris WJ, Gandek B, Rogers BW (1997) MAP-R for Windows: Multitrait-Multi-Item Analysis Program - Revised User's Guide. Boston: Health Assessment Lab. JE WareWJ HarrisB. GandekBW Rogers1997MAP-R for Windows: Multitrait-Multi-Item Analysis Program - Revised User's GuideBostonHealth Assessment Lab

29. Cronbach LJ (1951) Coefficient alpha and the internal structure of tests. Psychometrika 16: 297–334.LJ Cronbach1951Coefficient alpha and the internal structure of tests.Psychometrika16297334

30. Lautenschlager GJ (1989) Testing for differences in values of coefficient alpha. Appl Psychol Measurement 13: 284.GJ Lautenschlager1989Testing for differences in values of coefficient alpha.Appl Psychol Measurement13284

31. Wright BD, Stone MH (1979) Best test design: Rasch measurement. Chicago: Mesa press. BD WrightMH Stone1979Best test design: Rasch measurementChicagoMesa press

32. Lowry R (2011) R. Lowry2011Significance of the Difference Between Two Correlation Coefficients http://faculty.vassar.edu/lowry/rdiff.html accessed 23 Apr 2011. Significance of the Difference Between Two Correlation Coefficients http://faculty.vassar.edu/lowry/rdiff.html accessed 23 Apr 2011.

33. David HP, Morgall JM, Osler M, Rasmussen NK, Jensen B (1990) United States and Denmark: different approaches to health care and family planning. Stud Fam Plann 21: 1–19.HP DavidJM MorgallM. OslerNK RasmussenB. Jensen1990United States and Denmark: different approaches to health care and family planning.Stud Fam Plann21119

34. Patrick DL, Deyo RA (1989) Generic and disease-specific measures in assessing health status and quality of life. Med Care 27: S217–232.DL PatrickRA Deyo1989Generic and disease-specific measures in assessing health status and quality of life.Med Care27S217232

35. Amato MP, Grimaud J, Achiti I, Bartolozzi ML, Adeleine P, et al. (2004) European validation of a standardized clinical description of multiple sclerosis. J Neurol 251: 1472–1480.MP AmatoJ. GrimaudI. AchitiML BartolozziP. Adeleine2004European validation of a standardized clinical description of multiple sclerosis.J Neurol25114721480

36. Achiron A, Barak Y (2006) Cognitive changes in early MS: a call for a common framework. J Neurol Sci 245: 47–51.A. AchironY. Barak2006Cognitive changes in early MS: a call for a common framework.J Neurol Sci2454751

37. Fischer JS, Priore RL, Jacobs LD, Cookfair DL, Rudick RA, et al. (2000) Neuropsychological effects of interferon beta-1a in relapsing multiple sclerosis. Multiple Sclerosis Collaborative Research Group. Ann Neurol 48: 885–892.JS FischerRL PrioreLD JacobsDL CookfairRA Rudick2000Neuropsychological effects of interferon beta-1a in relapsing multiple sclerosis. Multiple Sclerosis Collaborative Research Group.Ann Neurol48885892

38. Baumstarck-Barrau K, Simeoni MC, Reuter F, Klemina I, Aghababian V, et al. (2011) Cognitive function and quality of life in multiple sclerosis patients: a cross-sectional study. BMC Neurol 11: 17.K. Baumstarck-BarrauMC SimeoniF. ReuterI. KleminaV. Aghababian2011Cognitive function and quality of life in multiple sclerosis patients: a cross-sectional study.BMC Neurol1117

39. Rao SM, Leo GJ, Ellington L, Nauertz T, Bernardin L, et al. (1991) Cognitive dysfunction in multiple sclerosis. II. Impact on employment and social functioning. Neurology 41: 692–696.SM RaoGJ LeoL. EllingtonT. NauertzL. Bernardin1991Cognitive dysfunction in multiple sclerosis. II. Impact on employment and social functioning.Neurology41692696

40. Feinstein A, Kartsounis LD, Miller DH, Youl BD, Ron MA (1992) Clinically isolated lesions of the type seen in multiple sclerosis: a cognitive, psychiatric, and MRI follow up study. J Neurol Neurosurg Psychiatry 55: 869–876.A. FeinsteinLD KartsounisDH MillerBD YoulMA Ron1992Clinically isolated lesions of the type seen in multiple sclerosis: a cognitive, psychiatric, and MRI follow up study.J Neurol Neurosurg Psychiatry55869876

41. Benito-Leon J, Morales JM, Rivera-Navarro J (2002) Health-related quality of life and its relationship to cognitive and emotional functioning in multiple sclerosis patients. Eur J Neurol 9: 497–502.J. Benito-LeonJM MoralesJ. Rivera-Navarro2002Health-related quality of life and its relationship to cognitive and emotional functioning in multiple sclerosis patients.Eur J Neurol9497502

42. Prakash RS, Snook EM, Lewis JM, Motl RW, Kramer AF (2008) Cognitive impairments in relapsing-remitting multiple sclerosis: a meta-analysis. Mult Scler 14: 1250–1261.RS PrakashEM SnookJM LewisRW MotlAF Kramer2008Cognitive impairments in relapsing-remitting multiple sclerosis: a meta-analysis.Mult Scler1412501261

43. Gerbaud L, Deffond D, Mulliez A, Benausse F, Vernay D, et al. (2006) [Cognitive impairment and quality of life in multiple sclerosis patients]. Rev Neurol (Paris) 162: 970–979.L. GerbaudD. DeffondA. MulliezF. BenausseD. Vernay2006[Cognitive impairment and quality of life in multiple sclerosis patients].Rev Neurol (Paris)162970979

44. Sartori E, Edan G (2006) Assessment of cognitive dysfunction in multiple sclerosis. J Neurol Sci 245: 169–175.E. SartoriG. Edan2006Assessment of cognitive dysfunction in multiple sclerosis.J Neurol Sci245169175

45. Vitkovitch M, Bishop S, Dancey C, Richards A (2002) Stroop interference and negative priming in patients with multiple sclerosis. Neuropsychologia 40: 1570–1576.M. VitkovitchS. BishopC. DanceyA. Richards2002Stroop interference and negative priming in patients with multiple sclerosis.Neuropsychologia4015701576

46. Pujol J, Vendrell P, Deus J, Junque C, Bello J, et al. (2001) The effect of medial frontal and posterior parietal demyelinating lesions on stroop interference. Neuroimage 13: 68–75.J. PujolP. VendrellJ. DeusC. JunqueJ. Bello2001The effect of medial frontal and posterior parietal demyelinating lesions on stroop interference.Neuroimage136875

47. Lezak M (2004) Neuropsychological assessment. New York: Oxford university press. M. Lezak2004Neuropsychological assessmentNew YorkOxford university press

48. Houx PJ, Jolles J, Vreeling FW (1993) Stroop interference: aging effects assessed with the Stroop Color-Word Test. Exp Aging Res 19: 209–224.PJ HouxJ. JollesFW Vreeling1993Stroop interference: aging effects assessed with the Stroop Color-Word Test.Exp Aging Res19209224

49. Foong J, Rozewicz L, Quaghebeur G, Davie CA, Kartsounis LD, et al. (1997) Executive function in multiple sclerosis. The role of frontal lobe pathology. Brain 120(Pt 1): 15–26.J. FoongL. RozewiczG. QuaghebeurCA DavieLD Kartsounis1997Executive function in multiple sclerosis. The role of frontal lobe pathology.Brain120Pt 11526

50. Denney DR, Lynch SG (2009) The impact of multiple sclerosis on patients' performance on the Stroop Test: processing speed versus interference. J Int Neuropsychol Soc 15: 451–458.DR DenneySG Lynch2009The impact of multiple sclerosis on patients' performance on the Stroop Test: processing speed versus interference.J Int Neuropsychol Soc15451458

51. Moller JT, Cluitmans P, Rasmussen LS, Houx P, Rasmussen H, et al. (1998) Long-term postoperative cognitive dysfunction in the elderly ISPOCD1 study. ISPOCD investigators. International Study of Post-Operative Cognitive Dysfunction. Lancet 351: 857–861.JT MollerP. CluitmansLS RasmussenP. HouxH. Rasmussen1998Long-term postoperative cognitive dysfunction in the elderly ISPOCD1 study. ISPOCD investigators. International Study of Post-Operative Cognitive Dysfunction.Lancet351857861