Dutch Validation of the Self-Evaluation of Negative Symptoms Scale (SNS)

Abstract

Background/objectives: Negative symptoms in schizophrenia spectrum disorders are related to impaired social functioning and lower quality of life, making accurate assessment important. To date, most tools for assessing negative symptoms are observational, which can be influenced by the raters’ experience and opinion. Self-rating scales, like the Self-Evaluation of Negative Symptoms (SNS), could complement observer ratings by adding information from the patient’s perspective. Here, we aim to evaluate the psychometric properties of the Dutch translation of the SNS and the relationship between the SNS and functional outcomes. Methods: The SNS was added to the Pharmacotherapy Monitoring Outcome Survey (PHAMOUS)-protocol for adults with a DSM-5 classification of a disorder in the psychosis spectrum. Internal consistency was assessed by Cronbach’s alpha. Confirmatory factor analysis (CFA) was used to evaluate the construct validity of the five subscales of the SNS. Correlational analyses were performed between the SNS and the Positive and Negative Syndrome Scale (PANSS), the Health of Nation Outcomes Scales (HoNOS), the Global Assessment of Functioning (GAF), Functional Remission tool (FR) and the Manchester Short Assessment of Quality of Life (ManSA). Results: A total of 247 patients participated in this study. Internal consistency was good (α = 0.87). CFA confirmed the five-factor structure of the SNS. The SNS was significantly correlated (all p < 0.001) with the PANSS positive (r = 0.31), PANSS negative (r = 0.33), HoNOS (r = 0.37), FR (r = 0.27) and the ManSA (r = −0.40). Conclusions: The Dutch SNS shows good psychometric properties and is related to functional outcomes and quality of life. The SNS can be valuable in complementing current observational-based instruments, and future research may investigate whether the SNS can be used as a standalone measurement tool for the assessment of negative symptoms.

1. Introduction

Negative symptoms have been regarded as core features of schizophrenia spectrum disorders (SSDs) for a long time and are persistently present in 35–70% of the people with a classification in this spectrum [1,2,3,4,5]. They are considered a major burden and are related to impaired global and social functioning [6,7] and a lower quality of life [8]. Furthermore, negative symptoms are often present during the prodromal phase of SSD and have been characterized as risk factors for developing psychosis [9,10,11]. Therefore, accurate treatment of these negative symptoms is important, but remains challenging. To date, limited effective treatments for negative symptoms are available [12,13].

In 2006, the National Institute of Mental Health (NIMH) initiated a Consensus meeting to examine problems that may interfere with treatment development. Here, it was agreed upon that negative symptoms are considered a distinct therapeutic area, and that factor analyses support five domains: social withdrawal, blunted affect (or reduced emotional range), alogia, avolition and anhedonia [14]. Widely used assessment tools, such as the Positive and Negative Syndrome Scale (PANSS) [15], the Scale for the Assessment of Negative Symptoms (SANS) [16], the Brief Negative Symptom Scale (BNSS) [17] and the Clinical Assessment Interview for Negative Symptoms (CAINS) [18], evaluate negative symptom severity based on ratings by an interviewer and can thus be influenced by the raters’ experience and opinion [19]. Clinician ratings are important since people who experience symptoms are not always capable of recognizing their symptoms [20,21,22], and the personally experienced burden of negative symptoms may differ from observation by the raters. In addition, while people with schizophrenia often express positive and negative emotions to a lesser extent than healthy controls, they experience the same range of these emotions [23,24,25]. Self-rating scales could complement observer-based measurements by including information from the patient’s perspective, which is particularly important as these scales often capture inner, non-observable outcomes that may be less visible to medical staff or caregivers [26]. Furthermore, self-rating scales allow people with an SSD classification to evaluate their functioning and self-reflect on their symptoms. This may improve illness insight and, thereby, improve their involvement and commitment to the therapeutic process [26].

To this end, Dollfus et al. [27] created the Self-Evaluation of Negative Symptoms (SNS), a self-assessment scale for negative symptoms that can be completed in five minutes. The SNS was developed in French and has been translated into 27 languages so far, including English [27], Polish [28], Spanish [29], Arabic [30], Lithuanian [31], Chinese [32], Persian [33] and Turkish [34]. Recently, a large European and multicenter study demonstrated that the SNS has adequate psychometric properties and an association with functional outcomes (as measured with the Personal and Social Performance Scale (PSP)) [35]. In addition, the SNS has demonstrated its validity in screening negative symptoms in subjects with first psychotic episode [36].

The SNS has not been translated to Dutch before. Here, we aim to evaluate the psychometric properties (internal consistency and the construct, convergent, and discriminant validity) of the Dutch translation of the SNS and the relationship between self-rated negative symptoms, observer-rated negative symptoms, functional outcomes and quality of life in patients with SSD.

2. Materials and Methods

2.1. Study Design and Participants

Between 2020 and 2022, the SNS was added to the Pharmacotherapy Monitoring Outcome Survey (PHAMOUS)-protocol. This is an annual screening of mental and physical health of people receiving care in one of the four participating mental health institutions in the North of the Netherlands as a part of regular clinical practice [37]. Data were requested from individuals who (1) have a DSM-5 classification of a psychotic disorder and (2) are 18 years or older. The minimal required sample size of 175 people was based on the recommendation of the COnsensus-based Standards for the selection of health Measurement Instruments (COSMIN) [38]: the number of items of the SNS times was seven after taking into account 25% of missing data. The following demographics were assessed: age, gender, education and daily activities. Education was categorized as lower (no education, primary school or prevocational education), middle (secondary vocational education) and higher education (tertiary vocational education). In accordance with the declaration of Helsinki, patients were informed that aggregated and anonymized data may be used for healthcare optimization and scientific research to improve treatment and guidance [37].

2.2. Measurement Instruments

2.2.1. SNS

The Self-Evaluation of Negative Symptoms (SNS) includes five domains of negative symptoms (i.e., social withdrawal, blunted affect, alogia, avolition and anhedonia) and contains 20 items: 4 items for each domain. There are three possible responses, “strongly agree”, “somewhat agree” and “strongly disagree”, and the total score ranges from 0 to 40, where a higher score indicates more severe symptoms.

The Dutch version was translated from the English version [27] and was thereafter back-translated into French. Discrepancies were discussed with the scale author (Prof. Sonia Dollfus), upon which Prof. Dollfus approved the Dutch translation.

2.2.2. PANSS

Symptom severity was assessed by using the Positive and Negative Syndrome Scale (PANSS), a widely used 30-item semistructured interview with high internal consistency and good construct validity [15]. The PANSS evaluates the presence and severity of positive and negative symptoms and general psychopathology within the past week. The scale consists of seven items regarding positive symptoms, seven for negative symptoms and sixteen for general psychopathology. Rating is based on a seven-point scale, and the total score ranges from 30 to 210, where higher scores indicate more severe symptoms [15].

2.2.3. HoNOS

The Health of the Nation Outcome Scale (HoNOS) is an observational scale consisting of twelve items measuring social and general functioning in the past two weeks. The HoNOS consists of four domains: behavioral problems, impairment, symptomatology and social problems. Each item is rated on a five-point scale ranging from ‘no problem’ to ‘severe to very severe problem’ where higher scores indicate more severe problems [39].

2.2.4. GAF

The Global Assessment of Functioning (GAF) is a numeric observational scale ranging from 0 to 100, which measures global functioning. The level of the psychological, social and occupational functioning of the individual integrates within a single score of global functioning rated by a clinician. A high score indicates better outcomes [40].

2.2.5. Functional Remission

This observational scale is an indication of functional remission in several domains: (1) daily living and self-care, (2) work, study and housekeeping and (3) social contacts. These domains are rated on a three-point scale by a clinician (absence of the problem, presence of the problem that is marginal or covered by support, presence of the problem that is insufficiently covered by support), where lower scores indicate more remission [41].

2.2.6. MANSA

The Manchester Short Assessment of Quality of Life (MANSA) is a self-report questionnaire to assess quality of life. It addresses patients’ satisfaction within several psychosocial domains, including satisfaction with life as a whole, job (or unemployment/retirement), financial situation, number and quality of friendships, leisure activities, accommodation, physical and mental health and safety. The twelve items rated on a 7-point scale, ranging from “couldn’t be worse” to “couldn’t be better” were used for analysis (four items are dichotomous (yes/no) and were excluded for methodological reasons). Total scores range from 12 to 84, and higher scores indicate higher reported quality of life.

2.3. Statistical Analysis

For the internal consistency of the total score and the five subscales of the SNS, Cronbach’s alpha was calculated, where values $\ge$ 0.7 are considered acceptable [42]. Furthermore, we calculated ‘alpha if item deleted’; Cronbach’s alpha was calculated when any of the items were dropped from the SNS, to test whether the deletion of any question would influence the reliability of the (sub)scale. The construct validity of the five-factor structure was evaluated through confirmatory factor analysis (CFA). Social withdrawal, reduced emotional range, alogia, avolition and anhedonia subscores were entered as latent variables. To measure the goodness-of-fit (GOF) of the factor structure, the following criteria were used: the Comparative Fit Index (CFI), the Tucker–Lewis Index (TLI) and the Root Mean Square Error of Approximation (RMSEA). An acceptable fit of the model is suggested by RMSEA values < 0.06 and CFI and TLI values > 0.9 [43]. Correlational analyses between the SNS and PANSS negative subscore was performed to assess convergent validity. Discriminant validity was calculated using the correlation coefficient between the SNS and the PANSS positive subscore. To evaluate the relationship between self-rated negative symptoms, observer-based negative symptoms and functional outcomes, correlational analyses between the SNS, the PANSS negative subscale, the HoNOS total score, GAF, functional remission total score and the ManSA total score were performed. To assess whether correlation coefficients of SNS subdomains with outcomes were statistically different, Fisher Z-transformation was performed. Data were not normally distributed, and therefore, Spearman’s rho correlations were used to check for associations between the different outcome variables. A p-value < 0.05 was considered significant. However, p-values are generally highly significant even for very low correlation coefficients with large sample sizes (N > 200). Therefore, the absolute value of the correlation coefficient is an even more meaningful estimation of the association than with smaller sample sizes. Statistical analysis was performed using R version 4.3.1. [44] by using the ‘psych’ [45] (internal consistency), ‘lavaan’ [46] (confirmatory factor analysis) and ‘naniar’ [47] (handle missing data) packages. If missing data were <25% of a (sub)domain of an outcome variable, mean imputation of this domain was applied to address missing data in our sample (Supplementary Data).

3. Results

3.1. Demographics

In total, 247 people were included in this study. The mean age was 42.0 ± 12.5 years, and 34.4% (N = 85) was female. Data on demographics and outcome variables are provided in Table 1.

3.2. Psychometric Properties

3.2.1. Internal Consistency

The Cronbach’s alpha coefficient of the SNS total scale was 0.87. The coefficients for the different subscales varied between 0.64 and 0.79 (Table 2). Withdrawal of any question did not lead to a higher coefficient for the total scale. In the different subscales, only the withdrawal of items 12 (With friends and family, I want to talk about things but it doesn’t come out) and 20 (I am not interested in having sex) resulted in an increase in the value of Cronbach’s alpha in the alogia (0.71 vs. 0.72) and anhedonia (0.69 vs. 0.72) subscales, respectively.

Furthermore, the intercorrelations of the SNS subscores were all highly significant, ranging from 0.26 to 0.52 (Table 3).

3.2.2. Confirmatory Factor Analysis

The good fit of a five-factor model is supported by TLI and CFI > 0.9 and RMSEA < 0.06. The CFI (0.919) and the TLI (0.904) as well as the RMSEA (0.054) showed an adequate fit for the five-factor structure of the SNS.

3.2.3. Convergent and Discriminant Validity

Correlational analyses between the SNS and the PANSS negative and positive subscale were performed to assess convergent and discriminant validity, respectively. Regarding convergent validity, the total score of the SNS showed a significant but modest positive correlation with the PANSS negative symptom subscale (r = 0.33; p < 0.001) (Table 4). For discriminant validity, the total score of the SNS resulted in a significant positive modest correlation of r = 0.31 (p < 0.001) with the PANSS positive subscale (Table 4).

A total of 61.5% (N = 152) of the people did not score a three or higher (mild symptoms) in any of the seven positive subscale items. To examine to which extent the absence of symptoms influenced the corelation coefficient, a post hoc analysis was performed with a subset of people who scored a three or higher in any of the seven positive subscale items (n = 95). This resulted in a weak non-significant correlation between the SNS total and the PANSS positive subscale (r = 0.066; p = 0.52) (Table 5). Further, the correlation coefficient between the SNS and the PANSS negative subscale remains highly significant.

3.3. Functional Outcomes

The total score of the SNS showed a significant, moderate positive correlation with the HoNOS (r = 0.37; p < 0.001) and a significant weak correlation with FR (r = 0.27; p < 0.001) (Table 4). The negative correlation between the SNS total score and the GAF was weak and not significant (r = −0.12; p = 0.16). The PANSS negative subscale was significantly correlated with the HoNOS (r = 0.44; p < 0.001), FR (r = 0.53; p < 0.001), and the GAF (r = −0.43; p < 0.001) (Table 4). In the different subscales, the correlation coefficient of reduced emotional range was significantly lower than social withdrawal (p = 0.045) and avolition (p = 0.026) in the HoNOS, and significantly lower than social withdrawal (p = 0.016) and avolition (p = 0.044) in the FR (Supplementary Data). The correlation coefficient of alogia was significantly lower than social withdrawal (p = 0.016) and almost significantly lower than avolition (p = 0.050) in the FR (Supplementary Data).

3.4. Quality of Life

The total score of the SNS showed a significant negative moderate correlation with the self-report questionnaire MANSA (r = −0.40; p < 0.001) (Table 4). The negative subscale of the PANSS showed a significant negative moderate correlation with the MANSA (r = −0.35; p < 0.001) (Table 4). In the different subscales, the negative correlation coefficient of avolition was significantly higher than social withdrawal (p < 0.001), reduced emotional range (p < 0.001), and alogia (p = 0.0080) (Supplementary Data). The negative correlation coefficient of anhedonia was significantly higher than social withdrawal (p = 0.025) and reduced emotional range (p = 0.0010) (Supplementary Data).

4. Discussion

Negative symptoms have been regarded as a core feature of SSD and represent an unmet treatment need. Accurate assessment tools that capture the internal experience of people suffering from SSD are highly important. Therefore, we examined the reliability, validity and associations with functional outcomes of the Dutch translation of the SNS as a self-assessment tool for negative symptoms. Our findings indicate that the SNS has good internal consistency, construct validity and discriminant validity, and modest convergent validity, and is related to functional outcome and quality of life.

4.1. Psychometric Properties

4.1.1. Internal Consistency

Reliability analysis showed good internal consistency of the SNS, similar to those of the French [27], Arabic [30], Polish [28], Spanish [29], Turkish [34] and Lithuanian [31] versions, ranging from 0.82 to 0.97. Similarly, looking at the subscales individually (i.e., social withdrawal, alogia, avolition, reduced emotional range and anhedonia), we found good internal consistency, though Cronbach’s alpha for the subscales reduced emotional range and anhedonia was moderate. Two prior SNS validation studies examined the reliability of the individual subscales and found similar results: the Turkish and Spanish version of the SNS reported a lower value of the reduced emotional range subscale compared to the other subscales (α = 0.523 and 0.59, respectively) [29,34]. The reliability may be affected by items 5 (People say I’m not sad or happy and that I’m not often angry) and 8 (it is difficult for people to know how I feel), since these items depend on observation of others and on Theory of Mind (the ability to understand others’ beliefs, emotions, and thoughts), which is often impaired in people with psychotic disorders [48,49]. Next, we found moderate reliability for the anhedonia subscale, again in line with the Turkish and Spanish translation (α = 0.69 and 0.61, respectively). Interestingly, the deletion of item 20 (I am not interested in having sex) increased the value of Cronbach’s alpha. It is possible that other factors also play a role in a decreased interest in sex. Reduced interest in sexual activities may, next to the limited ability to experience pleasure, result from emotional, motivational, and social factors [50], as well as the limited opportunity to engage in sexual relationships, which could explain the increase in reliability of the anhedonia scale after removing this item. Next, the deletion of item 12 (With friends and family, I want to talk about things, but it doesn’t come out) increased the reliability of the alogia scale. However, the subscale including this item still showed adequate reliability and the effect of deleting the question was considered to be minimal and therefore was not removed from the questionnaire. Taken together, despite the moderate reliability of reduced emotional range and anhedonia, significant correlations with the other subscales and good construct validity of the total scale suggest that the factors are valid and confirm the importance of these five subdomains and questions.

4.1.2. Construct Validity

For the construct validity, CFA confirmed the five-factor structure of the SNS. The different scores indicated a good fit, as described in the literature [43]. Together with the Turkish [34], Spanish [29] and Chinese [32] versions and a large European multicenter study [35], our results show good construct validity. This suggests that the SNS is able to distinguish between the five subdomains of negative symptoms and can produce meaningful subscores for each domain. Future research may use the SNS to evaluate the effect of interventions and therapeutic agents on each subdomain individually rather than the effect on negative symptoms as a whole.

4.2. Convergent and Discriminant Validity

For convergent validity, we examined the correlation between the SNS and the PANSS negative subscale. The total score and the different subscores of the SNS were all significantly associated with the PANSS negative subscale. However, the correlation coefficient was modest (r = 0.33). This is consistent with a previous study from several European centers [35], that reported a correlation of 0.37. Possibly, the SNS is a more specific and elaborate measure of negative symptoms than the PANSS, which could contribute to the limited correlation, i.e., there is no full overlap. In addition, the difference in perspective between the clinician and the participant could help explain this modest correlation. The literature suggests that people with schizophrenia experience a similar range of positive and negative emotions as healthy controls, but fail to express these feelings [23,24,25]. This discrepancy in what clinicians report about emotional expressions and what people with schizophrenia actually experience could explain this modest association we found in this study. If so, it would mean that the SNS is able to add information about the internal experience that is lacking from the PANSS. Alternatively, it is possible that self-reporting negative symptoms is particularly challenging for some individuals. While discrepancies between expressed and experienced emotions have been documented, some participants may struggle to recognize or accurately identify their own symptoms. This difficulty in self-awareness could partly explain the low correlation observed between the PANSS negative symptom scale and the SNS. In both scenarios, the self-experience of negative symptoms could offer valuable insights for clinicians, enhancing their understanding and informing treatment approaches.

For discriminant validity, the correlation between the SNS and the PANSS positive subscale was investigated. The significant modest correlation (r = 0.31; p < 0.001) we found was higher compared to other translations: the Arabic (r = −0.11; p = 0.53) [30], Turkish (r = 0.14; p = 0.90) [34] and European multicenter validation study (r = 0.12; p = non-significant) [35] all reported weak non-significant correlations between the total SNS score and the PANSS positive subscale. This may be due to the relatively low mean PANSS positive subscale and SNS scores of our sample (9.79 ± 3.18). Both the SNS scores [27,30,31,33,34,35] and the PANSS positive subscale [34,35] of our sample were the lowest compared to other translations, with 37.68% (N = 78) scoring minimal in the positive domain. It may be that the absence of symptoms influenced the correlation coefficient, so to examine to what extent this was true, a post hoc analysis with a subgroup of people with mild positive symptoms in at least one domain was performed. This resulted in a weak non-significant correlation, which is in line with the other translations, suggesting that the relatively low scores may have affected the discriminant validity of the total sample. Future research in a more heterogeneous sample may be needed to confirm this.

4.3. Functioning and Quality of Life

As previously stated, negative symptoms are related to impaired global and social functioning [6,7]. Here, the total score of the SNS was significantly correlated with the HoNOS and the FR. Interestingly, the correlation between PANSS negative subscale and the HoNOS, GAF, and the FR was stronger than the correlation between the SNS and these instruments. Like the difference in emotional expression, this may be due to the discrepancy between observational measurement tools and self-report questionnaires.

In the different subscales, the strongest correlations in both the FR and the HoNOS are reported in the social withdrawal and avolition domains. This is in line with prior studies, which reported that avolition, defined as the reduced initiation and persistence of goal-directed activity due to reduced motivation [51], is a great determinant of social functioning [52,53,54,55,56]. Additionally, the European SNS validation study found that avolition (together with the PANSS positive subscores, level of education, and the lack of judgment and insight PANSS items) explained 23,4% of the variance of functional outcome [35]. Noteworthy, although the correlation coefficients of social withdrawal and avolition were the strongest, they were not significantly different from some of the subdomains. Further research may be needed to confirm that social withdrawal and avolition are suitable predictors of functioning. Additionally, reduced emotional range and alogia showed the weakest correlation with the HoNOS, and were not significantly associated with the FR. In the HoNOS, the correlation coefficient of reduced emotional range was significantly lower compared to social withdrawal and avolition. In the FR, the correlation coefficient of reduced emotional range was significantly lower than social withdrawal and avolition, and the correlation coefficient of alogia was significantly lower than social withdrawal and almost significantly lower than avolition (p = 0.050). The literature suggests that alogia and blunted affect have similar underpinnings and are frequently clustered together in an ‘expressive deficits factor’, which reflects a reduced emotional expression and a reduced verbal output [57,58]. Several studies reported that expressive deficits are a poor predictor of functional outcome [55,59,60,61,62,63]. Our results support these findings, and this, together with the results from the CFA, suggests that the SNS is able to successfully differentiate between the motivational and expressive domains of negative symptoms and capture these constructs by measuring the internal experience.

For quality of life, we found that the SNS is negatively correlated with the MANSA, supporting the relationship between negative symptoms and quality of life. This is in agreement with a study that found an association between the SNS and health-related quality of life [64]. Moreover, they found that in the different subdomains, avolition was the most predictive of reduced health-related quality of life. Here, in agreement with these results, avolition showed the strongest negative correlation with the MANSA. The correlation coefficient of avolition was significantly higher than social withdrawal, reduced emotional range and alogia, and almost reached significance compared to anhedonia (p = 0.059). Avolition has been proposed as an important factor for determining social functioning, and this indicates that it additionally may be an important factor in determining quality of life. However, the exact mechanisms underlying the relationship between avolition, functioning and quality of life remain unclear. Future research may look into this mechanism. Next, compared to the association between the SNS and the MANSA, the correlation coefficient between the PANSS negative score and the MANSA was less robust. However, the difference in correlations between the SNS and the PANSS negative score with the MANSA was small. Possibly, together with the moderate correlation between the SNS and the PANSS negative subscale, and the differences in relationships between these scales and functional outcome scales, self-report questionnaires can add valuable information which is difficult to detect with observer-based measurement tools. However, since the difference in correlation coefficients was small, a replication in a more heterogeneous sample may be needed to confirm this hypothesis.

4.4. Strengths and Limitations

Some strengths can be reported. First, to the best of our knowledge, this is the first SNS validation to extensively evaluate the correlation with both functional outcomes and quality of life. Second, this study focused on the differences between self-report and observer-based measurements in terms of the relationship with functional outcome and quality of life. Third, the large sample size of this study can be considered a strength. Compared to other SNS validation studies, this study included the largest sample. There are also some limitations in our study. First, the lack of heterogeneity in symptom severity is a drawback. The PANSS total score and subscores are relatively low in our sample compared to other studies and are highly intercorrelated. Second, the use of a subgroup with mild positive symptoms to establish discriminant validity can be considered a limitation. However, since we wanted to investigate to what extent the PANSS positive subscore and the SNS measure different constructs, it was decided to exclude the influence of symptom absence. Third, during the period that the SNS was added to the PHAMOUS-protocol, there was no self-report questionnaire measuring functional outcomes incorporated in the protocol, and therefore, we were not able to examine the relationship between the SNS and experienced functioning. However, since self-assessment of functioning also captures internal experience, it is likely that the SNS will show significant associations with these assessment tools. Future research may be needed to confirm this hypothesis.

5. Conclusions

In conclusion, the Dutch translation of the SNS shows good psychometric properties and is associated with functional outcomes and quality of life, suggesting that more self-reported negative symptoms are related to lower levels of functioning and reported quality of life. Despite the importance of accurate assessment of negative symptoms, current observational measurement tools do not include the internal experience of people suffering from SSD. The SNS is able to differentiate between the different NIMH domains, adds information from the patients’ perspective, can be completed in approximately 10 min and requires little help from the caregivers, suggesting that it can be meaningful to complement current assessment tools. Furthermore, this could make the SNS useful in the research setting, since specific training in conducting interviews and the administration of instruments like the PANSS, SANSS, BNSS, and the CAINS can be time-consuming, which can cause time issues in conducting scientific research. However, a replication in a heterogeneous sample is needed to establish if the SNS is suitable as a standalone measurement tool for negative symptoms. If so, the SNS could be a promising instrument for the focus on the effect of interventions and therapeutics on the individual domains of negative symptoms.

Author Contributions

Conceptualization: L.v.d.M., A.P., S.D., A.A. and G.H.M.P.; methodology: L.v.d.M., A.P., S.D., G.H.M.P. and T.v.B.; investigation: T.v.B., L.v.d.M., A.P., G.H.M.P. and PHAMOUS-researchers; software: T.v.B. and A.P.; validation: T.v.B.; formal analysis: T.v.B.; data curation: T.v.B.; formal analysis: T.v.B.; visualization: T.v.B.; writing—original draft: T.v.B.; writing—review and editing: L.v.d.M., S.G., N.B., A.P., G.H.M.P., A.A., S.D., W.V., S.C., J.A.d.V. and E.L.; supervision: L.v.d.M., S.G., N.B. and A.P.; project administration: T.v.B. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

According to the declaration of Helsinki, service users are informed that their pseudonymized data could be used for scientific research, and they have the option to opt out of data use. Research on this dataset has been deemed exempt from the Medical Research Involving Human Subjects Act (WMO) by the Medical Ethical Committee of the University Medical Center Groningen. This exemption was granted because no additional burden is placed on service users, the research involves pre-existing routine healthcare data, and research on the data serves a broad public interest.

Informed Consent Statement

All patient data were obtained from the PHAMOUS-protocol, where patients are informed that aggregated and anonymized data may be used for healthcare optimization and scientific research to improve treatment and guidance.

Data Availability Statement

Under the General Data Protection Regulation (GDPR), our data are considered pseudonymized rather than anonymized and therefore still regarded as personal data. Given that participants have not given informed consent to have their personal data publicly shared, we are legally and ethically not allowed to publish our dataset. Data are therefore only available upon request at the Rob Giel Research center (Data Science Center), due to privacy and ethical reasons.

Acknowledgments

We would like to thank all participants and the following mental health institutions: University Center for Psychiatry of the University Medical Center Groningen, GGZ Friesland Mental Health Institution, GGZ Drenthe Mental Health Institution and Lentis Psychiatric Institute. We thank the PHAMOUS-investigators for their contributions to the PHAMOUS-protocol.

Conflicts of Interest

The authors declare no conflicts of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Tables

Table 1

Demographics.

	Mean (SD)	% (N)
Age	42.9 (12.5)
Gender (Female)		34.4 (85)
Diagnosis
Schizophrenia		37.80 (93)
Schizoaffective disorder		11.38 (28)
Bipolar disorder		6.91 (17)
Other psychotic disorder		44.31 (109)
Education 1
Lower		20.65 (51)
Middle		51.82 (128)
Higher		26.32 (65)
Daily activities
Job		27.53 (68)
Volunteering		21.05 (52)
Day care		19.43 (48)
Household		12.15 (30)
Study		8.10 (20)
Family care		1.21 (3)
None		23.48 (58)
SNS total	12.6 (7.43)
Social withdrawal	2.42 (2.03)
Reduced emotional range	2.79 (1.95)
Alogia	2.49 (2.02)
Avolition	2.83 (2.24)
Anhedonia	2.11 (1.86)
PANSS total 2	40.4 (10.3)
Positive subscale	9.79 (3.18)
Negative subscale	9.57 (3.73)
General subscale	21.0 (5.07)
HoNOS 3	6.60 (5.06)
GAF 4	57.0 (11.1)
FR 5	2.42 (1.67)
MANSA 6	59.3 (12.0)

¹: missing 3, ²: missing 40, ³: missing 80, ⁴: missing 113, ⁵: missing 49, ⁶: missing 65.

Table 2

Reliability of the SNS and the different subscales.

	Cronbach’s Alpha [CI 95%]
SNS total	0.87 [0.85–0.90]
Social withdrawal	0.76 [0.71–0.80]
Reduced emotional range	0.64 [0.56–0.71]
Alogia	0.71 [0.65–0.77]
Avolition	0.79 [0.75–0.83]
Anhedonia	0.69 [0.62–0.75]

Table 3

Intercorrelations between the SNS subscales (Spearman’s rho scores [CI 95%]).

SNS Subscores	Reduced Emotional Range	Alogia	Avolition	Anhedonia
Social withdrawal	0.37 *** [0.25–0.48]	0.48 *** [0.38–0.58]	0.43 *** [0.31–0.54]	0.48 *** [0.37–0.57]
Reduced emotional range		0.42 *** [0.31–0.53]	0.25 *** [0.13–0.37]	0.42 *** [0.30–0.52]
Alogia			0.38 *** [0.27–0.48]	0.51 *** [0.40–0.60]
Avolition				0.52 *** [0.41–0.61]

*** p < 0.001.

Table 4

Correlational analyses (Spearman’s rho scores [CI 95%]).

	PANSS Total (N = 207)	PANSSPositive	PANSSNegative	PANSSGeneral	HoNOS (N = 187)	GAF (N = 134)	FR(N = 198)	MANSA (N = 182)
SNStotal	0.36 *** [0.24–0.48]	0.31 *** [0.19–0.42]	0.33 ***[0.20–0.43]	0.30 *** [0.16–0.42]	0.37 *** [0.21–0.49]	−0.12[−0.28–0.037]	0.27 *** [0.14–0.39]	−0.40 ***[−0.52–−0.25]
Socialwithdrawal	0.29 *** [0.15–0.42]	0.31 *** [0.18–0.43]	0.31 *** [0.17–0.43]	0.19 * [0.042–0.32]	0.32 *** [0.16–0.46]	−0.19 *[−0.34–−0.028]	0.27 *** [0.12–0.40]	−0.22 **[−0.37–−0.075]
Reduced emotional range	0.19 ** [0.053–0.31]	0.18 ** [0.042–0.31]	0.19 ** [0.057–0.31]	0.13 [−0.0078–0.26]	0.18 * [0.045–0.32]	−0.046[−0.20–0.13]	0.097[−0.029–0.22]	−0.12[−0.27–0.028]
Alogia	0.21 ** [0.075–0.33]	0.13[−0.013–0.26]	0.21 ** [0.081–0.34]	0.20 ** [0.072–0.32]	0.23 ** [0.060–0.37]	−0.015[−0.18–0.15]	0.12[−0.0082–0.26]	−0.29 ***[−0.43–−0.16]
Avolition	0.34 *** [0.22–0.46]	0.29 *** [0.17–0.42]	0.25 *** [0.12–0.37]	0.31 *** [0.18–0.43]	0.35 *** [0.21–0.48]	−0.084[−0.26–0.090]	0.25 *** [0.12–0.37]	−0.46 ***[−0.59–−0.32]
Anhedonia	0.27 *** [0.13–0.40]	0.20 ** [0.066–0.32]	0.22 ** [0.090–0.36]	0.25 *** [0.12–0.38]	0.23 ** [0.074–0.38]	−0.086[−0.25–0.077]	0.19 ** [0.045–0.32]	−0.36 ***[−0.48–−0.21]
PANSStotal	x	0.82 *** [0.75–0.87]	0.78 *** [0.71–0.84]	0.92 ***[0.90–0.94]	0.56 *** [0.44–0.65]	−0.57 ***[−0.69–−0.42]	0.57 *** [0.46–0.66]	−0.43 ***[−0.55–−0.30]
PANSSpositive		x	0.53 *** [0.41–0.64]	0.65 *** [0.55–0.75]	0.50 *** [0.37–0.61]	−0.48 ***[−0.62–−0.32]	0.52 *** [0.40–0.63]	−0.34 ***[−0.47–−0.19]
PANSSnegative			x	0.61 *** [0.51–0.70]	0.44 *** [0.30–0.56]	−0.43 ***[−0.59–−0.26]	0.53 *** [0.42–0.63]	−0.35 ***[−0.48–−0.20]
PANSSgeneral				x	0.50 *** [0.37–0.61]	−0.53 ***[−0.65–−0.38]	0.49 *** [0.37–0.60]	−0.42 ***[−0.55–−0.29]

* p < 0.05; ** p < 0.01; *** p < 0.001.

Table 5

Correlations between SNS and PANSS positive and negative (any item in p1–p7 ≥ 3) (Spearman’s rho scores [CI 95%]).

	PANSS Positive	PANSS Negative
SNS total	0.066 [−0.14–0.26]	0.26 ** [0.063–0.44]
Social withdrawal	0.27 ** [0.090–0.44]	0.31 ** [0.12–0.50]
Reduced emotional range	−0.11 [−0.30–0.087]	0.061 [−0.14–0.27]
Alogia	−0.059 [−0.26–0.16]	0.24 * [0.055–0.41]
Avolition	0.11 [−0.075–0.29]	0.21 * [−0.0038–0.39]
Anhedonia	−0.010 [−0.22–0.19]	0.12 [−0.092–0.32]

* p < 0.05; ** p < 0.01.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/brainsci15010015/s1, Tabel S1: Missing values of items; Table S2: p-values of differences between correlation coefficients in HoNOS; Table S3: p-values of differences between correlation coefficients in GAF; Table S4: p-values of differences between correlation coefficients in FR; Table S5: p-values of differences between correlation coefficients in MANSA.

References

1. Bleuler, E. Dementia Praecox or the Group of Schizophrenias; International Universities Press: New York, NY, USA, 1950.

2. Kraepelin, E. Dementia Praecox and Paraphrenia; Chicago Medical Book Company: Chicago, IL, USA, 1919.

3. van der Heiden, W.; Leber, A.; Häfner, H. Negative symptoms and their association with depressive symptoms in the long-term course of schizophrenia. Eur. Arch. Psychiatry Clin. Neurosci.; 2016; 266, pp. 387-396. [DOI: https://dx.doi.org/10.1007/s00406-016-0697-2]

4. Mäkinen, J.; Miettunen, J.; Isohanni, M.; Koponen, H. Negative symptoms in schizophrenia—A review. Nord. J. Psychiatry; 2008; 62, pp. 334-341. [DOI: https://dx.doi.org/10.1080/08039480801959307]

5. Bobes, J.; Arango, C.; Garcia-Garcia, M.; Rejas, J. Prevalence of Negative Symptoms in Outpatients With Schizophrenia Spectrum Disorders Treated With Antipsychotics in Routine Clinical Practice. J. Clin. Psychiatry; 2010; 71, pp. 280-286. [DOI: https://dx.doi.org/10.4088/JCP.08m04250yel]

6. Fervaha, G.; Foussias, G.; Agid, O.; Remington, G. Impact of primary negative symptoms on functional outcomes in schizophrenia. Eur. Psychiatry; 2014; 29, pp. 449-455. [DOI: https://dx.doi.org/10.1016/j.eurpsy.2014.01.007] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/24630742]

7. Rabinowitz, J.; Levine, S.Z.; Garibaldi, G.; Bugarski-Kirola, D.; Berardo, C.G.; Kapur, S. Negative symptoms have greater impact on functioning than positive symptoms in schizophrenia: Analysis of CATIE data. Schizophr. Res.; 2012; 137, pp. 147-150. [DOI: https://dx.doi.org/10.1016/j.schres.2012.01.015] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/22316568]

8. Correll, C.U.; Schooler, N.R. Negative Symptoms in Schizophrenia: A Review and Clinical Guide for Recognition, Assessment, and Treatment. Neuropsychiatr. Dis. Treat.; 2020; 16, pp. 519-534. [DOI: https://dx.doi.org/10.2147/NDT.S225643]

9. van der Heiden, W.; Häfner, H. The epidemiology of onset and course of schizophrenia. Eur. Arch. Psychiatry Clin. Neurosci.; 2000; 250, pp. 292-303. [DOI: https://dx.doi.org/10.1007/s004060070004] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/11153964]

10. Devoe, D.J.; Braun, A.; Seredynski, T.; Addington, J. Negative Symptoms and Functioning in Youth at Risk of Psychosis: A Systematic Review and Meta-analysis. Harv. Rev. Psychiatry; 2020; 28, pp. 341-355. [DOI: https://dx.doi.org/10.1097/HRP.0000000000000273] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33156155]

11. Salazar de Pablo, G.; Catalan, A.; Vaquerizo Serrano, J.; Pedruzo, B.; Alameda, L.; Sandroni, V.; Armendariz, A.; Rodriguez, V.; Arango, C.; Moreno, C. et al. Negative symptoms in children and adolescents with early-onset psychosis and at clinical high-risk for psychosis: Systematic review and meta-analysis. Br. J. Psychiatry; 2023; 223, pp. 282-294. [DOI: https://dx.doi.org/10.1192/bjp.2022.203]

12. Aleman, A.; Lincoln, T.M.; Bruggeman, R.; Melle, I.; Arends, J.; Arango, C.; Knegtering, H. Treatment of negative symptoms: Where do we stand, and where do we go?. Schizophr. Res.; 2017; 186, pp. 55-62. [DOI: https://dx.doi.org/10.1016/j.schres.2016.05.015] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/27293137]

13. Galderisi, S.; Kaiser, S.; Bitter, I.; Nordentoft, M.; Mucci, A.; Sabé, M.; Giordano, G.M.; Nielsen, M.; Glenthøj, L.B.; Pezzella, P. et al. EPA guidance on treatment of negative symptoms in schizophrenia. Eur. Psychiatry; 2021; 64, e21. [DOI: https://dx.doi.org/10.1192/j.eurpsy.2021.13] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33726883]

14. Kirkpatrick, B.; Fenton, W.S.; Carpenter, W.T.; Marder, S.R. The NIMH-MATRICS Consensus Statement on Negative Symptoms. Schizophr. Bull.; 2006; 32, pp. 214-219. [DOI: https://dx.doi.org/10.1093/schbul/sbj053] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/16481659]

15. Kay, S.R.; Fiszbein, A.; Opler, L.A. The Positive and Negative Syndrome Scale (PANSS) for Schizophrenia. Schizophr. Bull.; 1987; 13, pp. 261-276. [DOI: https://dx.doi.org/10.1093/schbul/13.2.261]

16. Andreasen, N.C. Scale for the Assessment of Negative Symptoms (SANS). Br. J. Psychiatry; 1989; 155, pp. 53-58. [DOI: https://dx.doi.org/10.1192/S0007125000291496]

17. Kirkpatrick, B.; Strauss, G.P.; Nguyen, L.; Fischer, B.A.; Daniel, D.G.; Cienfuegos, A.; Marder, S.R. The Brief Negative Symptom Scale: Psychometric Properties. Schizophr. Bull.; 2011; 37, pp. 300-305. [DOI: https://dx.doi.org/10.1093/schbul/sbq059]

18. Kring, A.M.; Gur, R.E.; Blanchard, J.J.; Horan, W.P.; Reise, S.P. The Clinical Assessment Interview for Negative Symptoms (CAINS): Final Development and Validation. Am. J. Psychiatry; 2013; 170, pp. 165-172. [DOI: https://dx.doi.org/10.1176/appi.ajp.2012.12010109] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/23377637]

19. Khan, A.; Yavorsky, W.C.; Liechti, S.; DiClemente, G.; Rothman, B.; Opler, M.; DeFries, A.; Jovic, S. Assessing the Sources of Unreliability (Rater, Subject, Time-Point) in a Failed Clinical Trial Using Items of the Positive and Negative Syndrome Scale (PANSS). J. Clin. Psychopharmacol.; 2013; 33, pp. 109-117. [DOI: https://dx.doi.org/10.1097/JCP.0b013e3182776ebe]

20. Andreasen, N.C. Negative Symptoms in Schizophrenia Definition and Reliability. Arch. Gen. Psychiatry; 1982; 39, pp. 784-788. [DOI: https://dx.doi.org/10.1001/archpsyc.1982.04290070020005]

21. David, A.S. Insight and psychosis. Br. J. Psychiatry; 1990; 156, pp. 798-808. [DOI: https://dx.doi.org/10.1192/bjp.156.6.798] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/2207510]

22. Tandon, R.; Nasrallah, H.A.; Keshavan, M.S. Schizophrenia, “just the facts” 4. Clinical features and conceptualization. Schizophr. Res.; 2009; 110, pp. 1-23. [DOI: https://dx.doi.org/10.1016/j.schres.2009.03.005]

23. Myin-Germeys, I.; Delespaul, P.A.E.G.; deVries, M.W. Schizophrenia Patients Are More Emotionally Active Than Is Assumed Based on Their Behavior. Schizophr. Bull.; 2000; 26, pp. 847-854. [DOI: https://dx.doi.org/10.1093/oxfordjournals.schbul.a033499] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/11087017]

24. Kring, A.M.; Kerr, S.L.; Smith, D.A.; Neale, J.M. Flat affect in schizophrenia does not reflect diminished subjective experience of emotion. J. Abnorm. Psychol.; 1993; 102, pp. 507-517. [DOI: https://dx.doi.org/10.1037/0021-843X.102.4.507]

25. Berenbaum, H.; Oltmanns, T.F. Emotional experience and expression in schizophrenia and depression. J. Abnorm. Psychol.; 1992; 101, pp. 37-44. [DOI: https://dx.doi.org/10.1037/0021-843X.101.1.37]

26. Song, J.; Borlido, C.; De Luca, V.; Burton, L.; Remington, G. Patient versus rater evaluation of symptom severity in treatment resistant schizophrenia receiving clozapine. Psychiatry Res.; 2019; 274, pp. 409-413. [DOI: https://dx.doi.org/10.1016/j.psychres.2019.02.050]

27. Dollfus, S.; Mach, C.; Morello, R. Self-Evaluation of Negative Symptoms: A Novel Tool to Assess Negative Symptoms. Schizophr. Bull.; 2016; 42, pp. 571-578. [DOI: https://dx.doi.org/10.1093/schbul/sbv161]

28. Wójciak, P.; Górna, K.; Domowicz, K.; Jaracz, K.; Szpalik, R.; Michalak, M.; Rybakowski, J. Polish version of the Self-evaluation of Negative Symptoms (SNS). Psychiatr. Pol.; 2019; 53, pp. 551-559. [DOI: https://dx.doi.org/10.12740/PP/OnlineFirst/97352]

29. Rodríguez-Testal, J.F.; Perona-Garcelán, S.; Dollfus, S.; Valdés-Díaz, M.; García-Martínez, J.; Ruíz-Veguilla, M.; Senín-Calderón, C. Spanish validation of the self-evaluation of negative symptoms scale SNS in an adolescent population. BMC Psychiatry; 2019; 19, 327. [DOI: https://dx.doi.org/10.1186/s12888-019-2314-1]

30. Hajj, A.; Hallit, S.; Chamoun, K.; Sacre, H.; Obeid, S.; Haddad, C.; Dollfus, S.; Khabbaz, L.R. Validation of the Arabic version of the “self-evaluation of negative symptoms” scale (SNS). BMC Psychiatry; 2020; 20, 240. [DOI: https://dx.doi.org/10.1186/s12888-020-02647-4]

31. Montvidas, J.; Adomaitienė, V.; Leskauskas, D.; Dollfus, S. Validation of the lithuanian version of the self-evaluation of negative symptoms scale (SNS). Nord. J. Psychiatry; 2021; 75, pp. 351-355. [DOI: https://dx.doi.org/10.1080/08039488.2020.1862295]

32. Tam, M.H.; Ling-Ling, W.; Cheng, K.M.; Wong, J.O.; Cheung, E.F.; Lui, S.S.; Chan, R.C. Latent structure of self-report negative symptoms in patients with schizophrenia: A preliminary study. Asian J. Psychiatr.; 2021; 61, 102680. [DOI: https://dx.doi.org/10.1016/j.ajp.2021.102680]

33. Mazhari, S.; Karamooz, A.; Shahrbabaki, M.E.; Jahanbakhsh, F.; Dollfus, S. Validity and reliability of a Persian version of the self- evaluation of negative symptoms (SNS). BMC Psychiatry; 2021; 21, 516. [DOI: https://dx.doi.org/10.1186/s12888-021-03521-7]

34. Polat, I.; Ince Guliyev, E.; Elmas, S.; Karakaş, S.; Aydemir, Ö.; Üçok, A. Validation of the Turkish version of the self-evaluation of negative symptoms scale (SNS). Int. J. Psychiatry Clin. Pract.; 2022; 26, pp. 221-227. [DOI: https://dx.doi.org/10.1080/13651501.2022.2082985]

35. Dollfus, S.; Mucci, A.; Giordano, G.M.; Bitter, I.; Austin, S.F.; Delouche, C.; Erfurth, A.; Fleischhacker, W.W.; Movina, L.; Glenthøj, B. et al. European Validation of the Self-Evaluation of Negative Symptoms (SNS): A Large Multinational and Multicenter Study. Front. Psychiatry.; 2022; 13, 826465. [DOI: https://dx.doi.org/10.3389/fpsyt.2022.826465]

36. Mallet, J.; Guessoum, S.B.; Tebeka, S.; Le Strat, Y.; Dubertret, C. Self-evaluation of negative symptoms in adolescent and young adult first psychiatric episodes. Prog. Neuropsychopharmacol. Biol. Psychiatry; 2020; 103, 109988. [DOI: https://dx.doi.org/10.1016/j.pnpbp.2020.109988] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32474008]

37. Bartels-Velthuis, A.A.; Visser, E.; Arends, J.; Pijnenborg, G.H.; Wunderink, L.; Jörg, F.; Veling, W.; Liemburg, E.J.; Castelein, S.; Knegtering, H. et al. Towards a comprehensive routine outcome monitoring program for people with psychotic disorders: The Pharmacotherapy Monitoring and Outcome Survey (PHAMOUS). Schizophr. Res.; 2018; 197, pp. 281-287. [DOI: https://dx.doi.org/10.1016/j.schres.2018.01.016] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/29395613]

38. Prinsen, C.A.C.; Mokkink, L.B.; Bouter, L.M.; Alonso, J.; Patrick, D.L.; de Vet, H.C.W.; Terwee, C.B. COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual. Life Res.; 2018; 27, pp. 1147-1157. [DOI: https://dx.doi.org/10.1007/s11136-018-1798-3] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/29435801]

39. Wing, J.K.; Beevor, A.S.; Curtis, R.H.; Park, S.G.B.; Hadden, J.; Burns, A. Health of the Nation Outcome Scales (HoNOS). Br. J. Psychiatry; 1998; 172, pp. 11-18. [DOI: https://dx.doi.org/10.1192/bjp.172.1.11] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/9534825]

40. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders; 4th ed. American Psychiatric Association: Washington, DC, USA, 2000.

41. Wiersma, D.; Visser, E.; Bähler, M.; Bruggeman, R.; A Delespaul, P.; Van Der Gaag, M.; De Haan, L.; Keet, I.P.M.; Nijssen, Y.; Van Os, J. et al. [Functional remission of people with serious mental illness (SMI): Psychometric properties of a new ROM-instrument]. Tijdschr. Psychiatr.; 2015; 57, pp. 395-404.

42. Taber, K.S. The Use of Cronbach’s Alpha When Developing and Reporting Research Instruments in Science Education. Res. Sci. Educ.; 2018; 48, pp. 1273-1296. [DOI: https://dx.doi.org/10.1007/s11165-016-9602-2]

43. Brown, T.A. Confirmatory Factor Analysis for Applied Research; The Guilford Press: New York, NY, USA, 2006.

44. R Core Team. R: A Language and Environment for Statistical Computing; R Foundation for Statistical Computing: Vienna, Austria, 2023.

45. Revelle, W. psych: Procedures for Psychological, Psychometric, and Personality Research. Available online: https://CRAN.R-project.org/package=psych (accessed on 1 January 2023).

46. Rosseel, Y. lavaan: An R Package for Structural Equation Modeling. J. Stat. Softw.; 2012; 48, pp. 1-36. [DOI: https://dx.doi.org/10.18637/jss.v048.i02]

47. Tierney, N.; Cook, D. Expanding Tidy Data Principles to Facilitate Missing Data Exploration, Visualization and Assessment of Imputations. J. Stat. Softw.; 2023; 105, pp. 1-31. [DOI: https://dx.doi.org/10.18637/jss.v105.i07]

48. Bora, E.; Pantelis, C. Theory of mind impairments in first-episode psychosis, individuals at ultra-high risk for psychosis and in first-degree relatives of schizophrenia: Systematic review and meta-analysis. Schizophr. Res.; 2013; 144, pp. 31-36. [DOI: https://dx.doi.org/10.1016/j.schres.2012.12.013]

49. van Neerven, T.; Bos, D.J.; van Haren, N.E. Deficiencies in Theory of Mind in patients with schizophrenia, bipolar disorder, and major depressive disorder: A systematic review of secondary literature. Neurosci. Biobehav. Rev.; 2021; 120, pp. 249-261. [DOI: https://dx.doi.org/10.1016/j.neubiorev.2020.11.011] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33246019]

50. Horan, W.P.; Kring, A.M.; Blanchard, J.J. Anhedonia in Schizophrenia: A Review of Assessment Strategies. Schizophr. Bull.; 2006; 32, pp. 259-273. [DOI: https://dx.doi.org/10.1093/schbul/sbj009] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/16221997]

51. Galderisi, S.; Mucci, A.; Buchanan, R.W.; Arango, C. Negative symptoms of schizophrenia: New developments and unanswered research questions. Lancet Psychiatry; 2018; 5, pp. 664-677. [DOI: https://dx.doi.org/10.1016/S2215-0366(18)30050-6] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/29602739]

52. Kiang, M.; Christensen, B.K.; Remington, G.; Kapur, S. Apathy in schizophrenia: Clinical correlates and association with functional outcome. Schizophr. Res.; 2003; 63, pp. 79-88. [DOI: https://dx.doi.org/10.1016/S0920-9964(02)00433-4]

53. Konstantakopoulos, G.; Ploumpidis, D.; Oulis, P.; Patrikelis, P.; Soumani, A.; Papadimitriou, G.N.; Politis, A.M. Apathy, cognitive deficits and functional impairment in schizophrenia. Schizophr. Res.; 2011; 133, pp. 193-198. [DOI: https://dx.doi.org/10.1016/j.schres.2011.07.003] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/21788116]

54. Strauss, G.P.; Horan, W.P.; Kirkpatrick, B.; Fischer, B.A.; Keller, W.R.; Miski, P.; Buchanan, R.W.; Green, M.F.; Carpenter, W.T. Deconstructing negative symptoms of schizophrenia: Avolition-apathy and diminished expression clusters predict clinical presentation and functional outcome. J. Psychiatr. Res.; 2013; 47, pp. 783-790. [DOI: https://dx.doi.org/10.1016/j.jpsychires.2013.01.015]

55. Mucci, A.; Vignapiano, A.; Bitter, I.; Austin, S.F.; Delouche, C.; Dollfus, S.; Erfurth, A.; Fleischhacker, W.W.; Giordano, G.M.; Gladyshev, I. et al. A large European, multicenter, multinational validation study of the Brief Negative Symptom Scale. Eur. Neuropsychopharmacol.; 2019; 29, pp. 947-959. [DOI: https://dx.doi.org/10.1016/j.euroneuro.2019.05.006]

56. Okada, H.; Hirano, D.; Taniguchi, T. Impact of Negative Symptom Domains and Other Clinical Characteristics on Functional Outcomes in Patients with Schizophrenia. Schizophr. Res. Treatment.; 2021; 2021, 8864352. [DOI: https://dx.doi.org/10.1155/2021/8864352] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33688435]

57. Messinger, J.W.; Trémeau, F.; Antonius, D.; Mendelsohn, E.; Prudent, V.; Stanford, A.D.; Malaspina, D. Avolition and expressive deficits capture negative symptom phenomenology: Implications for DSM-5 and schizophrenia research. Clin. Psychol. Rev.; 2011; 31, pp. 161-168. [DOI: https://dx.doi.org/10.1016/j.cpr.2010.09.002] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/20889248]

58. Foussias, G.; Siddiqui, I.; Fervaha, G.; Agid, O.; Remington, G. Dissecting negative symptoms in schizophrenia: Opportunities for translation into new treatments. J. Psychopharmacol.; 2015; 29, pp. 116-126. [DOI: https://dx.doi.org/10.1177/0269881114562092] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/25516370]

59. Green, M.F.; Hellemann, G.; Horan, W.P.; Lee, J.; Wynn, J.K. From perception to functional outcome in schizophrenia: Modeling the role of ability and motivation. Arch. Gen. Psychiatry; 2012; 69, pp. 1216-1224. [DOI: https://dx.doi.org/10.1001/archgenpsychiatry.2012.652]

60. Fervaha, G.; Foussias, G.; Agid, O.; Remington, G. Amotivation and functional outcomes in early schizophrenia. Psychiatry Res.; 2013; 210, pp. 665-668. [DOI: https://dx.doi.org/10.1016/j.psychres.2013.07.024] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/23911005]

61. Galderisi, S.; Rossi, A.; Rocca, P.; Gibertoni, D.; Aguglia, E.; Amore, M.; Bellomo, A.; Biondi, M.; Brugnoli, R.; Dell, L. et al. The Influence of Illness-Related Variables, Personal Resources and Context-Related Factors on Real-Life Functioning of People with Schizophrenia. World Psychiatry; 2014; 13, pp. 275-287. [DOI: https://dx.doi.org/10.1002/wps.20167] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/25273301]

62. Rocca, P.; Montemagni, C.; Zappia, S.; Piterà, R.; Sigaudo, M.; Bogetto, F. Negative symptoms and everyday functioning in schizophrenia: A cross-sectional study in a real world-setting. Psychiatry Res.; 2014; 218, pp. 284-289. [DOI: https://dx.doi.org/10.1016/j.psychres.2014.04.018] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/24814140]

63. Ventura, J.; Subotnik, K.L.; Gitlin, M.J.; Gretchen-Doorly, D.; Ered, A.; Villa, K.F.; Hellemann, G.S.; Nuechterlein, K.H. Negative symptoms and functioning during the first year after a recent onset of schizophrenia and 8years later. Schizophr. Res.; 2015; 161, pp. 407-413. [DOI: https://dx.doi.org/10.1016/j.schres.2014.10.043] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/25499044]

64. Montvidas, J.; Adomaitienė, V.; Leskauskas, D.; Dollfus, S. Correlation of Health-Related Quality of Life with Negative Symptoms Assessed with the Self-Evaluation of Negative Symptoms Scale (SNS) and Cognitive Deficits in Schizophrenia: A Cross-Sectional Study in Routine Psychiatric Care. J. Clin. Med.; 2023; 12, 901. [DOI: https://dx.doi.org/10.3390/jcm12030901] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36769548]