Measuring fidelity to manualised peer support for people with severe mental health conditions: development and psychometric evaluation of the UPSIDES fidelity scale

Abstract

Background

Peer support workers provide support for people experiencing mental health conditions based on their own lived experience of mental health problems. Assessing fidelity to core ingredients of peer support is vital for successful implementation and intervention delivery. Modifications to its implementation are needed when scaling up to different socio-economic settings, raising further uncertainty about fidelity. As part of a large multi-centre study on peer support called Using Peer Support In Developing Empowering Mental Health Services (UPSIDES), we developed and evaluated the psychometric properties of the UPSIDES Fidelity Scale.

Methods

We constructed the fidelity scale based on an initial item pool developed through international expert consultation and iterative feedback. Scale refinement involved site-level expert consultation and translation, resulting in a service user-rated 28-item version and a peer support worker-rated 21-item version assessing receipt, engagement, enactment, competence, communication and peer support-specific components. Both versions are available in six languages: English, German, Luganda, Kiswahili, Hebrew and Gujarati. The scale was then evaluated at six study sites across five countries, with peer support workers and their clients completing their respective ratings four and eight months after initial peer support worker contact. Psychometric evaluation included analysis of internal consistency, construct validity and criterion validity.

Results

For the 315 participants, item statistics showed a skewed distribution of fidelity values but no restriction of range. Internal consistency was adequate (range α = 0.675 to 0.969) for total scores and all subscales in both versions. Confirmatory factor analysis indicated acceptable fit of the proposed factor structure for the service user version (χ2/df = 2.746; RMSEA = 0.084) and moderate fit for the peer support worker version (χ2/df = 3.087; RMSEA = 0.093). Both versions showed significant correlations with external criteria: number of peer support sessions; perceived recovery orientation of the intervention; and severity of illness.

Conclusions

The scale demonstrates good reliability, construct and criterion validity, making it a pragmatic and psychometrically acceptable measure for assessing fidelity to a manualised peer support worker intervention. Recommendations for use, along with research and practical implications, are addressed. As validated, multi-lingual tool that adapts to diverse settings this scale is uniquely positioned for global application.

Trial registration

ISRCTN, ISRCTN26008944. Registered on 30 October 2019.