Measurement scales of mental health related to climate change: a scoping review protocol using artificial intelligence

Abstract

Introduction

Human actions have influenced climate changes around the globe, causing extreme weather phenomena and impacting communities worldwide. Climate change has caused, directly or indirectly, health effects such as injury and physical injuries, which impact morbidity and mortality. Similarly, there is evidence that exposure to climatic catastrophes has serious repercussions on psychological well-being, and rising temperatures and drought have detrimental effects on mental health.

Despite the recent effort of researchers to develop specific instruments to assess the effects of climate change on mental health, the evidence on measures of its impact is still scarce, and the constructs are heterogeneous. The aim of this scoping review is to describe the instruments developed and validated to assess the impact of mental health related to climate change.

Methods and analysis

This review is registered at Open Science Framework (https://osf.io/zdmbk). This scoping review will follow the reporting elements chosen for systematic review and meta-analysis (Preferred Reporting Items for Systematic Reviews and Meta-Analyses). We proposed a PO question, as it places no restrictions on the participants (P), and the outcome (O) are measurement instruments on mental health related to climate change. A search will be conducted in different databases (PubMed, Scopus, Web of Science, PsycINFO). We will use an open-source artificial intelligence screening tool (ASReview LAB) for the title and abstract review. The full-text review will be performed by three researchers. If there is a disagreement between two independent reviewers, a third reviewer will take the final decision. We will use the COnsensus-based Standards for the selection of health Measurement INstruments tool to assess the risk of bias for each included study. The review will be conducted starting in September 2023.

Ethics and dissemination

The planned scoping review does not require ethical approval since it will not involve an ethical risk to the participants. The results obtained from this study will be presented at conferences, congresses and scientific publications.

Correspondence to Dr Alejandro Dominguez-Rodriguez; [email protected]

STRENGTHS AND LIMITATIONS OF THIS STUDY

• This scoping review will follow the reporting elements chosen for systematic review and meta-analysis (Preferred Reporting Items for Systematic Reviews and Meta-Analyses), ensuring transparency, and the COnsensus-based Standards for the selection of health Measurement INstruments tool to assess the risk of bias for each included study will be used.

• The screening process will be supported by reference manager ASReview, an innovative artificial intelligence software that helps remove manually duplicated citations and also conducts a less biased review.

• Since this is a new topic, scientific production is carried out in grey literature documents (ie, doctoral theses and books). However, our study will not consider them.

• Nowadays, a limitation is the inability to screen the emergent climate change perception construct and better methods, which is a recent area, and its links to health and its measures.

Background

Human actions have influenced changes in climate around the globe, causing extreme weather phenomena and impacting communities worldwide.¹ As announced by the United Nations Environment Programme in its Emissions Gap Report,² it is expected that by the year 2030, global warming will reach more than a 1.5°C increase in contrast to pre-industrial values. Although various intergovernmental pledges have been made to reduce greenhouse gas emissions, these initiatives are still insufficient.² In addition, according to predictions conducted by Jafino et al,³ 44 million people could fall into extreme poverty due to the adverse effects of climate change on people’s health. Other climate change detriments are increases in heat-related diseases, infectious diseases and cardiopulmonary problems.⁴ Additionally, there is evidence that climate change induces events such as hurricanes, floods, heat waves and forest fires, among other environmental disasters,⁵ which affect people’s finances; climate change is expected to intensify deprivations and inequalities.⁶ Also, researchers warn that human-caused climate change could seriously impact food production and global food equal distribution.⁷

Another consequence of climate change is the impact on mental health. Humanity is increasingly aware of the effects of the environmental crisis, evidenced by natural disasters and extreme changes in the weather, which are considered fundamental challenges that the next generation will have to face.⁸ This situation has provoked concern associated with increased mental health symptoms such as (but not limited to) distress,⁹ anxiety, post-traumatic stress and depression.⁸ In this context, new terms such as solastalgia,⁹ eco-anxiety,¹⁰ climate anxiety^{11 12} or ecological grief,¹³ among others, have emerged to make visible the emotional discomfort derived from the growing awareness of the progressive environmental deterioration. Another aspect to mention is that climate change can impact mental health in two different manners: directly after experiencing a trauma derived from a natural disaster or indirectly affecting the physical health, individual emotional mood and community well-being.¹⁴ For example, there is evidence that exposure to climatic catastrophes has serious repercussions on psychological well-being, and rising temperatures and drought have detrimental effects on mental health. In general, climate change has directly or indirectly impacted morbidity and mortality.¹⁵

Nonetheless, it is crucial to bear in mind that environmental disasters or climate change are intricate occurrences, and assigning exclusive causality to individuals’ mental health, well-being or emotional state is not feasible. Therefore, comprehending that their influence is contingent on the context in which they are evaluated holds significant importance.

For evaluating the psychological repercussions of climate change, various studies have employed instruments traditionally used in clinical settings.¹⁶ Although these questionnaires assess mental health symptomatology in general, they have not been specifically developed to evaluate the psychological effects of climate change, which could limit knowledge in the study area.

In recent years, specific instruments have been developed to assess the direct and indirect psychological consequences of climate change. For example, Clayton and Karazsia developed the Climate Anxiety Scale (CAS), whose aim is to explore the dysphoric emotional responses derived from climate change.¹⁷ The CAS has been validated in various countries, such as Poland¹⁸ and Germany,¹⁹ showing good psychometric properties. Other instruments have been recently implemented for this purpose. For example, Ágoston et al developed three scales that evaluate eco-anxiety, eco-guilt and eco-grief, highlighting that the emotional consequences of climate change constitute a broad umbrella of psychological requirements.²⁰ In line with this, Ágoston et al conducted a qualitative study using interviews to explore in-depth subtypes and dimensions of the three concepts mentioned (ie, eco-anxiety, eco-guilt and eco-grief).²¹ They found that the three are multifactorial constructs, each including different components. These results exhibit the scientific interest in making visible characteristics and risk factors that allow continuing with the implementation of new measurement tools for a better understanding of these heterogeneous variables.

Despite the recent effort of researchers to develop specific instruments to assess the effects of climate change on emotional mood and mental health, the evidence on measures of its impact is still scarce, and the constructs are heterogeneous. The impact on mental health caused by climate change is still far from being totally understood, and this can be due to the large heterogeneity in what to measure and how to measure, along with discovering the underlying mechanisms of adaptation and the attempt to define the direct cause–effect relationship.²²

This situation makes it necessary to organise the existing information regarding the current state of the evaluation tools. In this sense, Martin et al conducted a systematic review to analyse which instruments exist in order to evaluate negative emotions related to climate change, but focused only on the young population.^{23 24}

In the face of the progressive global climate crisis, understanding risk factors and identifying potential preventive and coping strategies could help design new field steps. In addition, understanding the impact of mental health on the general population, such as the old and young population, is relevant since this problem affects the general population.

For this purpose, a scoping review will be conducted. This knowledge synthesis method, known as ‘mapping’ review, allows for a broad overview of a topic.²⁵ ‘Scoping reviews are therefore of particular use when a body of literature has not yet been comprehensively reviewed or exhibits a large, complex, or heterogeneous nature not amenable to a more precise systematic review.’²⁶ Additionally, this method allows for exploring emerging areas and identifying evidence gaps,^{26 27} which could help summarise the current research topic and lay the groundwork for future research.

This study arises to answer the question: what instruments are developed and validated to assess the impact of climate change on mental health specifically? To the best of our knowledge, this is the first study to use a scoping review to explore the available primary studies on existing instruments, the constructs they represent, and the terms used to refer to climate change and its consequences on mental health.

Methods

The present study proposes to carry out a scoping review, defined as a type of review whose structure is similar to a systematic review. However, they present methodological differences since the scoping review is aimed at mapping the key concepts that support a field of research, as well as clarifying working definitions and/or the conceptual limits of a topic, and maybe a precedent for systematic reviews.^{2 5 26 28}

Protocol and registration

The protocol of this scoping review is registered in Open Science Framework according to the Cochrane manual guidelines for systematic reviews (https://osf.io/zdmbk). In addition, the protocol of the scoping review was prepared according to the PRISMA-ScR recommendations, which is the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension for scoping reviews (see online supplemental material 1).²⁷ The steps to be followed during the scoping review are presented in online supplemental material 2 and follow PRISMA guidelines.²⁹

Our research question is what psychometric instruments exist to measure mental health in the context of climate change directly and indirectly in different population groups? Our scoping review focuses on a PO question, as it places no restrictions on the participants (P), and the outcome (O) are measurement instruments on mental health and climate change.

Eligibility criteria

We will consider studies as sources of evidence that must meet all six criteria:

• Original studies using primary or secondary data.

• Papers should assess the measurement properties of mental health and climate change psychometric instruments in general (ie, eco-anxiety, solastalgia, fear of climate change) or specific (ie, fear of specific climate disaster).

• Published studies that were conducted up to 16 August 2023.

• Language studies in full text will be considered: Spanish, English, German and Portuguese.

• The target population could be non-specific (eg, general population) or a subpopulation (eg, women of childbearing age).

• Only articles published in indexed journals will be considered.

Documents that have not passed a peer review process (ie, preprints) will not be considered. In addition, review articles will be removed (ie, scoping review and systematic review). Studies using clinical instruments that do not directly or indirectly measure climate change will be excluded.

Patient and public involvement

None.

Information sources

A search will be carried out in different databases of scientific articles (PubMed, Scopus, Web of Science and PsycINFO). Scopus and Web of Science are relevant databases because they cover publications from multidisciplinary fields of science, including those of interest for the present review. For completeness, the search in Web of Science will include its full collection (ie, including Web of Science Core Collection with Science Citation Index Expanded, Social Sciences Citation Index, Arts & Humanities Citation Index, Emerging Sources Citation Index; MEDLINE; and SciELO citation index). PubMed is relevant because it specialises in biomedical literature (it is mostly composed by MEDLINE but includes other materials as well). Similarly, PsycINFO is relevant because it covers literature from psychological and behavioural sciences. The date of the search will be 25 September 2023. The search will be performed on the same day for all databases.

Search strategy

This protocol is structured according to the reporting elements chosen for systematic review and meta-analysis (PRISMA).²⁹ Specifically, the present review will follow the procedure of the PRISMA-ScR guidelines.²⁶ The search strategy will include publications produced up to 16 August 2023.

The search strategy was designed with the support of an information specialist experienced in conducting systematic review searches, based on the methodologically rigorous scoping review approach proposed by Arksey and O’Malley for conducting a systematic search.²⁵ A limited search will be previously carried out in PubMed. The purpose of the search will be to identify articles focused on the application of instruments aimed at measuring (a) mental health specifically in the context of climate change, or (b) general mental health instruments applied in situations of climate change. These preliminary results will be analysed to identify relevant key terms for the final search. Based on this process, a first version of the search strategy will be developed, composed of key terms related to “mental health”, “climate change” and “psychometric instruments” (see online supplemental material 3). Additionally, the final search strategy will be peer reviewed by an information specialist not involved in the study, using the Peer Review of Electronic Search Strategies checklist.²⁷

Selection and characteristics of sources of evidence

The selection and identification procedure will consist of four stages. First, the searches in the databases will be imported into the EndNote software to eliminate duplicate articles. Second, two independent reviewers will enter and review the data (by title and abstract) in ASReview, an open-source software based on machine (active) learning. ASReview tool was chosen because it uses machine learning to support the finding of relevant publications in a more efficient manner.³⁰ In ASReview, the screening is still conducted manually, but it is supported with machine learning by reordering the records according to relevance scores that are modelled via the previous choices of the reviewer. Furthermore, in a study aimed at the analysis of ASReview, it was noticed that the tool effectively reduced the working time for researchers, within a dataset containing 4695 articles; the stopping criterion was attained after evaluating 1063 articles, resulting in 23% of them being reviewed by the researchers.³¹ At this stage, the two independent reviewers will adopt a mixed strategy to decide when to stop screening: (a) at least 50% of the dataset will be screened, and after that (b) screening will be stopped after 50 consecutive irrelevant papers, the independent (c) results and any conflicts will be discussed by the team to determine inclusion in the next stage. In the third stage, the selected articles will be reviewed in full text and verified by two independent reviewers. The Covidence web-based software platform (Veritas Health Innovation) will be used to facilitate collaborative screening of full texts. In case of disagreement, a third reviewer will evaluate the full text and discuss the areas of conflict with both reviewers to make a decision. At the fourth and final stage, the reference list of selected records will be checked to ensure all relevant articles are included in this scoping review (ie, backward reference tracking).

The selection process and the reasons for exclusion (at full text) will be evidenced using the PRISMA flow chart (see online supplemental material 2).

The search results and the study inclusion process will be reported in full in the final scoping review and presented in a flow chart.

Data items

Two reviewers will collaboratively extract the data and fill in the extraction form. This step will also be supported by the Covidence web-based software platform (Veritas Health Innovation). Subsequently, both reviewers will check that the information they extracted is similar; if there is a disagreement between the reviewers, a third reviewer will decide on the data to be extracted. Information will be extracted from the included studies: the characteristics of the publication (title, authors, year, journal of publication, country of the first author, institution of the first author), the design (ie, cross-sectional, cohort), characteristics of participants (% of women, age range, % of participants with higher education, % of participants from rural areas, brief description), characteristics of the instrument (measured variable, number of items, number of factors, operational definition), measurement properties (internal structure, invariance, reliability, relationship with other variables, sensitivity/specificity), funding and conflicts of interest.

Critical appraisal of individual sources of evidence

Although critical appraisal is traditionally not conducted in scoping reviews, it will be implemented in the present review because the aim is to evaluate the methodological quality of studies on measurement properties. We will report the risk of bias in the studies included, using the checklist developed by COSMIN (COnsensus-based Standards for the selection of health status Measurement INstruments).³² Thus, structural validity, internal consistency and reliability measures will be observed. Furthermore, using the Grading of Recommendations, Assesment, Development and Evaluations (GRADE) system, we will categorise the degree of quality of a given study, selecting the lowest grade of the reported options (‘very good’, ‘adequate’, ‘doubtful’ and ‘inadequate’).

Synthesis of results

First, a narrative synthesis will be made of the characteristics and measurement properties of the included studies, which will be stratified by each measurement instrument. Second, the results of the risk of bias analysis for each COSMIN criterion will be presented graphically. Third, a network visualisation bibliometric analysis will be performed using VOSviewer to identify clusters of coauthors and keywords.³³ The metadata of the included studies will be preprocessed before analysis in VOSviewer to standardise the metadata (creation of a thesaurus). Finally, evidence will be provided to visualise the available evidence on the measurement properties of each instrument.³⁴

Due to the previous information, there is an evident need to understand the level of the operational development and capacity of measurement of different instruments and their relationship with climate change and the impact on mental health. The evaluation and collection of data through psychometric instruments would allow evidence-based decision-making that responds to intersectoral needs and resources.^{35 36} Likewise, the health sector will be able to define and concentrate efforts on the adverse effects of climate change on physical and mental health at preventive and interventional levels.³⁷ Therefore, this scoping review aims to organise the available evidence to understand the state of the art better, develop future research, and support the decision-making of research, policymakers, and other stakeholders.

Ethics and dissemination

Our study does not involve an ethical risk for the participants as it reviews scientific papers available in bibliographical databases and does not collect primary or secondary data of any kind. In addition, our study is aligned with international PRISMA standards.

The authors thank Dr Axel Burger from Gesis Leibniz Institute in Mannheim, Germany, for his support in reviewing this manuscript. We also thank the University of Twente for supporting this manuscript through the Behavioural, Management and Social Sciences (BMS) Open Access Fund. The funding body had no role in the study design, manuscript writing, or submitting the paper to a specific journal.

Ethics statements

Patient consent for publication

Not required.

Footnote

Contributors ADR conceptualised and coordinated the project. ADR, DV-Z, SM-O and RRC-M drafted the methods. PEH-A, JOG-C and SMC-V wrote the introduction. All the authors developed, wrote and reviewed the manuscript.

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests None declared.

Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

References

¹ Hurlimann A, Cobbinah PB, Bush J, et al. Is climate change in the curriculum? An analysis of Australian urban planning degrees. Environ Educ Res 2021; 27: 970–91. doi:10.1080/13504622.2020.1836132

² United Nations Environment Programme. Emissions gap report 2022. 2022. Available: http://www.unep.org/resources/emissions-gap-report-2022 [Accessed 09 Dec 2022 ].

³ Jafino BA, Walsh B, Rozenberg J, et al. Revised estimates of the impact of climate change on extreme poverty by 2030. The World Bank, 2020. doi:10.1596/1813-9450-9417

⁴ Parker CL, Wellbery CE, Mueller M. The changing climate: managing health impacts. Am Fam Physician 2019; 100: 618–26.

⁵ Angra D, Sapountzaki K. Climate change affecting forest fire and flood risk—facts, predictions, and perceptions in central and South Greece. Sustainability 2022; 14: 13395. doi:10.3390/su142013395

⁶ Nazrul I, Winkel J. Climate change and social inequality. Department of Economic and Social Affairs. n.d. Available: https://www.un.org/en/desa/climate-change-and-social-inequality

⁷ Myers SS, Smith MR, Guth S, et al. Climate change and global food systems: potential impacts on food security and undernutrition. Annu Rev Public Health 2017; 38: 259–77. doi:10.1146/annurev-publhealth-031816-044356

⁸ Hrabok M, Delorme A, Agyapong VIO. Threats to mental health and well-being associated with climate change. J Anxiety Disord 2020; 76: 102295. doi:10.1016/j.janxdis.2020.102295

⁹ Albrecht G, Sartore G-M, Connor L, et al. Solastalgia: the distress caused by environmental change. Australas Psychiatry 2007; 15 Suppl 1: S95–8. doi:10.1080/10398560701701288

¹⁰ Usher K, Durkin J, Bhullar N. Eco-anxiety: how thinking about climate change-related environmental decline is affecting our mental health. Int J Ment Health Nurs 2019; 28: 1233–4. doi:10.1111/inm.12673

¹¹ Clayton S. Climate anxiety: psychological responses to climate change. J Anxiety Disord 2020; 74: 102263. doi:10.1016/j.janxdis.2020.102263

¹² O’Brien AJ, Elders A. Climate anxiety. When it’s good to be worried. J Psychiatr Ment Health Nurs 2022; 29: 387–9. doi:10.1111/jpm.12817

¹³ Comtesse H, Ertl V, Hengst SMC, et al. Ecological grief as a response to environmental change: a mental health risk or functional response Int J Environ Res Public Health 2021; 18: 734. doi:10.3390/ijerph18020734

¹⁴ Berry HL, Bowen K, Kjellstrom T. Climate change and mental health: a causal pathways framework. Int J Public Health 2010; 55: 123–32. doi:10.1007/s00038-009-0112-0

¹⁵ Clayton S, Manning CM, Speiser M, et al. Mental health and our changing climate: impacts, inequities, responses. American Psychological Association, 2021.

¹⁶ Howard M, Ahmed S, Lachapelle P, et al. Farmer and rancher perceptions of climate change and their relationships with mental health. J Rural Ment Health 2020; 44: 87–95. doi:10.1037/rmh0000131

¹⁷ Clayton S, Karazsia BT. Development and validation of a measure of climate change anxiety. J Environ Psychol 2020; 69: 101434. doi:10.1016/j.jenvp.2020.101434

¹⁸ Larionow P, Sołtys M, Izdebski P, et al. Climate change anxiety assessment: the psychometric properties of the Polish version of the climate anxiety scale. Front Psychol 2022; 13: 870392. doi:10.3389/fpsyg.2022.870392

¹⁹ Wullenkord MC, Tröger J, Hamann KRS, et al. Anxiety and climate change: a validation of the climate anxiety scale in a German-speaking quota sample and an investigation of psychological correlates. Clim Change 2021; 168: 20. doi:10.1007/s10584-021-03234-6

²⁰ Ágoston C, Urbán R, Nagy B, et al. The psychological consequences of the ecological crisis: three new questionnaires to assess ECO-anxiety, ECO-guilt, and ecological grief. Clim Risk Manag 2022; 37: 100441. doi:10.1016/j.crm.2022.100441

²¹ Ágoston C, Csaba B, Nagy B, et al. Identifying types of ECO-anxiety, ECO-guilt, ECO-grief, and ECO-coping in a climate-sensitive population: a qualitative study. Int J Environ Res Public Health 2022; 19: 2461. doi:10.3390/ijerph19042461

²² Cianconi P, Betrò S, Janiri L. The impact of climate change on mental health: a systematic descriptive review. Front Psychiatry 2020; 11: 74. doi:10.3389/fpsyt.2020.00074

²³ Martin G, Cosma A, Roswell T, et al. Measuring negative emotional responses to climate change among young people in survey research: a systematic review protocol. BMJ Open 2022; 12: e062449. doi:10.1136/bmjopen-2022-062449

²⁴ Martin G, Cosma A, Roswell T, et al. Measuring negative emotional responses to climate change among young people in survey research: a systematic review. Soc Sci Med 2023; 329: 116008. doi:10.1016/j.socscimed.2023.116008

²⁵ Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol 2005; 8: 19–32. doi:10.1080/1364557032000119616

²⁶ Peters MDJ, Godfrey CM, Khalil H, et al. Guidance for conducting systematic scoping reviews. Int J Evid Based Healthc 2015; 13: 141–6. doi:10.1097/XEB.0000000000000050

²⁷ Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-SCR): checklist and explanation. Ann Intern Med 2018; 169: 467–73. doi:10.7326/M18-0850

²⁸ Munn Z, Peters MDJ, Stern C, et al. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol 2018; 18: 143. doi:10.1186/s12874-018-0611-x

²⁹ Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021; 372: n71. doi:10.1136/bmj.n71

³⁰ ASReview. Asreview - active learning for systematic reviews. 2022. Available: https://asreview.nl/ [Accessed 10 Dec 2022 ].

³¹ van Dijk SHB, Brusse-Keizer MGJ, Bucsán CC, et al. Artificial intelligence in systematic reviews: promising when appropriately used. BMJ Open 2023; 13: e072254. doi:10.1136/bmjopen-2023-072254

³² Mokkink LB, Prinsen CAC, Bouter LM, et al. The consensus-based standards for the selection of health measurement instruments (COSMIN) and how to select an outcome measurement instrument. Braz J Phys Ther 2016; 20: 105–13. doi:10.1590/bjpt-rbf.2014.0143

³³ van Eck NJ, Waltman L. Software survey: Vosviewer, a computer program for Bibliometric mapping. Scientometrics 2010; 84: 523–38. doi:10.1007/s11192-009-0146-3

³⁴ Miake-Lye IM, Hempel S, Shanman R, et al. What is an evidence map? A systematic review of published evidence maps and their definitions, methods, and products. Syst Rev 2016; 5: 28. doi:10.1186/s13643-016-0204-x

³⁵ Konrad-Adenauer-Stiftung. Annual report 2021,. 2021 Available: https://www.kas.de/en/annual-reports/detail/-/content/jahresbericht-2021 [Accessed 10 Dec 2022 ].

³⁶ Figueroa R. Climate change and health [Cambio Climático Y Salud]. Rev Peru Med Exp Salud Publica 2016; 33: 7–9. doi:10.17843/rpmesp.2016.331.2006

³⁷ Burstein Roda T. Role of the health sector in the face of climate change [ROL del sector Salud ante El Cambio Climático]. Rev Peru Med Exp Salud Publica 2016; 33: 139–42. doi:10.17843/rpmesp.2016.331.2015