Correspondence to Ana F Oliveira; [email protected]
Strengths and limitations of this study
This review will apply the methodology of systematic review and meta-analysis to report the literature regarding the relationship between cancer-related cognitive impairment and work-related outcomes of adult non-central nervous system cancer survivors at working age.
The protocol of this work was written in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols guidelines, to ensure quality of the study in terms of reporting, planning and execution.
A broad and comprehensive search strategy will be used in multiple databases to maximise the identification of eligible studies, and study selection, data extraction and risk of bias assessment will be performed independently by two reviewers to ensure that the included studies are free from personal bias.
Unpublished articles/grey literature and publications that consist of abstracts only will not be included, due to lacking important data.
We anticipate possible heterogeneity across studies to be included, related to the population, type of treatment received, outcome measures and methods/tools, which may increase difficulties in interpreting the meta-analysis.
Introduction
Increased attention to the care of cancer survivors has led to a growing interest in studying how cancer diagnosis and cancer treatments affect their overall quality of life, their capacity to function and live independently, and their ability to return to work.1 2 Cancer-related cognitive impairment (CRCI), cognitive problems associated with cancer and cancer treatments experienced by individuals with cancer, is one of the most feared and commonly experienced problems reported by non-central nervous system (CNS) cancer survivors,2–6 with a significant impact on the functional ability and quality of life of survivors and their families.4 Although much of the research has focused on breast cancer, it also affects patients with other non-CNS cancers.7 8 Some of the most frequent complaints are related to difficulties with short-term memory, concentration, attention, executive functions, multitasking and processing speed.4 8 Cognitive impairments verified in this population are consistent with the patients’ self-report (through self-report questionnaires) and performance-based assessment (with neuropsychological tests).9 10 However, in some cases, subjective measures of cognitive impairment are poorly associated with objective measures, which indicates the need to include both measures in the assessment of cognitive function.9 11
Worldwide, 40%–50% of cancer survivors are of working age,12 which means that many cancer survivors are at an age at which cancer and its treatments could affect their career and work-related outcomes; this can have an important impact on individual and family lives.13 14 Work is associated with having a purpose in life, a sense of contributing and a distraction; it can also improve one’s self-esteem and sense of well-being, and provide financial security.15–17 Several studies have shown that survivors with cognitive impairments may experience challenges in their work,18–20 namely memory problems, reduced efficiency and impaired processing speed,5 19–21 attention and concentration problems,5 19 verbal ability, language competences or word finding difficulties,19 21 and problems with planning and executing their work.19–21 Therefore, due to CRCI, cancer survivors report less confidence and capability to do their job well,21 and less prospect of being promoted or assigned to important projects, since they and their employers realise they are no longer able to manage their premorbid level of work.5 Investigating the potential impact of CRCI on work-related outcomes is further complicated by the association these variables share with multiple factors, such as sociodemographic (eg, age, education), tumour (eg, tumour type, tumour stage), treatment modality (eg, chemotherapy, endocrine therapy), psychological (eg, anxiety, depression) and physiological (eg, fatigue, sleep) factors.2
Theoretical models suggest a link between cognitive limitations and work-related outcomes in cancer survivors. The latest model was developed by Mehnert et al22 and corresponds to a cancer survivorship and work model adapted from Feuerstein et al16 and Mehnert,14 considering several work-related outcomes, such as employment/return to work, work ability, work performance, job opportunities, income, work satisfaction, job promotion and training, and sustainability. Nevertheless, since the publication of these theoretical models, a growing body of research has been developed to study the relationship between CRCI and work-related outcomes, including some reviews to systematise studies’ findings,23–26 considering that many occupationally active survivors identify these symptoms as very problematic and bothersome.18
Von Ah et al23 were the first to explore the impact of cognitive impairment on work outcomes in 2016. Findings from this integrative review showed that most studies found cognitive impairment to be a common troubling symptom that had a negative impact on work-related outcomes, affecting work ability, job performance and productivity for cancer survivors returning to work after cancer and cancer treatments. Bijker et al24 performed a systematic review in 2018 that explored the association between functional impairments (including cognitive functioning) and work-related outcomes in breast cancer survivors. Findings were inconsistent across studies: studies measuring cognitive functioning with objective neuropsychological tests found no association with work-related outcomes, while results of studies using self-reported measures of cognitive functioning were ambiguous. Lewis and Mackenzie25 performed a scoping review, aiming to identify what is known about how cognitive changes impact work ability or performance for women with breast cancer. Although discrepancies were found between results from neuropsychological testing and self-report measures, breast cancer survivors can experience challenges in their employment due to cognitive deficits, which may lead to the loss of their employment. In the same year, Tan et al26 performed a systematic review of studies that quantified the impact of cancer-related symptoms (namely cognitive impairment) on work outcomes among cancer survivors. Results indicated that only a small proportion of studies assessing work status (employment status/return to work or early retirement/work disability) and cognitive impairment reported significant findings. The authors indicated that, ideally, estimates from studies should be pooled in meta-analysis for the purpose of quantitative synthesis; however, this approach was not feasible in their work.
To summarise, the collective conclusions of these reviews describe that cognitive functioning can impact work-related outcomes, although inconsistencies are found when cognitive functioning is evaluated by self-report and neuropsychological measures. We note that some reviews had a broader scope, since they explored the association between functional impairments or cancer-related symptoms (including, but not limited to, cognitive functioning) and work-related outcomes24 26; therefore, considering that cognitive functioning was not the main focus of the reviews, important studies might have been missed for non-CNS cancer survivors. Furthermore, considering that the reviews focused only on breast cancer24 25 or included all types of cancer, including CNS cancers,23 26 the conclusions for non-CNS cancer survivors may not be generalisable or may be confounding. Finally, despite the valuable contribution of these reviews to the current state of knowledge on the association between CRCI and work-related outcomes, this has been done via narrative synthesis; no review has quantitatively synthesised the literature specifically concerning CRCI and work-related outcomes. While narrative synthesis through text and tables is an initial, and in some cases the only, step to summarise and explain the characteristics and findings of the included studies and to provide an analysis of the relationships within and between studies, it is a more subjective process than quantitative synthesis, that is, meta-analysis.27 The meta-analysis will provide information on the magnitude of the effect, and will allow to investigate reasons for variations between studies and differences between studies and group of studies, and settle conflicting claims.28 This is important to clinical practice because, by combining information from all relevant studies, meta-analyses can provide more objective and precise estimates of the effects of healthcare than those derived from the individual studies included within a review, assisting clinicians decisions.29 30 Consequently, a quantitative investigation of this association is necessary and timely, as suggested by Tan et al.26
Considering the growing attention that has been given to this area in recent years, the primary aim of the present systematic review with meta-analysis is to comprehensively review the literature and synthesise relevant data to identify the relationship between CRCI and work-related outcomes of adult non-CNS cancer survivors at working age, and to quantify this association, exploring how strong and consistent is the relationship across studies. If possible, a secondary examination of patient and clinical characteristics (eg, age, type of cancer, type of treatment), psychological variables (eg, depression, anxiety) and other influencing factors (eg, fatigue, sleep) and their influence on any association between CRCI and work-related outcomes will also be conducted. Therefore, this work will allow us to further understand the relationship between CRCI and work-related outcomes and help propose recommendations for clinical practice and interventions. Previous CRCI interventional research has been limited and has failed to examine implications on work-related outcomes,23 although recent efforts have been made to address this gap.31 32 Therefore, findings from this work can guide the development of patient-centred interventions to address implications of CRCI on work-related outcomes, considering the relationships between these two variables.
Methods and analysis
In accordance with the guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Protocols (PRISMA-P) 2015 statement33 34 (see Research Checklist), our systematic review protocol was established and registered with the International Prospective Register of Systematic Reviews (PROSPERO) database.
The systematic review will be conducted following the PRISMA 2020 updated guidelines.35
Eligibility criteria
Eligibility criteria were established according to the Population, Intervention/Exposure, Comparisons, Outcome and Study type (PICOS) framework25 36 37 and considering the International Cognition and Cancer Task Force (ICCTF) recommendations for the study of CRCI.38 39 Therefore, studies will be included according to the following criteria: (1) Population—Adults with a non-CNS cancer diagnosis (survivors) at working age (18 years or older) with CRCI; (2) Interventions/Exposure—Non-CNS cancer survivors exposed to any cancer-related treatments, with primary treatment completed at the time of the study; (3) Comparisons—No comparison group will be required; () Outcomes—Association between CRCI and work-related outcomes and, where available, associations between CRCI and work-related outcomes and additional potential moderator variables (eg, measures of anxiety, depression) and (5) Study type/design—Quantitative studies. Further details about the eligibility criteria are outlined in table 1. The primary search question thus formulated is ‘What is the relationship between cognitive functioning and work-related outcomes in adult non-CNS cancer survivors at working age provided by quantitative evidence?’.
Table 1PICOS framework components and description for the eligibility criteria
| PICOS framework components | Description |
| Population | Only survivors of adult-onset cancers (ie, diagnosed with cancer at 18 years or older) will be considered, considering the potential developmental impact on cognitive functioning. Survivors of CNS cancers will be excluded, due to potential differences in cognitive function between non-CNS and CNS cancer survivors. When studies with mixed populations have more than 20% of participants meeting exclusion criteria, they will be excluded37; authors of these articles will also be contacted to clarify this issue and confirm if they did not include CNS cancer survivors. No setting restrictions will be considered. |
| Interventions/exposure | No restrictions will be applied regarding type of cancer treatments. Cancer survivors should not have been engaged in primary treatment (radiotherapy, chemotherapy, immunotherapy) at the time of enrolment, due to the focus of this work being related to late survivorship issues; hormone-related therapies will not be considered primary treatment and therefore studies reporting that will not be excluded. |
| Comparisons | Although a comparison group will not be required, studies including a comparison/control group (disease specific and/or healthy controls) will be included, whereas the outcomes of interest are evaluated for the population. |
| Outcomes | Report a measure of cognition (assessed through subjective self-report measures and/or by objective neuropsychological assessment instruments) and have measured a work-related outcome (assessed by subjective self-report measures and/or by patient’s perspective; (see14 16 22 for the models considered for the types of work-related outcomes included)). Associations between the cognitive and work-related outcomes need to be directly reported to be included. Studies that focus on psychological or mental health that do not have a separate measure of cognitive functioning and those in which the cognitive measure is delirium/dementia/geriatric-related will be excluded. Additionally, studies that do not have an identified work-related outcome will also be excluded, considering that the aim of this review is to examine the impact of CRCI on work-related outcomes. Secondary outcomes, such as age, type of cancer and treatment, anxiety, depression, fatigue and sleep, will also be considered if they are reported as potential moderator variables on the association between CRCI and work-related outcomes. |
| Study design/type | Empirical articles published (or ahead of print) in a peer-reviewed journal that report original quantitative data will be included, both cross-sectional, longitudinal or retrospective (only baseline data will be extracted where multiple timepoint assessments are made). Studies reporting an intervention or only qualitative evidence will be excluded. Unpublished articles/grey literature (eg, thesis, conference proceedings, technical reports) and publications that consist of abstracts only (eg, conference abstracts) will be excluded, due to lacking important data. Literature/systematic reviews or meta-analysis, case studies and studies that do not report original data (eg, commentaries, editorials) will not be included. |
CNS, central nervous system; CRCI, cancer-related cognitive impairment.
For the purpose of this article, the following definitions are used to avoid ambiguity in terms of cognitive functioning, work-related outcomes and non-CNS cancer survivors. Cognitive functioning refers to a higher order mental process related to the capacity to process information, regulated by numerous areas of the brain. It consists of a multidimensional concept encompassing multiple inter-related domains, including attention and concentration, executive function, information processing speed, language, psychomotor function, visuospatial ability and learning and memory.40–42 Therefore, impairments in cognitive functioning correspond to a decline in function in one or more of these cognitive processes.41 42 Work-related outcomes are conceptualised following the ‘Cancer Survivorship and Work Model’ by Mehnert et al,22 which is an adaptation from Feuerstein et al16 and Mehnert14 models, considering several work-related outcomes (detailed information about these outcomes are defined in the aforementioned articles). CNS tumours may either be primary, originating and developing within the CNS (eg, gliomas, meningioma, medulloblastoma and primary CNS lymphoma) or metastatic, originating in non-CNS tissue and migrating to the CNS. Therefore, in the present work, non-CNS tumours refer to diagnosis other than CNS location, including, for example, breast, colorectal, prostate, skin, head and neck, or lymphoma.42 43 Non-CNS cancer survivors refers to men and women who have been diagnosed with a non-CNS cancer and have completed primary treatment, regardless of stage, time since diagnosis and type of treatment.24
Studies will be included if they are written in English, Portuguese, French or Spanish to avoid idiom-related bias; considering the language knowledge and proficiency of the review team in these languages and the lack of resources to include articles written in other languages, only studies written in these languages will be considered. The authors will search for articles from January 2000 onwards, considering that CRCI began to be recognised around this date.25
Information sources
The literature search will be performed by the first author by searching title, abstract and keyword fields. The following six electronic databases will be searched for the relevant publications: Web of Science, Scopus, PubMed, ProQuest, PsycINFO and CINAHL (through EBSCOhost). Searches in these databases will be complemented by a manual search of the reference lists of other relevant and key publications related to CRCI and work-related outcomes in cancer survivors (eg, systematic reviews excluded at title/abstract stage, studies mentioned in screened and/or included articles) to include any additional studies not previously identified and to ensure saturation of data. A snowball procedure will also be used, to manually search the references cited in the included articles to identify additional studies. When these studies are not duplicates already included in the search, they will be added for screening. Attempts will be made to find unavailable articles by contacting authors, as well as to clarify information.
Search strategy
The search strategy will encompass the identification of the main terms/concepts based on the PICOS framework, consisting of three primary topics: “cancer”, “cognition”, and “work”. The authors will also identify associated keywords and/or synonyms; different spellings; singular/plural forms, verbal forms, and adjectives (eg, searching the keywords of key articles and other reviews of the topic), and controlled vocabulary/medical descriptors (eg, Medical Subject Headings (MeSH) terms) to improve the sensitivity of the search and obtain the maximum number of publications (table 2).
Table 2Main search terms/concepts, free-text terms and medical headings
| Main search terms/concepts | Keywords and/or synonyms, different spellings, singular/plural forms | MeSH terms |
| 1) Cancer | cancer* OR oncolog* OR neoplasm* OR tumour* OR tumor* OR carcinoma* OR malignan* OR “non-central nervous system” AND surviv* OR patient* | Neoplasms Carcinoma Cancer Survivors |
| 2) Cognition | cogniti* OR cognition OR cognitive OR “cognitive functioning” OR “cognitive impairment” OR “cognitive concern*” OR “cognition disorder” | Cognition Cognition Disorders Cognitive Dysfunction |
| 3) Work | work OR “work-related outcome*” OR occupation* OR vocational* OR “work function*” OR “work capacity” OR “work activity” OR “work status” OR “work ability” OR “work limitation*” OR “return to work” OR “work performance” OR “work productivity” OR “work* hours” OR “work retention” OR “job retention” OR “work environment” OR “work tolerance schedule” OR employ* OR unemployment OR “job performance” OR “job accommodation” OR absenteeism OR “sick leave” | Work Employment Absenteeism Sick Leave Return to Work Work Performance Unemployment |
MeSH, Medical Subject Headings.
The truncation symbol (*), quotation marks (“ ”) and Boolean terms (OR, AND) will be applied to combine the different search terms/concepts and to narrow the search based on the eligibility criteria; multiple search terms will be used for each main search term/concept combined with the “OR” operator, and the “AND” operator will be used between terms within each of the main search terms/concepts. Specific filters related to publication date, language, and document type will be used whenever possible, considering the eligibility criteria mentioned above. The search terms will be adjusted to the specificities of the different databases. Detailed information about the draft search strategy is available in table 3 to facilitate its replication.
Table 3Draft of search strategy
| Database | Query |
| Web of Science | TS=(cancer* OR oncolog* OR neoplasm* OR tumour* OR tumor* OR carcinoma* OR malignan* OR “non-central nervous system”) AND TS=(surviv* OR patient*) AND TS=(cogniti* OR cognition OR cognitive OR “cognitive functioning” OR “cognitive impairment” OR “cognitive concern*” OR “cognition disorder”) AND TS=(work OR “work-related outcome*”) AND TS=(occupation* OR vocational* OR “work function*” OR “work capacity” OR “work activity” OR “work status” OR “work ability” OR “work limitation*” OR “return to work” OR “work performance” OR “work productivity” OR “work* hours” OR “work retention” OR “job retention” OR “work environment” OR “work tolerance schedule” OR employ* OR unemployment OR “job performance” OR “job accommodation” OR absenteeism OR “sick leave”) |
| Scopus | TITLE-ABS-KEY(cancer* OR oncolog* OR neoplasm* OR tumour* OR tumor* OR carcinoma* OR malignan* OR “non-central nervous system”) AND TITLE-ABS-KEY(surviv* OR patient*) AND TITLE-ABS-KEY(cogniti* OR cognition OR cognitive OR “cognitive functioning” OR “cognitive impairment” OR “cognitive concern*” OR “cognition disorder”) AND TITLE-ABS-KEY(work OR “work-related outcome*” OR occupation* OR vocational* OR “work function*” OR “work capacity” OR “work activity” OR “work status” OR “work ability” OR “work limitation*” OR “return to work” OR “work performance” OR “work productivity” OR “work* hours” OR “work retention” OR “job retention” OR “work environment” OR “work tolerance schedule” OR employ* OR unemployment OR “job performance” OR “job accommodation” OR absenteeism OR “sick leave”) |
| PubMed | ((cancer*[Title/Abstract] OR oncolog*[Title/Abstract] OR neoplasms[MeSH Terms] OR neoplasm* OR tumour*[Title/Abstract] OR tumor*[Title/Abstract] OR carcinoma[MeSH Terms] OR carcinoma* OR malignan* OR "non-central nervous system") AND (surviv*[Title/Abstract] OR patient*[Title/Abstract] OR cancer survivors[MeSH Terms]) AND (cogniti*[Title/Abstract] OR cognition[MeSH Terms] OR cognition disorders[MeSH Terms] OR cognitive dysfunction[MeSH Terms] OR cognitive[Title/Abstract] OR "cognitive functioning"[Title/Abstract] OR "cognitive impairment"[Title/Abstract] OR "cognitive concern*" OR "cognition disorder") AND (work[Title/Abstract] OR "work-related outcome*"[Title/Abstract] OR occupation* OR vocational* OR "work function*" OR "work capacity" OR "work activity" OR "work status" OR "work ability" OR "work limitation*" OR return to work[MeSH Terms] OR work performance[MeSH Terms] OR "work productivity" OR "work* hours" OR "work retention" OR "job retention" OR "work environment" OR "work tolerance schedule" OR employment[MeSH Terms] OR employ* OR unemployment[MeSH Terms] OR "job performance" OR "job accommodation" OR absenteeism[MeSH Terms] OR sick leave[MeSH Terms])) |
| ProQuest | ab((cancer* OR oncolog* OR neoplasm* OR tumour* OR tumor* OR carcinoma* OR malignan* OR “non-central nervous system”) AND (surviv* OR patient*) AND (cogniti* OR cognition OR cognitive OR “cognitive functioning” OR “cognitive impairment” OR “cognitive concern*” OR “cognition disorder”) AND (work OR “work-related outcome*” OR occupation* OR vocational* OR “work function*” OR “work capacity” OR “work activity” OR “work status” OR “work ability” OR “work limitation*” OR “return to work” OR “work performance” OR “work productivity” OR “work* hours” OR “work retention” OR “job retention” OR “work environment” OR “work tolerance schedule” OR employ* OR unemployment OR “job performance” OR “job accommodation” OR absenteeism OR “sick leave”)) NOT ab(“review”) |
| PsycInfo | AB(cancer* OR oncolog* OR neoplasm* OR tumour* OR tumor* OR carcinoma* OR malignan* OR “non-central nervous system”) AND AB(surviv* OR patient*) AND AB(cogniti* OR cognition OR cognitive OR “cognitive functioning” OR “cognitive impairment” OR “cognitive concern*” OR “cognition disorder”) AND AB(work OR “work-related outcome*”) AND TX(occupation* OR vocational* OR “work function*” OR “work capacity” OR “work activity” OR “work status” OR “work ability” OR “work limitation*” OR “return to work” OR “work performance” OR “work productivity” OR “work* hours” OR “work retention” OR “job retention” OR “work environment” OR “work tolerance schedule” OR employ* OR unemployment OR “job performance” OR “job accommodation” OR absenteeism OR “sick leave”) |
| CINAHL | AB(cancer* OR oncolog* OR neoplasm* OR tumour* OR tumor* OR carcinoma* OR malignan* OR “non-central nervous system”) AND AB(surviv* OR patient*) AND AB(cogniti* OR cognition OR cognitive OR “cognitive functioning” OR “cognitive impairment” OR “cognitive concern*” OR “cognition disorder”) AND AB(work OR “work-related outcome*”) AND TX(occupation* OR vocational* OR “work function*” OR “work capacity” OR “work activity” OR “work status” OR “work ability” OR “work limitation*” OR “return to work” OR “work performance” OR “work productivity” OR “work* hours” OR “work retention” OR “job retention” OR “work environment” OR “work tolerance schedule” OR employ* OR unemployment OR “job performance” OR “job accommodation” OR absenteeism OR “sick leave”) |
Data management
All literature results identified during the database search will be imported into Mendeley, and then duplicate studies will be removed based on title and author. Remaining results will be exported into Rayyan QCRI (Qatar Computing Research Institute) online software (https://rayyan.qcri.org/), a support tool for references selection in the framework of systematic reviews that facilitates collaboration among reviewers.
Selection process
The selection process will be conducted by the first and second authors considering the review team’s preestablished inclusion and exclusion criteria. The two authors will independently review the titles and abstracts of all identified search results, labelling them as included, excluded or maybe. Articles labelled as included and maybe will then be retrieved for full-text examination, and the full text will be reviewed, in both situations. For articles that will be excluded, reasons for not including them will be documented. The authors will seek additional information from study authors where necessary to resolve questions about eligibility. All documents raising any doubts or disagreements will be discussed and resolved in a consensus meeting; a third author of the review team will be consulted in cases in which consensus is not achieved.
Inter-rater agreement will be assessed and interpreted using Cohen’s kappa coefficient to explore the consistency of the study selection performed by the two authors (kappa values: <0—less than chance agreement; 0.01–0.20—slight agreement; 0.21–0.40—fair agreement; 0.41–0.60—moderate agreement; 0.61–0.80—substantial agreement; 0.81–0.99—almost perfect agreement).44 This analysis will be performed using IBM SPSS V.28.0 (IBM).
The PRISMA flow diagram will be displayed to provide details on the selection process of the studies, documenting included and excluded studies with the reasons for exclusion33 34 36 as presented in figure 1.
Enlarge this image.Figure 1. PRISMA flow diagram presenting the selection process for the studies. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.
Data collection process
The main information from all eligible studies will be independently extracted by the first and second authors to ensure that all relevant information is captured and to minimise risk of bias. When the studies have assessed multiple outcomes, only the information that is relevant to this systematic review research question will be extracted. The same standard data extraction form will be used to charter the descriptive data about each study. Extracted data will be confirmed by the two researchers, and disputes will be resolved through consultation and referring to data in original papers; the data extracted will be reviewed and validated by the third author. The authors will also contact the study author to resolve any uncertainties or if there are missing and incomplete data about study characteristics, methods or measures used, or where clarification on data is needed.
Data items
The following information will be extracted from all studies selected (where available): (1) information about the article; (2) participants’ characteristics; (3) characteristics of the study; (4) data collection and (5) main findings/conclusions of the study. Table 4 presents the items that will be collected.
Table 4Data items to be collected from the selected articles
| Information about the article | Participant’s characteristics | Characteristics of the study | Data collection | Main findings/conclusion of the study |
| First author | Sample size | Study design | Outcome measures of cognitive functioning | Variables in association |
| Year of publication | Age(mean, SD) | Study setting | Outcome measures of work-related outcomes | Data extraction for correlation (with p values) |
| Country of origin | Gender | Methodology | Outcome measures of other relevant variables | Interpretation |
| Study main aims | Cancer type | |||
| Inclusion/exclusion criteria |
Outcomes and prioritisation
The primary outcome of this work is the relationship between cognitive functioning (subjective or objective, assessed through self-report questionnaires and by neuropsychological tests, respectively) and work-related outcomes (assessed through self-report questionnaires or from the patient’s perspective; see the Eligibility criteria section for a description of the work-related outcomes). Among the most common self-report measures of subjective cognitive functioning are the Functional Assessment of Cancer Therapy-Cognitive Function version 3 and the European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire-C30 (EORTC QLQ-C30)-Cognitive Functioning Scale.4 Objective cognitive functioning is usually assessed with neuropsychological tests; several tests have been used across studies, but the ICCTF recommends prioritising the use of the Hopkins Verbal Learning Test-Revised, the Trail Making Tests parts A and B and the Controlled Oral Word Association of the Multilingual Aphasia Examination to harmonise CRCI research studies.4 38 39 The Work Limitations Questionnaire and the Work Ability Index are two examples of the most commonly used self-report validated measures to assess work-related outcomes; other outcomes are more frequently reported from the patient’s perspective (eg, absenteeism, return to work, working hours).23–26
Secondary outcomes, such as anxiety and depression, fatigue and sleep (measured with self-reported instruments, for instance, the Hospital Anxiety and Depression Scale and the EORTC QLQ-C30, respectively26), age, type of cancer and type of treatment (as presented in demographic and clinical characterisation of the sample), will also be considered.
Methodological quality (risk of bias) in individual studies
Each study will be rated by two reviewers (first and second authors) independently using two types of procedures. The methodology quality of the retrieved articles will be critically appraised following the PRISMA 2020 recommendations.36 The authors will use an appropriate tool for the study design to assess risk of bias in the included studies. To keep the appraisals consistent, the Joanna Briggs Institute (JBI) Critical Appraisal Tool Studies will be used. The goal of assessment of risk of bias (critical appraisal) by using these checklists is to assess the methodological quality of a study and to determine the possibility of bias in its design, conduct and analysis.45 The checklists consist of several items with the response options ‘yes’, ‘no’, ‘unclear’ and ‘not applicable’, and cover sample details, the validity and the reliability of the applied measures, identification of confounders, follow-up time and losses, and the adequacy of the statistical analysis applied. The critical appraisal checklists used will be the JBI Critical Appraisal Checklist for Analytical Cross-Sectional Studies (8-item checklist) for cross sectional studies and the JBI Critical Appraisal Checklist for Cohort Studies (12-item checklist) to appraise cohort studies. Each study receives a total score consisting of how many items on the appropriate appraisal tool are satisfactorily met: out of 8 and out of 12, respectively. These checklists can be consulted online at https://jbi.global/critical-appraisal-tools.
The level of evidence, strengths and limitations of each manuscript included in the systematic review will also be evaluated. Level of evidence will be assigned based on the criteria of the Rating System for the Hierarchy of Evidence for Intervention/Treatment Questions developed by Melnyk and Fineout-Overholt46: level I (Evidence from a systematic review or meta-analysis of all relevant RCTs), level II (Evidence obtained from well-designed RCTs), level III (Evidence obtained from well-designed controlled trials without randomisation), level IV (Evidence from well-designed case–control and cohort studies), level V (Evidence from systematic reviews of descriptive and qualitative studies), level VI (Evidence from single descriptive or qualitative studies) and level VII (Evidence from the opinion of authorities and/or reports of expert committees).
Study methodology and level of evidence ratings will be used to assess the robustness and confidence in study findings. Ratings of each reviewer will be compared using inter-rater agreement (Cohen’s kappa),44 and any disagreements will be resolved by discussion during a consensus meeting; a third member of the review team will be consulted if a consensus is not reached.
Data synthesis
A systematic narrative synthesis47 48 will be conducted with information presented in text and tables to summarise and explain the methodological characteristics, strengths and limitations, and findings of the included studies. Tables and narrative summaries will compile and explore the relationship and findings both within and between included studies, in line with the guidance from the PRISMA 2020 recommendations36 and the Centre for Reviews and Dissemination.27 Synthesis will be conducted by the first and second authors, and reviewed by the third author, with discussion and final agreement involving all review authors.
This systematic review will include a quantitative meta-analysis. These statistical analyses will be performed using the meta-package49 in statistical software R based on the random effects model. Meta-analysis will be performed by the third and fourth authors. The p values will be two sided and values <0.05 will be considered statistically significant. The effect of interest is the association between CRCI and work-related outcomes expressed as a correlation. We will calculate the pooled z values using a Pearson correlation coefficient transformed by the Fisher z-transformation. If study values for the Pearson correlation coefficient (r) are not available, the Pearson correlation coefficient (r) will be calculated from the existing Spearman correlation coefficient (rs), standardised regression coefficient (β), or OR. Using this transformation, it will be possible to convert the data for correlation to z-scores (normal distribution) to obtain approximate normality and then calculate the mean and standard errors of the transformed correlation. The transformation will be performed taking into account Lipsey and Wilson50 recommendations, using the Campbell Collaboration online calculator (https://www.campbellcollaboration.org/escalc/html/EffectSizeCalculator-R3.php).51–53
Statistical consistency and heterogeneity of the studies will be tested by the χ2 test and quantified by the Higgins I2 statistic. Studies with an I2 value of <25%, ~50% and ~75% will be considered to have low, moderate and high heterogeneities, respectively.54
Subgroup analysis
Subgroup analyses will be conducted to investigate the possible source(s) of heterogeneity. To access the stability of the meta-analysis results, a sensitivity analysis will be conducted by omitting individual studies in turn and transforming the random effect model into the fixed-effects model. Furthermore, as the existence of subgroups and heterogeneity is expected, we will undertake random-effects meta-regression analyses if at least 10 studies were collected regarding the primary outcome; this analysis will allow us to examine the impact of important factors and effect modifiers on our results. We will conduct meta-regression analyses to examine, for instance, the association between variables known to adversely impact CRCI and work-related outcomes, including, among others, age, type of cancer and treatment, and presence of anxiety or depression. Meta-regression analyses will serve to investigate unexplained heterogeneity between studies. Each study will be weighted in the regression models using the inverse of its variance; studies with the lowest amount of variance will be given a bigger weight in the regression model than those with the largest amount of variance. The association between each variable of interest and the primary outcome will be illustrated in table format where, for each variable, we will report its regression coefficient (B), SE, 95% CI, and statistical significance. To perform the meta-regressions, we will use the r2jags package55 in statistical software R.
Meta-bias(es)
Visual inspection of funnel plots for overall survival and a complemental Egger’s test to quantify the plot’s asymmetry will be used to determine the potential publication bias.
Confidence in cumulative evidence
The strength of the body of evidence will be assessed using the Grading of Recommendations Assessment, Development and Evaluation system.56
Patient and public involvement
This article reports a protocol of a systematic review with meta-analysis that will be based on previously published data. Therefore, patients and/or public were not involved.
Discussion
Understanding the impact of CRCI on work-related outcomes is important, considering the implications for quality of life and other areas of the cancer survivor life.23 To the best of our knowledge, this study will be the first to present a systematic review with meta-analysis of quantitative studies published in peer-reviewed journals regarding the relationship between CRCI and work-related outcomes. Following the PRISMA-P guidelines, the protocol was previously registered in PROSPERO, to help avoid duplication. Multiple bibliographic databases will be systematically searched to ensure saturation of data, and two independent reviewers will perform study selection, data extraction and risk of bias assessment to reduce personal bias. This review will advance the field of CRCI research by investigating how CRCI influences work-related outcomes and examining how strong this relationship is, and, consequently, this will allow to identify potential targets of intervention, proposing clinical implications of the findings.
It should be noted that there might be limitations in this review. There may be some clinical heterogeneities due to differences in the type of cancer, type of treatment received, outcome measures and methods/tools, as well as due to definitions of work-related outcomes among the included studies. This may have a certain impact on the results of the meta-analysis. There may also be a risk of publication bias as we will only include published articles. Finally, this review will not include qualitative studies, considering that the primary goal is to determine the strength of the relationship between CRCI and work-related outcomes, given that the existence of a relationship has already been documented previously;23 however, this may limit the contextualisation of our findings by not having a complete understanding of the cancer survivors’ experiences.
Ethics and dissemination
Ethics approval is not required for this review, because it relies on secondary data. Any modification/amendment to the systematic review protocol will be submitted to review and approval on the PROSPERO registry and described in the final report of the systematic review. The work is planned to be completed by February 2023. The systematic review findings will be published in a peer-review indexed journal, presented at scientific meetings and included in a chapter of a Doctoral thesis (first author).
The authors thank Professor Michael Feuerstein for comments on a previous version of this protocol and Professor Vera Afreixo for comments on this version.
Ethics statements
Patient consent for publication
Not applicable.
Contributors AFO is the guarantor of the review. AFO conceived the study idea, and together with SF, AT, IMS and DVA developed the design of the protocol, including development of the search strategy, selection criteria, risk of bias assessment strategy and data extraction criteria. JDR and AT designed the meta-analysis protocol. DVA provided expertise on CRCI and work-related outcomes. AFO and SF wrote the first draft of the manuscript. All authors critically read, provided feedback and approved the final manuscript.
Funding This work was supported by National Funds through FCT - Fundação para a Ciência e a Tecnologia, I.P. (http://www.fct.pt/), within the R&D Units CINTESIS - Center for Health Technology and Services Research (reference UIDB/4255/2020 and UIDP/4255/2020) and WJCR - William James Center for Research (reference UIDB/04810/2020), and a PhD fellowship to the first author (SFRH/BD/138785/2018); and by The Ohio State University Comprehensive Cancer Center (OSUCCC) (PI: Von Ah).
Disclaimer The funding agencies had no role in the study design; decision to publish the manuscript; or preparation of the manuscript.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
1 Asher A. Cognitive dysfunction among cancer survivors. Am J Phys Med Rehabil 2011; 90: S16–26. doi:10.1097/PHM.0b013e31820be463 http://www.ncbi.nlm.nih.gov/pubmed/21765260
2 Ahles TA, Root JC. Cognitive effects of cancer and cancer treatments. Annu Rev Clin Psychol 2018; 14: 425–51. doi:10.1146/annurev-clinpsy-050817-084903 http://www.ncbi.nlm.nih.gov/pubmed/29345974
3 Ahles TA, Root JC, Ryan EL. Cancer- and cancer treatment-associated cognitive change: an update on the state of the science. J Clin Oncol 2012; 30: 3675–86. doi:10.1200/JCO.2012.43.0116 http://www.ncbi.nlm.nih.gov/pubmed/23008308
4 Mayo SJ, Lustberg M, M Dhillon H, et al. Cancer-Related cognitive impairment in patients with non-central nervous system malignancies: an overview for oncology providers from the MASCC neurological complications Study Group. Support Care Cancer 2021; 29: 2821–40. doi:10.1007/s00520-020-05860-9 http://www.ncbi.nlm.nih.gov/pubmed/33231809
5 Boykoff N, Moieni M, Subramanian SK. Confronting chemobrain: an in-depth look at survivors' reports of impact on work, social networks, and health care response. J Cancer Surviv 2009; 3: 223–32. doi:10.1007/s11764-009-0098-x http://www.ncbi.nlm.nih.gov/pubmed/19760150
6 Wefel JS, Kesler SR, Noll KR, et al. Clinical characteristics, pathophysiology, and management of noncentral nervous system cancer-related cognitive impairment in adults. CA Cancer J Clin 2015; 65: 123–38. doi:10.3322/caac.21258 http://www.ncbi.nlm.nih.gov/pubmed/25483452
7 Allen DH, Myers JS, Jansen CE, et al. Assessment and management of cancer- and cancer treatment-related cognitive impairment. J Nurse Pract 2018; 14: 217–24. doi:10.1016/j.nurpra.2017.11.026 http://www.ncbi.nlm.nih.gov/pubmed/30906237
8 Cerulla Torrente N, Navarro Pastor J-B, de la Osa Chaparro N. Systematic review of cognitive sequelae of non-central nervous system cancer and cancer therapy. J Cancer Surviv 2020; 14: 464–82. doi:10.1007/s11764-020-00870-2 http://www.ncbi.nlm.nih.gov/pubmed/32146576
9 O'Farrell E, MacKenzie J, Collins B. Clearing the air: a review of our current understanding of "chemo fog". Curr Oncol Rep 2013; 15: 260–9. doi:10.1007/s11912-013-0307-7 http://www.ncbi.nlm.nih.gov/pubmed/23483375
10 Lange M, Castel H, Le Fel J, et al. How to assess and manage cognitive impairment induced by treatments of non-central nervous system cancer. Neurosci Biobehav Rev 2019; 107: 602–14. doi:10.1016/j.neubiorev.2019.09.028 http://www.ncbi.nlm.nih.gov/pubmed/31553925
11 Costa DSJ, Fardell JE. Why are objective and perceived cognitive function weakly correlated in patients with cancer? J Clin Oncol 2019; 37: 1154–8. doi:10.1200/JCO.18.02363 http://www.ncbi.nlm.nih.gov/pubmed/30920881
12 de Boer AGEM, AGEM deB. The European cancer and work network: CANWON. J Occup Rehabil 2014; 24: 393–8. doi:10.1007/s10926-013-9474-5 http://www.ncbi.nlm.nih.gov/pubmed/24002638
13 de Boer AGEM, Taskila T, Ojajärvi A, et al. Cancer survivors and unemployment: a meta-analysis and meta-regression. JAMA 2009; 301: 753–62. doi:10.1001/jama.2009.187 http://www.ncbi.nlm.nih.gov/pubmed/19224752
14 Mehnert A. Employment and work-related issues in cancer survivors. Crit Rev Oncol Hematol 2011; 77: 109–30. doi:10.1016/j.critrevonc.2010.01.004 http://www.ncbi.nlm.nih.gov/pubmed/20117019
15 de Boer AG, Torp S, Popa A, et al. Long-Term work retention after treatment for cancer: a systematic review and meta-analysis. J Cancer Surviv 2020; 14: 135–50. doi:10.1007/s11764-020-00862-2 http://www.ncbi.nlm.nih.gov/pubmed/32162193
16 Feuerstein M, Todd BL, Moskowitz MC, et al. Work in cancer survivors: a model for practice and research. J Cancer Surviv 2010; 4: 415–37. doi:10.1007/s11764-010-0154-6 http://www.ncbi.nlm.nih.gov/pubmed/20945110
17 Von Ah D, Russell KM, Storniolo AM, et al. Cognitive dysfunction and its relationship to quality of life in breast cancer survivors. Oncol Nurs Forum 2009; 36: 326–36. doi:10.1188/09.ONF.326-334 http://www.ncbi.nlm.nih.gov/pubmed/19596650
18 Duijts SFA, van der Beek AJ, Boelhouwer IG, et al. Cancer-Related cognitive impairment and patients' ability to work: a current perspective. Curr Opin Support Palliat Care 2017; 11: 19–23. doi:10.1097/SPC.0000000000000248 http://www.ncbi.nlm.nih.gov/pubmed/27898512
19 Von Ah D, Habermann B, Carpenter JS, et al. Impact of perceived cognitive impairment in breast cancer survivors. Eur J Oncol Nurs 2013; 17: 236–41. doi:10.1016/j.ejon.2012.06.002 http://www.ncbi.nlm.nih.gov/pubmed/22901546
20 Klaver KM, Duijts SFA, Engelhardt EG, et al. Cancer-Related cognitive problems at work: experiences of survivors and professionals. J Cancer Surviv 2020; 14: 168–78. doi:10.1007/s11764-019-00830-5 http://www.ncbi.nlm.nih.gov/pubmed/31768861
21 Munir F, Burrows J, Yarker J, et al. Women's perceptions of chemotherapy-induced cognitive side affects on work ability: a focus group study. J Clin Nurs 2010; 19: 1362–70. doi:10.1111/j.1365-2702.2009.03006.x http://www.ncbi.nlm.nih.gov/pubmed/20500346
22 Mehnert A, de Boer A, Feuerstein M. Employment challenges for cancer survivors. Cancer 2013; 119 Suppl 11: 2151–9. doi:10.1002/cncr.28067 http://www.ncbi.nlm.nih.gov/pubmed/23695927
23 Von Ah D, Storey S, Tallman E, et al. Cancer, cognitive impairment, and work-related outcomes: an integrative review. Oncol Nurs Forum 2016; 43: 602–16. doi:10.1188/16.ONF.602-616 http://www.ncbi.nlm.nih.gov/pubmed/27541553
24 Bijker R, Duijts SFA, Smith SN, et al. Functional impairments and work-related outcomes in breast cancer survivors: a systematic review. J Occup Rehabil 2018; 28: 429–51. doi:10.1007/s10926-017-9736-8 http://www.ncbi.nlm.nih.gov/pubmed/29086111
25 Lewis J, Mackenzie L. Cognitive changes after breast cancer: a scoping review to identify problems encountered by women when returning to work. Disabil Rehabil 2021: 1–19. doi:10.1080/09638288.2021.1919216 http://www.ncbi.nlm.nih.gov/pubmed/33974469
26 Tan CJ, Yip SYC, Chan RJ, et al. Investigating how cancer-related symptoms influence work outcomes among cancer survivors: a systematic review. J Cancer Surviv 2021. doi: doi:10.1007/s11764-021-01097-5. [Epub ahead of print: 23 Aug 2021 ]. http://www.ncbi.nlm.nih.gov/pubmed/34424498
27 Centre for Reviews and Dissemination. Systematic Reviews: CRD’s guidance for undertaking reviews in health care University of York; 2009: 283 p.
28 Munn Z, Tufanaru C, Aromataris E. Data extraction and synthesis. Am J Nurs 2014; 114: 49–54.
29 Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ 2009; 339: b2700. doi:10.1136/bmj.b2700 http://www.ncbi.nlm.nih.gov/pubmed/19622552
30 Holly C, Slyer JT. Interpreting and using meta-analysis in clinical practice. Orthop Nurs 2013; 32: 106–10. doi:10.1097/NOR.0b013e3182879c34 http://www.ncbi.nlm.nih.gov/pubmed/23518755
31 Oliveira AF, Santos IM, Torres A. Cognition in cancer: proposal of a web-based cognitive rehabilitation program for Portuguese working non-central nervous system (CNS) cancer survivors. J Psychosoc Oncol Res Pract 2021; 3: 72–3.
32 Klaver KM, Duijts SFA, Geusgens CAV, et al. Internet-Based cognitive rehabilitation for working cancer survivors (i-WORC): study protocol of a randomized controlled trial. Trials 2020; 21: 664. doi:10.1186/s13063-020-04570-1
33 Moher D, Shamseer L, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 2015; 4: 1–9. doi:10.1186/2046-4053-4-1 http://www.ncbi.nlm.nih.gov/pubmed/25554246
34 Shamseer L, Moher D, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015; 349: g7647. doi:10.1136/bmj.g7647
35 Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021; 372: n71. doi:10.1136/bmj.n71 http://www.ncbi.nlm.nih.gov/pubmed/33782057
36 Page MJ, Moher D, Bossuyt PM, et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ 2021; 372: n160. doi:10.1136/bmj.n160 http://www.ncbi.nlm.nih.gov/pubmed/33781993
37 Coro D, Hutchinson A, Dahlenburg S, et al. The relationship between diet and cognitive function in adult cancer survivors: a systematic review. J Cancer Surviv 2019; 13: 773–91. doi:10.1007/s11764-019-00796-4 http://www.ncbi.nlm.nih.gov/pubmed/31399855
38 Joly F, Giffard B, Rigal O, et al. Impact of cancer and its treatments on cognitive function: advances in research from the Paris international cognition and cancer Task force symposium and update since 2012. J Pain Symptom Manage 2015; 50: 830–41. doi:10.1016/j.jpainsymman.2015.06.019 http://www.ncbi.nlm.nih.gov/pubmed/26344551
39 Wefel JS, Vardy J, Ahles T, et al. International cognition and cancer Task force recommendations to harmonise studies of cognitive function in patients with cancer. Lancet Oncol 2011; 12: 703–8. doi:10.1016/S1470-2045(10)70294-1 http://www.ncbi.nlm.nih.gov/pubmed/21354373
40 Bender CM, Thelen BD. Cancer and cognitive changes: the complexity of the problem. Semin Oncol Nurs 2013; 29: 232–7. doi:10.1016/j.soncn.2013.08.003 http://www.ncbi.nlm.nih.gov/pubmed/24183154
41 Hess LM, Insel KC. Chemotherapy-Related change in cognitive function: a conceptual model. Oncol Nurs Forum 2007; 34: 981–94. doi:10.1188/07.ONF.981-994 http://www.ncbi.nlm.nih.gov/pubmed/17878127
42 Ferguson RJ, Bender CM, MB C, et al. Cognitive Dysfunction. In: Feuerstein M, Nekhlyudov L, eds. Handbook of cancer survivorship. Second edition. Springer International Publishing AG, 2018: 199–225.
43 Holland JC, Breitbart WS, Butow PN. Psycho-Oncology. Third edition. New York: Oxford University Press, Inc, 2015.
44 Viera AJ, Garrett JM. Understanding interobserver agreement: the kappa statistic. Fam Med 2005; 37: 360–3. http://www.ncbi.nlm.nih.gov/pubmed/15883903
45 Joanna Briggs Institute. Critical appraisal tools [Internet]. 2021 [cited 2021 Jul 21]. Available from. Available: https://jbi.global/critical-appraisal-tools
46 Melnyk BM, Fineout-Overholt E. Making the case for evidence-based practice and cultivating a spirit of inquiry. In: Melnyk BM, Fineout-Overholt E, eds. Evidence-Based practice in nursing and healthcare: a guide to best practice. Third Edit. Philadelphia, PA: Wolters Kluwer Health Copyright, 2015: p. 3–23.
47 IOM (Institute of Medicine). Finding what works in health care: standards for systematic reviews. finding what works in health care. Washington, DC The National Academies Press; 2011.
48 Popay J, Roberts H, Sowden A, et al. Guidance on the conduct of narrative synthesis in systematic reviews. ESRC Methods Programme 2006.
49 Balduzzi S, Rücker G, Schwarzer G. How to perform a meta-analysis with R: a practical tutorial. Evid Based Ment Health 2019; 22: 153–60. doi:10.1136/ebmental-2019-300117 http://www.ncbi.nlm.nih.gov/pubmed/31563865
50 Lipsey MW, Wilson D. Practical meta-analysis. 1st edition. Sage Publications, Inc, 2001.
51 Hafdahl AR. Improved Fisher Z estimators for univariate random-effects meta-analysis of correlations. Br J Math Stat Psychol 2009; 62: 233–61. doi:10.1348/000711008X281633 http://www.ncbi.nlm.nih.gov/pubmed/18257972
52 Peterson RA, Brown SP. On the use of beta coefficients in meta-analysis. J Appl Psychol 2005; 90: 175–81. doi:10.1037/0021-9010.90.1.175 http://www.ncbi.nlm.nih.gov/pubmed/15641898
53 Cohen J. Statistical power analysis for the behavioral sciences. 2nd Editio. Lawrence Erlbaum Associates, Publishers, 1988.
54 Higgins JPT, Thompson SG, Deeks JJ, et al. Measuring inconsistency in meta-analyses. BMJ 2003; 327: 557–60. doi:10.1136/bmj.327.7414.557 http://www.ncbi.nlm.nih.gov/pubmed/12958120
55 Y-S S, Yajima M. Using R to Run “JAGS” - Package “R2jags.” 2021.
56 Guyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008; 336: 924–6. doi:10.1136/bmj.39489.470347.AD http://www.ncbi.nlm.nih.gov/pubmed/18436948
© 2022 Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ. http://creativecommons.org/licenses/by-nc/4.0/ This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/ . Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Fernandes, Sofia 2 ; Reis, Juliana D 3 ; Torres, Ana 4 ; Santos, Isabel M 5 ; Diane Von Ah 6 1 Center for Health Technology and Services Research (CINTESIS@RISE), Department of Education and Psychology, University of Aveiro, Aveiro, Portugal
2 Department of Education and Psychology, University of Aveiro, Aveiro, Portugal
3 Department of Mathematics, University of Aveiro, Aveiro, Portugal
4 Center for Health Technology and Services Research (CINTESIS@RISE), Department of Education and Psychology, University of Aveiro, Aveiro, Portugal; Department of Psychology and Education, Faculty of Human and Social Sciences, University of Beira Interior, Covilhã, Portugal
5 William James Center for Research (WJCR), Department of Education and Psychology, University of Aveiro, Aveiro, Portugal
6 The Ohio State University, College of Nursing, Columbus, Ohio, USA