Cultural adaptation, translation and validation of the Spanish version Debriefing Experience Scale

Abstract

Clinical simulation as a teaching methodology allows the student to train and learn technical abilities and/or non-technical abilities. One of the key elements of this teaching methodology is the debriefing, which consists of a conversation between several people, in which the participants go over a real or simulated event in order to analyze their actions and reflect on the role that thought processes, psychomotor skills and emotional states can play in maintaining, or improving their performance in the future. The Debriefing Experience Scale allows the experience of students in debriefing to be measured. The objective of this study is to translate the Debriefing Experience Scale (DES) into Spanish and analyze its reliability and validity to measure the experience of nursing students during the debriefing. The study was developed in two phases: One: the adaption of the instrument to Spanish, two: a transversal study carried out in a sample of 290 nursing students. The psychometric properties were analyzed in terms of reliability and construct validity using confirmatory factorial analysis (CFA). Cronbach’s alpha was adequate for all the scales and for each one of the dimensions. The confirmatory factorial analysis showed that the 4-dimensional model is acceptable for both scales (experience and opinion). The Spanish version Debriefing Experience Scale questionnaire is useful, valid and reliable for use to measure the debriefing experience of university students in a simulation activity.

Introduction

In health science studies in order to complete the proposals generated by the Bologna process new methodologies have been developed based on active participation by the student, amongst them clinical simulation.

Clinical simulation as a teaching methodology allows the student to train and learn technical abilities and/or non-technical abilities such as: communication; leadership; teamwork; situational awareness; decision-making; resource management; safe practice; adverse event minimization and mitigation; and professionalism [1], through reflective learning, which facilitates critical thinking in a safe environment, without risk to patients or the participants.

One of the key elements of this teaching method is the debriefing. Existing research [2, 3] provides evidence that the debriefing is the most important component of the learning process of any experience based on simulation [4]. It has to be planned and directed by a facilitator (debriefer) who orients the discussion from reflection, focusing on the learning objectives and on the application of knowledge. Maestre and Rudolph (2015) define it as a conversation between various people, who go over a real or simulated event, in which the participants analyze their actions and reflect on the role that thought processes, psychomotor skills and emotional states can play in maintaining or improving their performance in the future [5].

There are many definitions of debriefing, they all agree that it is the sum of feedback plus reflection on an experience [6], carried out through means of analysis of the thought process, guided by action and decision making during the simulation (what was done, why it was done, what could have been done differently) in order to apply the results obtained to future situations [3, 5, 7–10]. When the instructions encourage a high level of commitment from the participants, they have better retention and undergo deeper learning, raising the probabilities that new or reinforced knowledge, abilities and attitudes will be transferred to clinical practice, or better health performance in general [11].

Although there are different styles of debriefing, they all share characteristic defining elements and a structure, which is generally divided into three phases: 1)1a. Emotions and reactions, 1b description and summary, 2) analysis and 3) closure and conclusions [12]. The first phase of reactions takes place immediately after finishing the simulation experience, when the participants meet with the facilitator. It is a phase of emotional discharge which allows the transition which makes the reflexive part possible. Phase 1b. consists of constructing the reality lived and shared together with each of the participants, according to the perceptions of each individual. The second phase is an analysis phase, when the cognitive and learning processes take place for each participant. The goal is to discuss the pre-established and emerging learning objectives, to examine the thought processes of the participants more deeply, diagnose the participants’ position regarding the objectives they are trying to reach and reflect on how to improve when putting them into practice in the future and in different contexts. Lastly, before finishing the debriefing, there is a third phase of closure and conclusions, which has the objective of presenting the results of the learning through a process of refection and verbalization of what was learned, which makes better integration possible.

In the literature there are numerous studies that emphasize the role of the debriefer in the debriefing, however, there is still little known about how the participants experience a debriefing session in order to provide an understanding of the expected learning process occurring while it takes place [13, 14]. Because of this in 2012 Shelly J. Reed developed a questionnaire to find out how students experience debriefing: the Debriefing Experience Scale (DES). This 20 item questionnaire has been translated into Norwegian and Portuguese and validated [15, 16]. Both studies concluded that it is a useful, valid and reliable questionnaire, used so participants can evaluate the simulation experience. They suggest that it should be validated in more nursing programs in different cultural contexts.

The objective of this study was to translate the Debriefing Experience Scale (DES) to Spanish, and analyse its reliability and validity for finding out about nursing students’ experiences during debriefing.

Methods

Design

The study was designed in two phases: the first implies the translation and adaptation of the questionnaire and the second consists of validation using a transversal design.

Debriefing Experience Scale

The Debriefing Experience Scale was developed by Reed in the United States with the aim of measuring the experience of students during debriefing [14]. The questionnaire is configured with 20 items, grouped in four dimensions. It can be completed in approximately 12 minutes. The dimensions correspond to: D1 learning and making connections; D2. analysing ideas and feelings; D3. the ability of the facilitator in directing the debriefing; and D4. appropriate guidance from the facilitator.

The same scale allows each item to evaluate the opinion of the students of their experience in the debriefing, and on the other hand the experience based on the importance it has for each student.

Each item is evaluated using a Likert scale with 5 possible replies, which are:

1) strongly disagree, 2) disagree, 3) undecided, 4) agree, and 5) strongly agree.

The sum of the scores for all the items for each dimension gives us an estimation of the experience of the students in the debriefing and the importance the experience has for them.

The DES questionnaire has demonstrated good and moderate validity in the nursing student population in the United States of America and a reliability of .93 (experience) and .91 (importance).

Cultural and linguistic adaptation of the Debriefing Experience Scale

The cultural adaption and translation of the questionnaire was carried out in several stages and in agreement with standardized criteria [17]. In the first stage, two translators translate the English to Spanish (t1 and t2). In the second stage both versions, t2 and t2 are merged. In this stage a research team made up of two clinical simulation teachers, accredited by the Boston Children’s Hospital, Simulator Program, Boston; two teachers expert in psychomotor skills and three nurses with experience of advanced clinical practice. The research team resolved any discrepancies, thereby obtaining the version t3. In the third stage, the t3 version was translated from Spanish to English by two native English translators (t4 and t5). The research team checked both translations and compared them with the original. All the researchers agreed that the items in the Spanish version coincided with the original English version. However, to obtain the best degree of semantic equivalence, the committee of experts decided to modify “Aprender y hacer conexiones” to “Aprender y relacionar conceptos” in dimension 1, so they also modified item 1 “Aprender y hacer conexiones” to “Aprender y relacionar conceptos”. In item 8 they modified “hacer conexiones de la teoría con situaciones de la vida real” to “me ayudó a relacionar la teoría con situaciones de la vida real”. In item 10 they modified “equipo sanitario” to “equipo” and in item 11 “psíquicamente cómodo” to “psicológicamente seguro”.

Afterwards, in the fourth stage, version t6 was tested on a small sample of students (n = 30), who concluded that it was easy to understand and needed little time to complete; approximately 12 minutes. Table 1 shows the semantic equivalents of the original and the version adapted to Spanish.

[Figure omitted. See PDF.]

Finally the questionnaire was administered to the undergraduate nursing students included in the sample to analyze the psychometric properties of the Spanish version of the Debriefing Experience Scale questionnaire.

Participants and setting

The study sample consisted of 290 undergraduate nursing students registered on the 2019–2020 academic course. A convenience sample was carried out. All students who gave their consent to participate in the study, and had also taken part in a clinical simulation during the course, were included. Only those students who were not present at the time of administering the questionnaire were excluded. The sample size was calculated based on internal consistency and construct validity. The recommendations of Streiner, Norman & Cairney, were followed to estimate the internal consistency; they consider that between 5–20 individuals should be included for each item making up the questionnaire [18]. In this study, it was agreed to include a minimum of 15 individuals per item. Additionally, for the construct validity, Kline (2015) established that the minimum number of participants necessary to realize a confirmatory factorial analysis (CFA) should be 250 students [19].

Variables and source of information

All items related with the Debriefing Experience Scale questionnaire were collected as variables. Other sociodemographic variables, such as: age, sex, academic year, average grade on their transcript for the previous course, teaching shift, whether they were currently working, if they worked in healthcare, type of contract and working shift were also collected.

Statistical analysis

The reliability of the questionnaire was measured using the Cronbach’s alpha coefficient and values considered acceptable were those over.70 [20, 21].

To analyze the construct validity a confirmatory factorial analysis (CFA) was carried out using maximum likelihood estimation. The goodness of fit of the model was evaluated from various indices: the normalized chi-square, defined as the ratio of the chi-square value to the number of degrees of freedom (χ2/df); the adjusted Goodness of Fit Index (AGFI); Goodness of Fit Index (GFI); Comparative Fit Index (CFI); Bentler Bonnet Non-Normed Fit Index (BBNNFI); Bentler Bonnet Normed Fit Index (BNNFI); Root Mean Standard Error of Approximation (RMSEA); Root Mean Square Residual (RMR) and the Standardized Root Mean Squared Residue (SRMR). To consider a good global fit the criteria adopted was the obtention of the folloiwng adjustment value: X²/df values between 2 and 6 [22]; AGFI, GFI, CFI, BBNNFI y BBNFI values greater than or equal to .90.

For the RMSEA, RMR and SRMR indices, values equal to or less than .05 were considered excellent, while those over .05 and equal to or lower than .08 were acceptable [23–25].

CFA models were estimated using structural equation modeling (EQS 6.4 for Windows, Multivariate Software, Inc., Encino, CA, USA).

A descriptive analysis was carried out using frequencies and percentages, measures of central tendency and dispersion. Data analyses were performed using SPSS for Windows 27 (SPSS Institute, Chicago, IL, USA).

Ethical considerations

The study was approved by the Clinical Investigation Ethics Committee of the San Joan de Déu Foundation, with the assigned code CEIC PIC-42-19. All the participants gave their written consent to voluntarily participate in the study, having been informed of the aim of the research. The permission of the author was also obtained for the translation and adaption of the instrument to Spanish.

Results

Demographic characteristics

A total of 290 nursing students took part, with an average age of 22.9 (SD 5.4), and 85.5% were women. The average grade on their transcript for the previous academic course was 8.0 (SD 0.5). 56.2% of the students were registered in the group doing classes in the mornings. Half of the students declared they were working at the time (50.3%); and of these 41.8% had a permanent contract and 37.6% of the students working, were doing so in the healthcare sector (Table 2).

[Figure omitted. See PDF.]

Reliability

The Cronbach’s alpha or coefficient of internal consistency for the total of the Debriefing Experience Scale for experience and for importance of simulation in the questionnaire was .926 and .933 respectively (Table 3).

[Figure omitted. See PDF.]

Construct validity

Confirmatory factorial analysis (CFA).

The parameters were estimated using the maximum likelihood estimation method. A 4-dimensional model was proposed, identical to the structure of the original version of the questionnaire, with the aim of checking if the model was adequate.

The chi-squared test was statistically significant, although the adjustment ratio was 2.1 (experience) and 2.8 (importance), however, between 2 and 6 is considered reasonably good. The rest of the indices analyzed showed the same tendency, so we can conclude that the model fits correctly (Table 4).

All saturations were over 0.50. The correlations between the factors of experience and importance of the debriefing were high (Figs 1 and 2 respectively).

[Figure omitted. See PDF.]

[Figure omitted. See PDF.]

[Figure omitted. See PDF.]

Discussion

The aim of the study was to adapt the Debriefing Experience Scale (DES) to Spanish and evaluate the psychometric properties in nursing students in Spain. The instrument was developed to evaluate the experience and the importance of debriefing in simulation. The results obtained in this study show that the Spanish version of the Debriefing Experience Scale have adequate psychometric properties in terms of internal consistency and construct validity.

It consists of 20 items grouped in four dimensions, which aim to evaluate the experience and importance of the simulation debriefing for the nursing students.

The reliability of the questionnaire was adequate, it obtained a Cronbach’s alfa over .70 for the whole questionnaire and for each of the dimensions of both subscales (experience and importance). The highest value was obtained for dimension D1 Learning and making connections. For the rest of the dimensions (D2. Analyzing thoughts and feelings, D3. Facilitator skill in conducting the debriefing and D4. Appropriate facilitator guidance) the alpha varied between .725 and .800. This instrument has been translated into different languages in different countries (Norwegian and Portuguese) and these studies reported similar values to those found in our study [15, 16].

The original scale, created by Reed was validated in a sample of 130 nursing students in the USA and obtained a Cronbach’s alpha for the total questionnaire, experience and importance of the debriefing of .930 and .910 respectively [Reed 2012]. The alpha values for each of the dimensions varied between .650 and .890, the highest being for dimension D1 Learning and making connections, as found in our study. The instrument translated into Portuguese [16] was validated in a sample of 103 nursing professionals and obtained similar results to our study and the original validation of the study (.940 for experience and .96 for importance of debriefing in simulation). However, the instrument translated to Norwegian [15], with a sample of 146 nursing students, obtained values of the total scale of .860 and .640 for experience and importance respectively.

The CFA of this study showed an adequate fit for the structure of 4 factors, consistent with the original version [14].

In our study a CFA was carried out using the maximum likelihood estimation method to determine whether the scores reproduced the structure of 4 dimensions, on which the original version is based. The results obtained for the indices used to carry out the factorial validity and the goodness of fit were acceptable assuming an adequate model fit.

The authors of the Norwegian translation and validation [15], decided to withdraw item 12 “Unsettled feelings from the simulation were resolved by debriefing” and item 13 “The facilitator allowed me enough time to verbalize my feelings before commenting”, obtaining a scale of 18 items instead of 20 and a Cronbach’s alpha for the total questionnaire of .91. Furthermore, using the exploratory factorial analysis (Keiser), pattern matrix and structure matrix, they proposed reducing the dimensions of the instrument to two dimensions “the experience of learning in debriefing” and “the facilitator’s ability in conducting debriefing”, however, as the first dimension explains the main part of the total variance, it could justify a scale of just one dimension. The Portuguese version of the questionnaire by Almeida et al. using exploratory factorial analysis (octagonal rotation) obtained unexpected results, with a model with three dimensions, despite this they decided to maintain the four dimensions of the original instrument and justify the unexpected fit with the sample taken, as they were not student nurses, but professionals [16].

Limitations

Our study has certain limitations. In the first place, we selected a convenience sample from only one university in Barcelona, so it is possible that our results cannot be generalized to all the nursing students. However, the sociodemographic and labor characteristics of the students in this study are similar to those in other universities in Spain.

A limitation that should be considered is that how the simulation is implemented could differ between institutions and countries, which could cause differences in the way the concepts used in the questionnaire are interpreted. However, to implement the simulation for this study the Standard VI: the Debriefing Process of the International Nursing Association for Clinical Simulation was used [26].

Conclusions

The Spanish version of the Debriefing Experience Scale (DES-sp) questionnaire is a valid and applicable instrument to find out about the debriefing experience of university students. The scale evaluated two aspects: experience of the debriefing and opinion of the debriefing in the simulation. It is a scale that requires very little time to be self-completed.

Supporting information

S1 Data.

https://doi.org/10.1371/journal.pone.0267956.s001

(XLSX)

Citation: Farrés-Tarafa M, Julian DB, Lorenzo-Seva U, Hurtado-Pardos B, Raurell-Torredà M, Casas I, et al. (2022) Cultural adaptation, translation and validation of the Spanish version Debriefing Experience Scale. PLoS ONE 17(5): e0267956. https://doi.org/10.1371/journal.pone.0267956

About the Authors:

Mariona Farrés-Tarafa

Roles: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Supervision, Visualization, Writing – original draft, Writing – review & editing

E-mail: [email protected]

Affiliations Campus Docent, Sant Joan de Déu—Fundació Privada, School of Nursing, University of Barcelona, Barcelona, Spain, Research Group GIES (Grupo de Investigación en Enfermería, Educación y Sociedad), Barcelona, Spain, Member Research Group GRISIMula (Grupo Emergente 2017 SGR 531, Grupo en Recerca Enfermera en Simulación), Barcelona, Spain

https://orcid.org/0000-0001-6151-3844

David Bande Julian

Roles: Investigation, Methodology, Writing – review & editing

Affiliation: Anesthesiologist, Servicio Anestesiología, Reanimación y Tratamiento del dolor, Parc de Salut Mar, Barcelona, Spain

Urbano Lorenzo-Seva

Roles: Formal analysis, Supervision, Writing – review & editing

Affiliation: Universitat Rovira I Virgili, Tarragona, Spain

Barbara Hurtado-Pardos

Roles: Data curation, Investigation, Resources, Writing – review & editing

Affiliations Campus Docent, Sant Joan de Déu—Fundació Privada, School of Nursing, University of Barcelona, Barcelona, Spain, Research Group GIES (Grupo de Investigación en Enfermería, Educación y Sociedad), Barcelona, Spain, Member Research Group GRIN (Grupo Consolidado de Recerca Infermeria, SRG:664), Barcelona, Spain

Marta Raurell-Torredà

Roles: Investigation, Methodology, Writing – review & editing

Affiliations Universidad de Barcelona, Barcelona, Spain, President Research Group GRISIMula (Grupo Emergente 2017 SGR 531, Grupo en Recerca Enfermera en Simulación), Barcelona, Spain

Irma Casas

Roles: Investigation, Methodology, Resources, Writing – review & editing

Affiliations Universitat Autònoma de Barcelona, Barcelona, Spain, Preventive Medicine Service, Hospital Germans Trias i Pujol, Barcelona, Spain, Research Group Innovation in Respiratory Infections and Tuberculosis Diagnosis (Group Consolidat 2017 SGR 494), Barcelona, Spain

Jaime Carballedo-Pulido

Roles: Data curation, Investigation, Resources, Writing – review & editing

Affiliations Campus Docent, Sant Joan de Déu—Fundació Privada, School of Nursing, University of Barcelona, Barcelona, Spain, Research Group GIES (Grupo de Investigación en Enfermería, Educación y Sociedad), Barcelona, Spain

Juan Roldán-Merino

Roles: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Supervision, Visualization, Writing – original draft, Writing – review & editing

Affiliations Campus Docent, Sant Joan de Déu—Fundació Privada, School of Nursing, University of Barcelona, Barcelona, Spain, Research Group GIES (Grupo de Investigación en Enfermería, Educación y Sociedad), Barcelona, Spain, Research Group GEIMAC (Consolidated Group 2017–1681: Group of Studies of Invarianza of the Instruments of Measurement and Analysis of Change in the Social and Health Areas), Barcelona, Spain

References

1. Lioce L, Lopreiato J, Downing D, Chang TP, Robertson JM, Anderson M, et al. Healthcare Simulation Dictionary. Rockville, MD: Agency for Healthcare Research and Quality [Internet]. Healthcare Simulation Dictionary. 2020. 75 p. Available from: https://www.ahrq.gov/sites/default/files/publications/files/sim-dictionary.pdf

2. Decker S, Fey M, Sideras S, Caballero S, Rocky Rockstraw L, Boese T, et al. Standards of Best Practice: Simulation Standard VI: The Debriefing Process. Clin Simul Nurs. 2013;9(6 SUPPL):e26–9.

3. Fanning RM, Gaba DM. The role of debriefing in simulation-based learning. Vol. 2, Simulation in Healthcare. 2007. p. 115–25.

4. Levett-Jones T, Lapkin S. A systematic review of the effectiveness of simulation debriefing in health professional education. Vol. 34, Nurse Education Today. 2014. p. e58–e63.

5. Maestre JM, Rudolph JW. Teorías y estilos de debriefing: el método con buen juicio como herramienta de evaluación formativa en salud. Rev Española Cardiol. 2015;68(4):282–5.

6. Waznonis AR. Methods and evaluations for simulation debriefing in nursing education. J Nurs Educ. 2014;53(8):459–65.

7. O’Donnell J, Rodgers D, Lee W, Edelson D, Haag J, Hamilton M, et al. Structured and supported debriefing. Dallas: Tex: American Heart Association; 2009.

8. Dreifuerst KT. Getting started with debriefing for meaningful learning. Clin Simul Nurs. 2015;11(5):268–75.

9. Lusk JM, Fater K. Postsimulation debriefing to maximize clinical judgment development. Vol. 38, Nurse Educator. 2013. p. 16–9. pmid:23222625

10. Morse CJ. Debriefing after simulated patient experiences. In: Human simulation for nursing and health professions [Internet]. 2012. p. 58–68. Available from: http://sfx.scholarsportal.info.cat1.lib.trentu.ca:8080/trent?url_ver=Z39.88-2004&rft_val_fmt=info:ofi/fmt:kev:mtx:book&genre=bookitem&sid=ProQ:PsycINFO&atitle=Debriefing+after+simulated+patient+experiences.&title=Human+simulation+for+nursing+and+health+pr

11. Simon R, Raemer D, Rudolph J. Debriefing Assessment for Simulation in Healthcare (DASH) © Score Sheet Rating. Cent Med Simulation, [Internet]. 2018;2–3. Available from: https://harvardmedsim.org/dash-rv-long-scoresheet-en-2018/.

12. Sawyer T, Eppich W, Brett-Fleegler M, Grant V, Cheng A. More Than One Way to Debrief: A Critical Review of Healthcare Simulation Debriefing Methods. Vol. 11, Simulation in Healthcare. 2016. p. 209–17. pmid:27254527

13. Neill MA, Wotton K. High-Fidelity Simulation Debriefing in Nursing Education: A Literature Review. Vol. 7, Clinical Simulation in Nursing. 2011. p. e161–8.

14. Reed SJ. Debriefing Experience Scale: Development of a Tool to Evaluate the Student Learning Experience in Debriefing. Clin Simul Nurs. 2012;8(6):e211–2017.

15. Tosterud R, Petzäll K, Wangensteen S, Hall-Lord ML. Cross-cultural validation and psychometric testing of the questionnaire: Debriefing experience scale. Clin Simul Nurs. 2015;11(1):27–34.

16. Almeida RG dos S, Mazzo A, Martins JCA, Coutinho VRD, Jorge BM, Mendes IAC. Validação para a língua portuguesa da Debriefing Experience Scale. Rev Bras Enferm. 2016;69(4):705–11. pmid:27508476

17. Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Vol. 25, Spine. 2000. p. 3186–91. pmid:11124735

18. Streiner DL, Norman G, Cairney J. Health Measurement Scales:A Practical Guide to their Development and Use, 5th Edition. In: Oxford University Press. 2015.

19. Kline R. Principles and Practice of Structural Equation Modeling Fourth Edition. Guilford publications. 2015. p. 534 p.

20. Cronbach LJ. Coeffiecient alpha and the internal structure of test. Psychometrika. 1951;16:297–334.

21. Nunnally JC, Bernstein IH. The theory of measurement error. In: Psychometric Theory. 1994. p. 209–47.

22. Hu L, Bentler P. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Model A Multidiscip J. 1999;6(1):1–55.

23. Byrne BM. Structural Equation Modeling With AMOS. Structural Equation Modeling With AMOS. 2016.

24. Brown TA. Confirmatory factor analysis for applied research. Second Edi. Confirmatory factor analysis for applied research. New York: The Guildford Press; 2015. 462 p.

25. Schermelleh-Engel K, Moosbrugger H, Müller H. Evaluating the Fit of Structural Equation Models: Tests of Significance and Descriptive Goodness-of-Fit Measures. Methods Psychol Res Online. 2003;8(2):23–74.

26. International Nursing Association for Clinical Simulation. INACSL Standards of Best Practice: SimulationSM Debriefing. Clin Simul Nurs [Internet]. Elsevier Inc.; 2016 Dec 1 [cited 2022 Feb 18];12:S21–5. Available from: https://click.endnote.com/viewer?doi=10.1016%2Fj.ecns.2016.09.008&token=WzI3NzkzODUsIjEwLjEwMTYvai5lY25zLjIwMTYuMDkuMDA4Il0.m29h0fdRGYi4rKXWcP4pn9z6sbw