Content area
Aim
To identify and confirm the three core elements for evaluating serious games: frameworks, methods and indicators, by integrating the application of serious games in nursing education.
Background
Serious games have great potential in nursing education. Despite extensive research on evaluating serious games, no widely accepted or standardized evaluation in nursing education.
Design
The scope review was conducted according to the Joanna Briggs Institute methodology.
Methods
Nine databases, including Web of Science, PubMed, CINAHL, Embase, Cochrane Library, CNKI, Wanfang, CBM and VIPC Database, were searched for relevant studies from inception to December 2024.
Results
28 studies were included in this paper. There are two main theoretical frameworks regarding evaluation: the TURF usability framework and the TAM usability framework. Evaluation methods were categorized based on timing into three main types: pre-intervention, mid-intervention and post-intervention evaluations. The evaluation indicators were grouped into three main types: knowledge objectives, competency objectives and emotional goals, encompassing 18 indicators. Among them, there were three evaluation indicators related to knowledge objectives, 10 evaluation indicators related to competence objectives and five evaluation indicators related to emotional objectives.
Conclusions
At present, the evaluation of serious games in nursing education requires refinement. A systematic and highly directive evaluation framework is needed. It is imperative to develop a thorough, holistic and dynamic theoretical framework for evaluation based on the educational targets, courses and game types, standardize evaluation tools and techniques and enrich the content of evaluation indicators.
1 Introduction
At present, nursing education faces many new challenges, such as nursing faculty shortages ( Lauzon Clabo et al., 2023), the shift in nursing education focus towards competency-based education ( American Association of Colleges of Nursing, 2021), nurses being required to adapt to an aging society and diverse patient populations ( National Academies of Sciences et al., 2021). The ways nurses were educated during the 20th century are no longer adequate for dealing with the realities of healthcare today ( Mannino, Cotter, 2016). Therefore, innovative reforms in nursing education are urgent. Fortunately, with the widespread adoption of educational strategies such as e-learning applications, game-based learning and mobile learning, serious games as a new form of educational learning have gradually gained attention in medical education ( Frehywot et al., 2013; Zohari et al., 2023). Several studies have pointed out that serious games shows positive significance in auxiliary teaching, such as knowledge, skills and emotions ( Bayram and Caliskan, 2019; Chee et al., 2019; Fijačko et al., 2024). Therefore, it is imperative to use serious games to promote the innovative development of nursing education.
The concept of "serious games" was first proposed by Clark C. Abt in 1970, who described it as "have an explicit and carefully thought-out educational purpose and are not primarily intended for entertainment ( Abt, 1987)." Although there is a lack of uniformity in the definitions of serious games, we use generally accepted definitions to describe it. As Bergeron describes, serious games are ‘an interactive computer application, with or without a significant hardware component, that has a challenging goal, is fun to play with, incorporates some concept of scoring and imparts in the user a skill, knowledge or attitude which can be applied in the real world’ ( Bergeron, 2006). Serious games are student-centered and have the characteristics of flexibility, interactivity, immersive experience and timely feedback. These can not only make up for the resource constraints of traditional teaching and the lack of interactivity of simulation teaching ( Jasemi et al., 2018) but also enable learners to actively learn to gain experience, practice and consolidate the necessary knowledge and skills in a risk-free virtual environment ( Al-Mugheed et al., 2022; Habes et al., 2020; Li et al., 2024; McConnell et al., 2024). In addition, the involvement of serious games in education is a perfect fit for millennial medical students who are proficient in technology and the use of digital devices ( Hernandez-de-Menendez et al., 2020; Sundareswaran et al., 2024). The advantages of these serious games are helpful to compensate for the abovementioned educational dilemma and to meet the needs of the times.
In recent years, there has been a gradual increase in the use of serious games and related research, with a diversity of research topics. This result is undoubtedly positive. The primary objectives of serious games research are to assess playability, effectiveness and teaching effects in a specific topic or environment in nursing education. However, Caserman et al. found that existing research on the evaluation of serious games often lacks a combination of both the game and serious aspects, which does not fully consider the dual mission of serious games ( Caserman et al., 2020). In other words, without an appropriate comprehensive evaluation that applies both game usability (the entertainment part) and application effectiveness (the serious part), it is not conducive to the high-quality development of serious games and the integration of serious games with nursing education will also become challenging. While one study ( Graafland et al., 2014) has offered suggestions for evaluating serious games, these recommendations are limited in scope and do not provide a specific evaluation. In light of the current research limitations, particularly the absence of a comprehensive literature review for evaluating serious games in nursing education, the practical evaluation of serious games in nursing education has become a crucial issue for nursing educators and researchers.
Considering the impact of serious games on nursing education, it is important to understand the breadth and depth of the literature on the topic. A scoping review is deemed suitable for this research question and objective, as it can describe research progress and identify research gaps ( Pham et al., 2014). Therefore, this scoping review follows the Joanna Briggs Institute (JBI) scoping review guidelines to examine the theoretical frameworks, methods and indicators related to serious games evaluation in nursing education, providing references for building comprehensive evaluation in this field and clarifying future research and practice directions.
2 Methods
This scoping review followed the JBI guideline ( Peters et al., 2020) and the PRISMA-ScR checklist, ensuring the rigor, transparency and consistency of the research ( Tricco et al., 2018). The review followed five steps: (a) defining research objectives and questions; (b) defining inclusion criteria; (c) search strategy design and search; (d) evidence screening and selection; (e) data extraction and analysis. The research protocol has been registered in the Open Science Framework (OSF) Registry, Registration No. osf.io/jk4d3.
2.1 Review questions
1. The theoretical framework for evaluating serious games in nursing education.
2. What are the evaluation methods of serious games in the field of nursing education?
3. What are the evaluation indicators of serious games in the field of nursing education?
2.2 Eligibility criteria
The inclusion criteria used the PCO (population, concept, outcomes) framework to ensure consistency in the selected literature and review questions ( Arksey and O'Malley, 2005). The inclusion criteria: 1) population: nursing professionals at all stages, including nursing students as well as clinical nurses; 2) concept: measures related to the teaching and evaluation of serious games for nursing students or nurses; 3) outcomes: serious games change learners' knowledge, competence and emotional outcomes in the field of nursing education and the effectiveness of serious games were evaluated based on these outcomes; 4) original research, including quantitative, qualitative and mixed studies; 5) language limitation is English or Chinese; and 6) available full texts.
The exclusion criteria are: 1) literature populations did not include nursing professionals; 2) literature topic was unrelated to serious game teaching and evaluation for nursing students or nurses; 3) literature with incomplete descriptions of outcomes of serious game applications and poorly described data; 4) conferences, research proposals, commentary, policies and guideliness, which may incomplete detailed descriptions of serious game teaching and evaluation processes or have limited generalizability of results, may limit the accuracy of the search results; and 5) non-English and Chinese literature.
2.3 Search strategy
Initial searches of CINAHL and Medline were performed, and terms and keywords were identified by reading the titles and abstracts of the literature to ensure the completeness of the search strategy. The search strategy was developed collaboratively by the research team and adjusted according to different databases. Due to the lack of uniformity in the definition of serious games, the keywords in the relevant literature show a diversity of terms. Therefore, keywords such as "educational game, virtual game, digital games, gamification, etc." were added when searching for serious games to ensure more relevant data were obtained. Although some of the keywords differed from the definition of serious games, studies that did not apply serious games were read and analyzed to exclude them to ensure the relevance of the search results about serious games.
Nine databases were searched, including Web of Science, PubMed, CINAHL, Embase, Cochrane Library, China National Knowledge Infrastructure (CNKI), Wanfang, China Biology Medicine disc (CBM) and China Science and Technology Journal Database (VIPC Database) for relevant studies. The time limit was from the establishment of the database to December 2024. In addition, references to studies were included retrospectively to supplement access to relevant studies. Detailed search strategy for each database (see Supplementary material).
2.4 Selection of sources of evidence
After obtaining the search results, one reviewer (WPY) imported the retrieved study titles into Endnote X9 software to remove duplicates. Initial screening was performed by two independent reviewers (WPY, LLL) trained in evidence-based care, who read the titles and abstracts to exclude non-compliant studies according to the inclusion and exclusion criteria. Then, the two reviewers (WPY, LLL) read the full text for rescreening. Details of the screening can be found in
2.5 Data extraction and analysis
Information was extracted from the included studies to report study characteristics and relevant evaluation strategies. Data were extracted independently by two reviewers (WPY, LLL) using an extraction tool. In case of disagreement, a discussion was held with a third reviewer (HQ). The extraction tool was modified from the JBI template data extraction tool. The information extracted by the tool was divided into two categories. The first was study characteristics, including authors, year, country, study population, sample size, study design, training content, type of serious game, duration of intervention, evaluation method and findings. The second was the re-extraction and generalization of information already available to provide a basis for reviewing the evaluation indicators, including evaluation indicators, outcomes, counts and sources of evidence. No quality and risk of bias assessment of the included studies were conducted in this research, as this was not required for the scoping review.
3 Results
After a comprehensive search of all databases and references included in the study, a total of 2945 records were retrieved. After excluding 692 duplicate studies, 1937 studies that did not match the subject content were excluded after reading the titles and abstracts. Additionally, 288 studies were excluded that could not access the full text or did not match the subject matter after reading the full text. Finally, 28 studies were included, including 5 in Chinese and 23 in English. The specific process of screening can be seen in Fig. 1.
3.1 Characteristics of the included evidence sources
Among the 28 studies included, China ranks first with nine articles (32.14 %), followed by South Korea, Spain, the United Kingdom, Brazil and Canada with two articles each (7.14 % each) and Nepal, France, Italy, the United States, Netherlands, Norway, Turkey, Iran and Singapore with one article each (3.57 % each). Studies were published in 2013–2024, with 46 % (n = 13) of them published in 2022–2024. In terms of population characteristics, including first-year nursing students (n = 1), mid-to-upper year nursing students (n = 10), nursing intern (n = 5) and nurses (n = 2). Additionally, 10 studies did not specify the nursing students' year level in detail. Randomized controlled trials (n = 12, 42.86 %) were the most common in study design, followed by cohort studies and quasi-experimental studies with six articles each (21.43 % each) and mixed methods studies (n = 4, 14.28 %). Regarding the serious games included studies, 22 games were digital (78.57 %), two were physical (in-person) (7.14 %) and four were a mix of physical and digital (14.28 %). More specifically, according to the development of serious games, it can be categorized into the following eight types: virtual reality game, web-based game, simulation game, game based on platform, hybrid game, escape room game, level game and card game. The basic characteristics of the included studies are shown in
3.2 Theoretical frameworks for evaluating serious games in nursing education
Only two studies ( Johnsen et al., 2016; Verkuyl et al., 2018) introduced theoretical frameworks for serious games evaluation among the 28 included studies; they were 1) TURF Usability Framework ( Johnsen et al., 2016): the TURF model was proposed by Zhang and Walji and its components were “Task, User, Representation and Function”. As a comprehensive framework, TURF includes both objective measures of usefulness and usability, as well as subjective measures of satisfaction, which helps to define, assess and measure usability objectively and to explain and predict differences in usability effects. Researchers using this framework can demonstrate that a game is usable if they find that the system is easy to learn, efficient to use and can correct and prevent user errors ( Zhang and Walji, 2011). Based on this framework, Johnsen et al. modified the questions in the PostStudy System Usability Questionnaire (PSSUQ). They developed follow-up interview guidelines, aiming to assess game usability and user satisfaction and provide direction for subsequent game improvements. 2) TAM Usability Framework ( Verkuyl et al., 2018): the Technology Acceptance Model (TAM) was originally proposed by Davis and developed by Davis and Warshaw ( Davis, 1989; Davis et al., 1992). The framework emphasizes the importance of perceived ease of use and perceived usefulness. Perceived ease of use is the degree to which users find a particular technology easy to use. Perceived usefulness is the degree to which users believe that using the system will enhance clinical decision-making ( Verkuyl et al., 2018). Verkuyl et al. set up a two-stage usability evaluation based on this framework, consisting of heuristic expert usability evaluation and user usability evaluation, to evaluate the ease of use and usability of the game and verify that the virtual game can bring a vivid learning experience to learners.
3.3 Evaluation methods for evaluating serious games in nursing education
According to the different timing of evaluations, evaluation methods can be categorized into three groups. 1) Pre-intervention evaluation ( Adhikari et al., 2021; Baek and Lee, 2024; Buttussi et al., 2013; Chen et al., 2015; Craig et al., 2023; De Beer et al., 2023; Johnsen et al., 2016; Kulakaç and Çilingir, 2024; Ma et al., 2021; Mitchell et al., 2021; Rodríguez-Ferrer et al., 2022; Sun et al., 2022; Tan et al., 2017; Wong et al., 2022; Yang and Oh, 2022; Yang et al., 2021, 2023; Zhou et al., 2023) (n = 18). The content of the pre-intervention evaluation is mainly basic demographic information, mastery of basic knowledge, types of learning styles and game usability, etc. The purpose was to exclude the interference of heterogeneity between groups, to explore the relationship between learning styles and learning effects, to compare and evaluate the application effects of serious games and to evaluate the feasibility of the game system. 2) Mid-intervention evaluation ( De Beer et al., 2023) (n = 1). The content of the mid-intervention evaluation is the learners' subjective feelings, exploring the process of competence acquisition through interviews and evaluating the effectiveness of serious games in competence acquisition. 3) Post-intervention evaluation (n = 28). Post-intervention evaluations can be categorized into immediate evaluations ( Adhikari et al., 2021; Blanié et al., 2020; Boada et al., 2015; Chen et al., 2015; Craig et al., 2023; Cunha et al., 2023; De Beer et al., 2023; Hara et al., 2021; Johnsen et al., 2016; Khan and Kapralos, 2019; Liu et al., 2022; Luo et al., 2020; Ma et al., 2021; Mitchell et al., 2021; Nasirzade et al., 2024; Sun et al., 2022; Tan et al., 2017; Verkuyl et al., 2018; Wong et al., 2022; Yang and Oh, 2022; Yang et al., 2021; Zhou et al., 2023) and follow-up evaluations ( Baek and Lee, 2024; Buttussi et al., 2013; Huang et al., 2021; Kulakaç and Çilingir, 2024; Rodríguez-Ferrer et al., 2022; Yang et al., 2023) according to the time interval. The former evaluation content includes the degree of knowledge, competence and emotion acquisition to ensure the authenticity of the evaluation results and the effectiveness of serious games applications. The latter evaluation content mainly focuses on the retention of knowledge and transfer effects, exploring the effectiveness of serious games in knowledge reinforcement. In general, the evaluation measures commonly used at different times above are self-developed questionnaires based on the Likert 5-point scale, theoretical and skill assessments, scale assessments and qualitative interviews. In addition, the evaluation content suggests that the application of serious games facilitates learners' integration of knowledge, shortens the distance between knowledge and practice, develops clinical core competencies, improves professionalism, enhances nursing quality and promotes patient safety.
3.4 Evaluation indicators for evaluating serious games in nursing education
Based on the outcome effects of serious games applications, the researchers categorized the indicators into three main categories: knowledge objectives, competence objectives and emotional objectives. The summary includes the indicator names, frequency of occurrence and evaluation tools (
4 Discussion
4.1 The application status of serious games in nursing education
In this study, the basics of the application of serious games in nursing education were identified. As serious games play an important role in nursing education ( Thangavelu et al., 2022), the whole world has been actively exploring it, including Asia, Europe, South America and North America especially in the last four years. The contribution made by Asia was particularly notable. This may be related to the emergence of COVID-19, which promoted the development and application of serious games ( Busca et al., 2024). Moreover, this impact was especially evident in Asia, which experienced repeated lockdowns ( Aristovnik et al., 2023). The 28 selected studies had a variety of study designs, but 16 studies had a control group, which made it easy to identify the strengths and weaknesses of serious games. The primary educational target is nursing students in the mid-to-upper year, as this stage is the best phase for combining theory and practice. The poor application of serious games in first-year nursing students may be related to the knowledge burden caused by using it too early ( Ignacio and Chen, 2020). Only two studies ( Buttussi et al., 2013; Khan and Kapralos, 2019) involved nurses, indicating a significant potential for evaluating the use of serious games in retraining courses.
The training content ranges from basic courses, such as dementia knowledge and obstetrics-related knowledge, to advanced nursing experience courses, quality courses and retraining courses, the application of serious games is exciting, especially in advanced life support. Considering the different educational objectives, the requirements for the visual and engaging methods of the game are also different. Therefore, the basic courses are mostly related to low-fidelity or offline games, such as simulation game, hybrid game, level game and card game; Advanced nursing experience courses, quality courses and retraining courses are more related to immersive games, such as virtual reality game and escape room game. As in advanced life support ( Buttussi et al., 2013), the use of virtual reality technology allows nurses to enhance their skills in emergency immersion environment. The game type of game based on platform and web-based game can be applied to any of the above courses. In addition, due to the complexity of the expected teaching effects or limitations in resources, the duration of the intervention shows unlimited possibilities. In the future, it is suggested that the focus could be on evaluating the corresponding game characteristics for different courses and game types to ensure a balance between seriousness and entertainment. For example, low-fidelity or offline games could be evaluated for fun and immersive games could be evaluated for realism of the game system.
4.2 Theoretical frameworks for evaluating serious games in the field of nursing education still needs to be optimized
The results of this study showed that there are few evaluation theoretical frameworks mentioned in the included studies, only the TURF usability framework and the TAM usability framework, which shows that the theoretical frameworks for evaluating strategy are still in the developmental stage. Although both TURF and TAM theoretical frameworks were used to evaluate the usability of game systems, they are still different. The former focused on user needs and task completion and although it can comprehensively evaluate usability in terms of tasks, users, representations and functions, it lacked depth in each aspect. The latter focused on user acceptance and experience of using the game. The TAM framework was not as comprehensive as the TURF framework. However, the depth of the framework in the assessment of perceived ease of use and perceived usefulness can make up for the lack of depth assessment of the TURF framework. Thus, both usability frameworks have their advantages and limitations. It is suggested that different frameworks can be selected according to the focus of evaluation when evaluating game systems. In addition, the above two theoretical frameworks were only for evaluating the usability of the system and did not guide evaluating the effectiveness of the application of serious games. Therefore, it is suggested that researchers may refer to Hauenstein's classification theory of educational goals ( Hauenstein, 1998) and Kirkpatric's ( Ahn et al., 2003; Mosley et al., 2012) evaluation model using four outcome levels: satisfaction (level 1), knowledge and skill acquisition (level 2), retention of knowledge and skills over time (level 2), behavioral change (level 3) and patient outcome (level 4). It is clear that serious games should at least address the issue of acquiring knowledge and skills (level 2) ( Drummond et al., 2017), optimizing the theoretical framework for evaluating and guiding the design of evaluation indicators. It is suggested to integrate different theoretical frameworks to improve the multi-dimensional evaluation design, including educational, gameplay, technical, artistic and user experience aspects. This will provide comprehensive theoretical and practical guidance on evaluation for serious games.
4.3 Evaluation methods of serious games in the field of nursing education need to be further improved and standardized
The results of this study showed that most of the researchers conducted pre-intervention and post-intervention evaluations. In contrast, only a few studies used mid-intervention evaluations (n = 1) and post-intervention follow-up evaluations (n = 6). The frequency of post-intervention follow-up evaluations was diversified, with intervals of 2 weeks, 4 weeks, 3 months, 6 months and 10 months. Learning evaluation is not a single quantitative evaluation at a specific point in time but a dynamic evaluation that considers the learner's developmental and developmental potential ( Xu et al., 2023). The lack of long-term evaluation during and after the intervention is not conducive to evaluating learning effect continuity and stability and there is a lack of comprehensive and objective evidence for the effectiveness of serious game application. Therefore, it is recommended to take the longitudinal evaluation method as the primary method, supplemented by the horizontal evaluation, combined with the Ebbinghaus forgetting curve to standardize the frequency of tracking evaluation to explore further the long-term effect of the application of the serious games in the future; in the intervention throughout the multiple testing of the learners, to collect stage information, to clarify the evaluation criteria at different stages, to prevent the occurrence of the expertise reversal effect and to build a more in-depth and dynamic evaluation for serious games in the field of nursing education. Second, the results of this study showed that most studies used self-developed questionnaires and a combination of theory and skill assessment to evaluate the effectiveness of applying serious games and only five studies used a single evaluation measure, showing the characteristics of mixed evaluation measures. However, the reliability and validity of the self-developed questionnaire still need to be considered, and it is also difficult to exclude the Hawthorne effect, which reduces the generalizability of the results. Therefore, it is recommended to construct and use scientific evaluation tools in the future, accurately grasp the timing of evaluation, adhere to the principle of evaluation throughout the whole process and adopt diversified evaluation measures for scientific evaluation to build a more objective evaluation for serious games.
4.4 The evaluation indicators of serious games in the field of nursing education need to be further integrated and expanded
In this study, a total of 18 evaluation indicators were included, among which there are three indicators related to knowledge objectives, 10 indicators related to competence objectives and five indicators related to emotional objectives. First, this study found that the evaluation indicators of knowledge transfer were mainly measured by subjective self-assessment. Although one study ( Mitchell et al., 2021) has proved that knowledge acquisition can change behavioral effects, its role is not significant and the relationship between the effect of knowledge transfer on behavior and serious games still needs to be further explored. In the future, evaluation indicators such as patient outcomes can be added to quantify the effect of knowledge transfer. Second, this study found that the fragmented content of evaluation indicators related to competency objectives and the non-uniformity of evaluation tools may reduce the comparability of evaluation results and make the quality of serious games uneven. Therefore, it is recommended to set and integrate the competency teaching objectives with reference to the guidelines of nursing core competencies and nursing skills, refer to them with the help of game performance and combine them with comprehensive competency scales for evaluation. Thirdly, emotional-related evaluation indicators are critical, as emotional objectives serve as a difficulty in the education of professional quality courses and the focus of clinical practice. However, this study found that most of the current studies on emotional objectives focus on subjective evaluations of self or system satisfaction, self-confidence and anxiety and there are few studies on emotional education on empathy, which may be related to higher requirements for game scenarios and fidelity. In addition, only one included a study ( Sun et al., 2022) evaluating the satisfaction of educational specialists. Therefore, it is suggested that nursing emotional education in a cross-cultural context can be added or the fidelity of serious games can be increased with the help of virtual reality technology in the future and the relevant evaluation indicators of emotional objectives can be further broadened. Supplementing the satisfaction evaluation of technical professionals and educational experts to provide direction for improving the quality of serious games. In summary, although the evaluation indicators of serious games in nursing education have diverse characteristics, they are still in the development stage. We need to generalize and integrate the content of evaluation indicators and continue to expand the depth and breadth of evaluation indicators in the future to effectively monitor the application effects of serious games, which will benefit the optimization of practical teaching methods.
4.5 Limitations
First, this study only included Chinese and English literature, which may have left out valuable studies in other languages. Second, we exclude some types of evidence in the selection of evidence, limiting the information that can be extracted. In addition, this review has systematically organized the application of serious games in nursing education from three aspects: theoretical frameworks, evaluation methods and evaluation indicators. However, these aspects are only in the preliminary construction stage and have not yet formed a mature system. There is significant room for development in this field, which requires further exploration and research.
5 Conclusion
The results of this study show that serious games primarily target mid-to-upper year nursing students and are applicable to various courses. Among them, low-fidelity or offline games are mostly related to basic courses, while immersive games are mostly related to advanced nursing experience courses, quality courses and retraining courses. In this context, the theoretical frameworks of the evaluation are all usability frameworks, the evaluation methods are mixed but lack comprehensiveness and scientificity and the evaluation indicators are diverse but still need to be further integrated and expanded. In the future, it is necessary to construct a comprehensive, integrated and dynamic evaluation theoretical framework based on the educational targets, courses and game types, standardize evaluation tools and evaluation methods, enrich the content of evaluation indicators and focus on the monitoring and evaluation of game system aspects. Researchers should conduct more high-quality randomized controlled trials, broaden the evaluation subjects with different educational backgrounds and years of experience and explore optimal auxiliary practical teaching programs to cultivate applied nursing professionals.
Funding
This study was supported by the
CRediT authorship contribution statement
Tan yifen: Writing – review & editing, Project administration, Methodology, Formal analysis, Conceptualization. Zhixu Kong: Methodology, Investigation, Formal analysis, Data curation. Yuhe Wu: Methodology, Investigation, Formal analysis, Data curation. Lili Liu: Writing – review & editing, Software, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. Qing Huang: Methodology, Investigation, Formal analysis, Data curation, Conceptualization. Peiyin Wang: Writing – review & editing, Writing – original draft, Software, Methodology, Investigation, Formal analysis, Data curation, Conceptualization.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
None.
Appendix A Supporting information
Supplementary data associated with this article can be found in the online version at
Appendix A Supplementary material
Supplementary material
Table 1
| Author, Year, Country | Population & sample size (T/C) | Designs | Training content | Type of serious game | Duration of intervention | Evaluation methods | Findings | |
| serious games system | Participants’ performance | |||||||
|
Adhikari et al., (2021)
Nepal |
Junior nursing student
19 |
Mixed methods research | Pneumonia combined with sepsis | Virtual reality game:
Play a game related to sepsis cases; Digital |
Not mentioned | Semi-structured interview | Nursing anxiety and clinical decision-making confidence scale, semi-structured interview | Self-confidence(+)
Anxiety(-) Decision-making ability(+) Game usability(+) |
|
Baek and Lee, (2024)
South Korea |
Sophomore and junior nursing student
22/22 |
Quasi-experimental | Advanced CPR | Web-based game:
Apply serious games on the basis of video lectures, and give debriefing and personal feedback at the end; Physical (in-person) and digital |
3 h | Knowledge questionnaire, advanced CPR performance competency, problem solving competency scale, learning transfer expectations scale | Theoretical test score(+)
Skills performance(+) Self-confidence(+) Problem-solving ability(+) Knowledge transfer(+) | |
|
Blanié et al., (2020)
France |
Sophomore nursing student
73/73 |
RCT | Interprofessional communication and ability to detect clinical deterioration in patients | Simulation game:
Includes 2 cases, requiring participants to promptly report the patient's condition and debrief; Physical (in-person) and digital |
2 h | Script Conformance Tests (SCTs) | Self-assessment of perceived changes in the clinical reasoning process | Reasoning ability(-)
Satisfaction(+) Motivation(+) Knowledge transfer(+) |
|
Boada et al., (2015)
Spain |
Senior nursing student
67/42 |
RCT | CPR | Virtual reality game:
Using 3D virtual technology, perform CPR operations with the keyboard and learn through the learning management system; Digital |
Not mentioned | Game operation score, survey questionnaire | Skills performance(+) | |
|
Buttussi et al., (2013)
Italy |
Nurse
40 |
Cohort study | Advanced life support(ALS) retraining | Virtual reality game:
Using 3D virtual technology, learning ALS knowledge by simulating the rescue process; Digital |
3 h 15 min | ALS knowledge test, questionnaire survey | Motivation(+)
Theoretical test score(+) Self-confidence(+) Knowledge retention(+) | |
|
Chen et al., (2015)
USA |
Sophomore nursing student
58 |
Quasi-experimental | Empathy and attitudes towards the elderly and understanding of the healthcare system | Character simulation game:
Students act as seniors to perform specific tasks in the healthcare system; Physical (in-person) |
3 h | Empathy Scale, aging simulation experience survey | Empathy(+)
Theoretical test score(+) | |
|
Craig et al., (2023)
UK |
First-year nursing students
560 |
Quasi-experimental | Dementia knowledge | Web-based game(HTML5):
Answer the questions correctly in the question bank to complete the task and get success; Digital |
7 weeks | Alzheimer's disease knowledge scale | Theoretical test score(+) | |
|
Cunha et al., (2023)
Brazil |
Junior nursing student
73 |
Cohort study | Knowledge of myocardial infarction | Escape room game based on platform:
The game conducted on the platform ZOOM® focused on acute myocardial infarction, with a total of 8 tasks; Digital |
Not mentioned | Education practice questionnaire, perception Scale | Satisfaction(+)
Self-confidence(+) | |
|
De Beer et al., (2023)
Netherlands |
Sophomore nursing student
110 |
Mixed methods research | Collaborative problem solving skills (CPS) | Hybrid game "Carion":
Game is a mixture of online learning activities and assignments. Students complete real or fictional tasks to facilitate the transfer of learning experiences; Digital |
840 h | Self-assessment tools for CPS skills, focus group interviews, semi-structured interviews | Teamwork ability(+)
Problem-solving ability(+) | |
|
Hara et al., (2021)
Brazil |
Nursing students and nursing educators
43 |
Cohort study | Communication ability | Virtual reality game:
Using 3D virtual technology and VR glasses, immerse themselves to learn communication skills in realistic patient rooms; Digital |
2 weeks | HEDEG | Heuristic Evaluation of Digital Educational Games (HEDEG) | Communication ability(+)
Game usability(+) |
|
Huang et al., (2021)
China |
Undergraduate nursing student
50/50 |
RCT | Cardiopulmonary resuscitation (CPR) | Virtual reality game:
Using 3D virtual technology, apply games based on traditional theoretical learning to learn about CPR knowledge; Digital |
12 h | Theoretical knowledge test, knowledge retention assessment questionnaire | Theoretical test score(+)
Knowledge retention(-) | |
|
Johnsen et al., (2016)
Norway |
Undergraduate nursing student
6 |
Mixed methods research | Chronic obstructive pulmonary disease(COPD) nursing, clinical reasoning and decision-making skills | Web-based game(HTML5):
Complete specific tasks in the context of a COPD patient's visit; Digital |
0.5 h | Post-learning system usability questionnaire | Semi-structured interviews, cognitive walkthrough evaluation | Reasoning ability(+)
Satisfaction(+) Game usability(+) Decision-making ability(+) |
|
Khan and Kapralos, (2019)
Canada |
Doctors and nurses
14 |
Cohort study | Cultural education | Web-based and low-fidelity game:
Students select questions from the list of potential replies and receive corresponding feedback from the patient, including answers, facial expressions and body language; Digital |
Not mentioned | Questionnaire for user interaction satisfaction (QUIS) | Game engagement questionnaire (GEQ) | Satisfaction(+) |
|
Kulakaç and Çilingir, (2024)
Turkey |
Sophomore nursing student
45/47 |
RCT | Stoma care | Web-based game:
Play a game to reinforce knowledge of stoma care after receiving traditional education; Digital |
2 weeks | Computer system usability questionnaire | Learning style scale, digital game motivation scale (DGMS), stoma care and colostomy irrigation knowledge test and skills assessment Form | Theoretical test score(+)
Knowledge retention(+) Skills performance(+) |
|
Liu et al., (2022)
China |
Nursing intern
60/60 |
RCT | Neurology | Hybrid game:
After 5 weeks of traditional education, the game types include offline knowledge competition, offline role-playing and online virtual reality game; Physical (in-person) and digital |
5 months | Objective structured clinical examination, registered nurse core competency scale, teaching satisfaction | Critical thinking(+)
Communication ability(+) Practical ability(+) Skills performance(+) Satisfaction(+) | |
|
Luo et al., (2020)
China |
Nursing students
113/115 |
RCT | Surgical Nursing | Virtual reality game:
Using 3D virtual technology. Playing game to enhance learning after traditional education; Digital |
68 h | Theoretical examination, operational assessment, independent learning evaluation scale, learning satisfaction questionnaire | Theoretical test score(+)
Self-learning ability(+) Skills performance(+) Satisfaction(+) | |
|
Ma et al., (2021)
China |
Sophomore nursing student
51/53 |
RCT | Disaster care capacity | Simulation game:
Comprehensive nursing tasks need to be performed in disaster environments (including disaster preparedness, disaster response, and post-disaster relief); Digital |
1.5 h 20 min | Disaster relief capability self-assessment questionnaire | Skills performance(+)
Empathy(+) | |
|
Mitchell et al., (2021)
UK |
Undergraduate nursing student
356 |
Cohort study | Influenza | Web-based game(HTML5):
Learn about the flu by answering questions correctly to achieve game success; Digital |
6 months | Questionnaire on perceptions and attitudes towards influenza, questionnaire on vaccination status, questionnaire on influenza knowledge | Motivation(+)
Self-confidence(+) | |
|
Nasirzade et al., (2024)
Iran |
Undergraduate nursing student
21/21 |
RCT | Burn assessment | Simulation game:
Answer questions within a limited time based on short videos or images, with a total of 8 stages; Digital |
2 weeks | Knowledge questionnaire, objective structured clinical examination | Theoretical test score(+)
Skills performance(+) | |
|
Rodríguez-Ferrer et al., (2022)
Spain |
Undergraduate nursing student
138/115 |
Quasi-experimental | Stigmatization of psychopaths | Escape room game:
Students must solve a series of puzzles, unlock locks, and find hidden clues to escape at any given time; Digital |
1 h | Collaborative learning questionnaire, public stigma questionnaire for mental disorders, final exam scores | Theoretical test score(+)
Teamwork ability(+) Knowledge retention(-) Empathy(+) | |
|
Sun et al., (2022)
China |
Nursing intern
276/293 |
RCT | Clinical nursing skills | Level game:
Utilize cocos2D game engine to set up level games (e.g. Train Puzzle, Snake, whack-a-mole, etc.) to enhance students' learning interest; Digital |
2 months | Software satisfaction evaluation | Clinical skills operation assessment | Skills performance(+)
Satisfaction(+) |
|
Tan et al., (2017)
Singapore |
Sophomore nursing student
57/46 |
RCT | Blood transfusion safety management | Game based on platform 3D Hive:
The game includes a scoring system, time challenges, and interactions with multiple characters; Digital |
0.5 h | Knowledge questionnaire, self-confidence scale, perception scale, simulated performance scale | Self-confidence(+)
Theoretical test score(+) Skills performance(-) | |
|
Verkuyl et al., (2018)
Canada |
Undergraduate nursing student
14 |
Mixed methods research | Mental health assessment | Virtual reality game:
The student needs to assess the depressed woman and make critical decisions related to communication and safety; Digital |
1 h | Heuristic expert usability testing, user usability testing | Mental health assessment survey, semi-structured interviews | Knowledge transfer(+)
Motivation(+) |
|
Wong et al., (2022)
China |
Undergraduate medical students and nursing students
62 |
Cohort study | Emergency, teamwork ability | Web-based game:
Participants need to collaborate with the team and divide tasks to complete missions related to saving emergency patients. Debriefing after the game ends; Physical (in-person) and digital |
2 h | Game characteristics and performance scores, game satisfaction questionnaire | Human factors attitude survey, learning style scale | Teamwork ability(+)
Satisfaction(+) |
|
Yang et al., (2021)
China |
Nursing intern
75/78 |
RCT | Medication practice | Card game:
Participate in an offline card game and complete the discussion of clinical cases through competitions; Physical (in-person) |
4 weeks | Critical thinking ability scale, clinical practice ability scale | Critical thinking(+)
Practical ability(+) | |
|
Yang and Oh, (2022)
South Korea |
Nursing intern
29/54 |
Quasi-experimental | Neonatal resuscitation | Virtual reality game:
Attend knowledge lectures in Week 1. In Week 2, use a head-mounted display to perform hands-on operations in a virtual game environment; Digital |
2 weeks | Neonatal resuscitation nursing knowledge, problem solving ability scale, clinical reasoning ability scale, confidence in performing resuscitation, anxiety level scale, learning motivation scale | Motivation(+)
Self-confidence(+) Problem-solving ability(+) Anxiety(-) Reasoning ability(-) | |
|
Yang et al., (2023)
China |
Undergraduate nursing student
21/21 |
RCT | Obstetric-related knowledge | Escape room game:
Students use their maternity-related knowledge to find and utilize the right clues to help them complete 5 questions and finally successfully escape the room; Digital |
5 weeks | Critical thinking questionnaire, problem solving scale, academic performance | Theoretical test score(+)
Problem-solving ability(+) Critical thinking(+) Knowledge retention(+) | |
|
Zhou et al., (2023)
China |
Nursing intern
44/44 |
Quasi-experimental | Pediatric nursing | Simulation game:
Applied game in week 3, the game has 10 consecutive levels with increasing difficulty. If students fails to pass a level twice in a row, the game will fail; Digital |
4 weeks | Clinical reasoning and reflective self-assessment scale, student learning satisfaction and confidence scale, exit examination scores | Reasoning ability(+)
Reflective ability(+) Self-confidence(+) Satisfaction(+) Theoretical test score(+) Skills performance(+) |
Table 2
| Evaluation indicators summary | Outcomes
(Indicators name) |
Counts | Sources(Author, Year) |
| knowledge objectives | Theoretical test score | 12 | Zhou et al., (2023); Luo et al., (2020); Huang et al., (2021); Yang et al., (2023); Buttussi et al., (2013); Chen et al., (2015); Craig et al., (2023); Kulakaç and Çilingir, (2024); Tan et al., (2017); Rodríguez-Ferrer et al., (2022); Nasirzade et al., (2024); Baek and Lee, (2024) |
| Knowledge retention | 5 | Huang et al., (2021); Yang et al., (2023); Buttussi et al., (2013); Kulakaç and Çilingir, (2024); Rodríguez-Ferrer et al., (2022) | |
| Knowledge transfer | 4 | Blanié et al., (2020); Verkuyl et al., (2018); Mitchell et al., (2021); Baek and Lee, (2024) | |
| Competence objectives | Skills performance | 10 | Zhou et al., (2023); Sun et al., (2022); Luo et al., (2020); Liu et al., (2022); Boada et al., (2015); Kulakaç and Çilingir, (2024); Ma et al., (2021); Tan et al., (2017); Nasirzade et al., (2024); Baek and Lee, (2024) |
| Reasoning ability | 4 | Zhou et al., (2023); Yang and Oh, (2022); Blanié et al., (2020); Johnsen et al., (2016) | |
| Problem-solving ability | 4 | Yang and Oh, (2022); Yang et al., (2023); De Beer et al., (2023); Baek and Lee, (2024) | |
| Teamwork ability | 3 | Wong et al., (2022); De Beer et al., (2023); Rodríguez-Ferrer et al., (2022) | |
| Communication ability | 2 | Liu et al., (2022); Hara et al., (2021) | |
| Critical thinking | 2 | Liu et al., (2022); Yang et al., (2023) | |
| Practical ability | 2 | Yang et al., (2021); Liu et al., (2022) | |
| Decision-making ability | 2 | Adhikari et al., (2021); Johnsen et al., (2016) | |
| Reflective ability | 1 | Zhou et al., (2023) | |
| Self-learning ability | 1 | Luo et al., (2020) | |
| Emotional objectives | Satisfaction | 9 | Zhou et al., (2023); Sun et al., (2022); Luo et al., (2020); Liu et al., (2022); Wong et al., (2022); Blanié et al., (2020); Cunha et al., (2023); Johnsen et al., (2016); Khan and Kapralos, (2019) |
| Self-confidence | 8 | Zhou et al., (2023); Yang and Oh, (2022); Adhikari et al., (2021); Buttussi et al., (2013); Cunha et al., (2023); Mitchell et al., (2021); Tan et al., (2017); Baek and Lee, (2024) | |
| Motivation | 4 | Yang and Oh, (2022); Blanié et al., (2020); Buttussi et al., (2013); Mitchell et al., (2021) | |
| Empathy | 3 | Chen et al., (2015); Ma et al., (2021); Rodríguez-Ferrer et al., (2022) | |
| Anxiety | 2 | Yang and Oh, (2022); Adhikari et al., (2021) |
© 2025 Elsevier Ltd