1. Introduction
Since December 2019, the world has suffered from an unprecedented strike of the COVID-19 pandemic. The World Health Organization (WHO) has reported 98,794,942 confirmed cases and 2,124,193 deaths by 25 January 2021 [1]. The pandemic has not only forced the global citizens to get accustomed to a new norm of social distancing, but it also makes a dramatic influence on education and pedagogy. Due to the implementation of social isolation strategies on campus, universities have to redevelop a curriculum that supports fully online instructions [2] and rethink the effectiveness of the pedagogy that is sustained in a fully online offering [3]. The ongoing social distancing protocol may not only impose challenges to the education system but also offer an opportunity for a rapid and massive curriculum and pedagogy reform [4]. Nursing pedagogy reform is especially important, because nurses, as key members of the frontline healthcare workers, play a crucial role in fighting disease and pandemics on the frontline, making decisions, and providing effective clinical care to patients. The quality of the nursing training directly affects the quality of clinical care that nurses provide to patients in their workplace practices [5]. The better-trained nurses will be able to protect themselves in complex and severe working situations and they can save thousands of lives from deadly disease or a pandemic such as COVID-19 [6]. In the early 2000s, nursing authorities and professional associations warned about the severe gap existing between the traditional nurse pedagogy and the actual clinical practice, and they urged for the reform of the nursing curriculum [5]. The call for a curriculum reform continues to be mentioned in the 2010s, such as in The Future of Nursing: Leading Change, Advancing Health [5] and Educating Nurses: A Call for Radical Transformation [6]. Even though the academic–practice gap has been perceived and the warning has been made, both the Institute of Medicine (IOM) and the National League of Nursing (NLN) acknowledged that the curriculum and pedagogy reform in nursing education would be difficult and the progress would be very slow. However, when the unanticipated coronavirus spread rapidly across nations and there was no effective treatment for COVID-19, nurses were not ready to confront the fatal disease, so they suffered great pressure during their daily work [7]. High anxiety issues were reported when they were not confident of clinical care to patients [8]. It is time for educators to rethink the nursing pedagogy, and to prepare novices with adequate skills to respond to the potential disasters they may encounter on their first day of work [9]. Furthermore, the pandemic and the campus suspension evoke the reconsideration of the role of e-learning and the alternative pedagogy in nursing education in a post-COVID era [10]. When the social distancing protocol limits the physical interaction between instructors and students, an alternative method must be prepared for students which enables them to learn remotely without constraining physical distance [11]. In light of the dramatic challenges imposed by the deadly pandemic, the present study aimed at discussing an alternative teaching approach, the Crowdsourcing Collaborative Learning Strategy (CCLS), to support preservice nurses in acquiring knowledge in a context-based and collaborative online community. The present study applied a quasi-experimental design. To investigate the effectiveness of the CCLS online platform, the present study experimented on evaluating the learning outcome differences of applying the CCLS online platform as an alternative pedagogy versus implementing the traditional teacher-centered lecture and teacher demonstration. A t-test on pre–post test scores between the control and experimental group was applied to discover the effectiveness of the CCLS online platform. For students in the experimental group, two questionnaires were used to explore students’ perception of the effectiveness of the CCLS and the user’s perception of using CCLS during the knowledge acquisition.
2. Literature Review
2.1. Mobile Learning in Nursing Education in Post-COVID Era
Mobile learning is not a new concept in the education field. Mobile learning is defined as the learning that can be delivered via mobile devices and wireless service [12]. Researchers in the 1990s devoted their efforts to incorporating media and technology-related activities into classrooms to add twists to traditional pedagogy [13]. In the early 2000s, with the vast development of information technology and the accessibility of the internet, the studies and practices of integrating web 2.0 tools into daily classroom activities, such as facilitating collaborative learning when students are posting, sharing, and discussing learning materials via online platform or software applications [14] and encouraging a transition from traditional teacher-centered instruction to a more learner-centered instruction [15]. The evolution of portable devices and wireless technology results in a radical change to the lifestyles of people living in the information age [16] and a tremendous change to the learning style of new millennium learners [17]. As more technology tools are being added in the daily classroom activities to facilitate learning, educators and professional associations call for a need to incorporate information technology into the nursing curriculum [18]. However, based on a systemic review of the literature, Abuatig and his colleagues [19] found only 13 articles published under “e-learning in nursing” with a time frame from 1971 to 2016. Similarly, Voutilainen [20] searched articles published between 2011 to 2015 with a wider range of keywords, such as “(‘computer’ OR ‘web’ OR ‘internet’ OR ‘online’ OR ‘information technology’ OR ‘mobile’ OR ‘phone’ OR ‘personal digital assistant’ OR ‘handheld’ OR ‘tablet’) AND (‘learning’ OR ‘education’ OR ‘instruction’ OR ‘problem solving’) AND (‘nurse student’ OR ‘student nurse’) AND (‘intervention’ OR ‘randomized controlled trial’ OR ‘comparison’)”; nine articles were related to the topic of e-learning in nursing education. No strong generalizations of the effect of e-learning on nursing education have been reported. Researchers encouraged future efforts to propose the e-learning method that can be more effective than the traditional teaching method.
Moreover, in a joint investigation, Crawford [21] provided a discussion on digital pedagogy responses of higher education to COVID-19 in 20 countries across all six WHO regions. The report shows that the developing economies decided to suspend classes and prolong the semester break before the pandemic is under control; meanwhile, the majority of developed economies chose not to extend the semester break, but to close the campus and switch to online instruction. However, the analysis reveals the implementation of online instructions is not as successful as researchers have anticipated. The online courses are largely based on convenient resources, such as narrated PowerPoint presentations, and the pedagogy is much closer to the traditional teacher-centered lecture, but is being delivered by technology tools such as Skype, Google Classroom, or Moodle. Teaching methods, such as learning groups and peer instruction, are reported to have been modified, reduced, or even eliminated from the options of online instruction methods and hence students are less engaged in online instruction during the pandemic [22]. The pandemic forces nursing educators and educators to rethink the possibility of developing an effective e-learning pedagogy to support engaging and collective online learning communities [23]. Researchers and nursing educators call for a further effort in integrating informational technology into the nursing curriculum design in the post-COVID era.
2.2. Crowdsourcing, Personalized Learning, and Nursing Education
The better-trained nurses are believed to be more capable of implementing proper treatment to the patients when they encounter complex situations in the future workplace [24]. However, novice nurses often feel a disconnect between the textbook knowledge and the actual practice and are not confident enough to solve problems and to provide clinical care to patients in the real work context [25]. Novice nurses are reported to have the highest dissatisfaction and insecurity in the workplace because they think they are not competent to provide high-quality healthcare to patients after a 4-year preservice training in nursing programs [26]. Therefore, it is important to not only help student nurses build the connection between their school tasks and their future workplace practice but also foster student nurses’ ability to solve workplace problems [27].
Crowdsourcing was originally proposed by Jeff Howe referring to an innovative business model (Howe, 2006). The term crowdsourcing is a combination of crowd and outsourcing, which refers to a company or a third party outsourcing a task to a large number of individuals, so the task can be completed under the wisdom of the crowd. The essence of crowdsourcing is to outsource a real-world problem to a group of people who actively participate in initiatives [28] and let those people utilize the collective intelligence to generate the content and to solve the problem [29]. The key features of crowdsourcing are, (a) depending on the internet and modern technology; (b) relying on collective intelligence; (c) creating user-generated content [30] to meet the need of developing a technology-enhanced, problem-based, and learner-centered nursing pedagogy. Researchers acknowledge that the evolution of social media makes it possible for nursing students to use internet-based platforms to share user-generated information in a medically focused professional community [31], engage in peer-to-peer knowledge construction [32], and acquire knowledge [33]. The emphasis on utilizing the information technology and the collective intelligence to create a productive content naturally fits crowdsourcing itself in the great interest of exploring the novel way of combining social network and technology to encourage students to actively participate in a learning process and to adapt to students’ individual needs based on collaborative learning [34]. The majority of publications addressing crowdsourcing in the collaborative learning scope can be found in the field of education and information technology. In an extensive review of articles published in databases foundaed by the Institute of Electrical and Electronics Engineers (IEEE Xplore) and the Association for Computing Machinery (ACM Digital Library) from 2013 to 2019, the implementation of crowdsourcing in the education genre was focused on discussing collaboration among a massive crowd in massive open online courses (MOOCs) [35]. However, as a newly emerging topic, no consensus is achieved on defining the member of the crowd [36]. It is worthwhile to investigate crowdsourcing and collaborative learning among a massive crowd, which the majority of studies have carried out. It is also important to explore whether learning can be enhanced in a small scope of crowd collaboration, for example, among members in a regular classroom. The variation of the crowd is crucial in helping clarify key elements of crowdsourcing and collaborative learning and its future implementation. Attempts have been made to discuss crowd collaboration in discovering learning pathways in a relatively small group. For example, learning patterns has been discussed in collaboration via the Self-Adaptive Learning through Teaching (SALT) system in a particular class [37]. Collaboration via Heurez, which is defined as allowing students to solve tasks by themselves through brainstorming and open discussion with 100 participants, was proven to be effective in a workshop trial [37]. Such studies showed the possibility that students may capable of forming their own “crowd” within a class and benefit from learning collaboratively within such a crowd. Yet more empirical evidence is expected and more attempts are encouraged to explore such a topic in various fields.
Personalized learning is tailored based on each student’s strengths, needs, and interests [38]. Personalized learning often be enhanced by the vast developed technology, which provides great flexibility for each student to decide how, what, when, and where they learn through apps and/or online platforms [39]. In contrast to the one-size-fits-all traditional teacher-centered instruction, personalized learning refers to student-centered instruction, in which “learning objectives, instructional approaches, and instructional content all vary based on learner needs”. Besides, learning activities are meaningful and relevant to learners, driven by their interests, and often self-initiated [40]. The key challenge of implementing personalized learning in the daily curriculum is how to efficiently collect students’ strengths, weaknesses, and needs during the learning process [41], One recent promising approach is to merge crowdsourcing and personalized learning together, which enables the instructor to collect students’ data during their learning process [42] and makes crowdsourcing and personalized learning more practical in the education field [43]. By taking advantage of “the wisdom of the crowd”, incorporating crowdsourcing in the nursing curriculum enables the instructor to collect data on students’ strengths and weaknesses and provide customized advice to students accordingly [44] and hence enhance personalized learning [45].
As promising as it sounds theoretically, putting crowdsourcing strategy into practice and creating a personalized learning environment has encountered great challenges in reality. Both crowdsourcing and personalized learning remain to be fuzzy topics and as a poorly studied domain that needs a clear definition and consensus of implementation as an innovative pedagogy [46]. Research has also showed that the majority of higher education students lack the competence to navigate themselves on the internet [47], and to eliminate false information [48], hence students easily get lost with the overabundance of information available online [49] and the user-generated information sometimes may lead students to the wrong path [45]. Therefore, the investigation on the quality control methods for applying crowdsourcing in education is significant in future research [50]. The present study explores an approach to apply crowdsourcing and personalized learning methods into clinical skill training in a nursing class. The integration of crowdsourcing and personalized learning in the nursing curriculum are expected to increase the possibilities of achieving a higher level of customization, interaction, clinic skill acquisition during the learning process [51]. The present study also echoes the call of nursing education reform that is urging a dramatic reform and a major overhaul of traditional teacher-centered pedagogies [52], minimizing the academic–practice gap and ensuring nursing students have sufficient clinical skills set to perform tasks in their future career [53], implementing a context-based, collaborative and student-centered pedagogy [54], and incorporating technology tools to facilitate learning and skill acquisition [55].
3. Methods
3.1. A Supportive and Personalized Learning Environment for Student Nurses
The proposed Crowdsourcing Collaborative Learning Strategy (CCLS) in the present study aims to combine the simulated workplace problem in the Objective Structured Clinical Examination (OSCE), the video recorded clinical skill performance, and crowdsourcing video annotation toward the clinical performance altogether. The CCLS aims to assist nursing students to acquire necessary clinical skills through content co-construction. In the CCLS online learning community, where the context is initiated by students (e.g., the recorded videos and annotations towards the video) and the knowledge co-construction is supervised by the instructor, student nurses can construct functional knowledge on clinical skills (e.g., OSCE scenarios) via online collaboration, and the quality of the co-constructed knowledge is controlled by the instructor (Figure 1).
Different from the traditional teacher-centered stand-alone lectures, in which students receive information passively from direct instruction and skill demonstration, the CCLS requires students to work actively and collaboratively to co-construct the knowledge. In the CCLS process, a student nurse has to first make a response to a proposed workplace problem, then record his/her solution to that problem, and uploads the response to the online platform. A proposed problem in this CCLS particularly means the clinical skills listed in the OSCE. The OSCE offers a controlled stimulated working environment to help nursing students integrate all functional knowledge, practice specific clinical skills to simulated patients, and to solve problems they may encounter in the future workplace [56]. The OSCE has proved to be highly valid and reliable in assessing nursing clinical competencies [57] and to be highly beneficial for student nurses to fill the academic–practice gap that enables preservice nurses to perform the clinical skills successfully and confidently on patients in the future workplace [58]. Therefore, the proposed problem in CCLS represents the real-world problems student nurses may encounter in their future workplace where each video solution uploaded by an individual student represents the readiness of that student to solve the real-world problems on the concurrent status. Once the solutions to the proposed problem are uploaded online by each student and reach the “crowd”, which are student nurses who have registered in this course and have access to the online platform, the crowd can review those videos and exchange ideas about their thoughts on each solution toward the same working scenario (the OSCE problem) via video annotation.
Under a crowdsourcing lens, the set of core features of crowdsourcing include distributed problem-solving [59], user-generated content [60], online collaboration [61], and information and knowledge sharing [62]. In CCLS, each uploaded video represents user-generated content, whereas online video annotation represents information and knowledge sharing and online collaboration. Once student nurses record their clinical practices with the simulated patient and upload the video online for a peer-annotation, the knowledge sharing and collaborative learning start to function. Research has shown that collaborative learning enables students to actively share information within a crowd, increases students’ engagement to the course content, and helps students achieve higher levels of thinking [63]. In CCLS, reflective learning is triggered by peer-annotation, knowledge sharing, and collaborative learning. Reflective learning often takes place in a learning community, in which students can exchange information on acquired knowledge and hence they can learn from each other [64]. Reflective learning is reported to be highly effective in fostering active and deep learning via students’ self-reflection on their past activities and collaboration on knowledge construction [65]. In CCLS, through video recording and video annotating, student nurses can reflect on their performances, compare their performances with their classmates’, recognize the difference among each uploaded performance, and rethink whether there is a better solution to the proposed problem. Such a process makes CCLS an effective strategy to cultivate students’ ability to analyze, synthesize, and evaluate information while acquiring information [66]. Hence, it enhances higher levels of cognitive skills [67], enhances deep learning [68], and leads to better knowledge acquisition [69]. Students who annotate the video can rehearse the operational procedures [70] and compare existing knowledge with the knowledge performed in the video [70]. Hence, they can either strengthen knowledge on correct information or correct the wrong information [71].
Student nurses are reported to often practice their nursing operational skills without expert supervision and lack feedback about their performance [72]. The fundamental role of the instructor in CCLS is to supervise the process of knowledge co-construction. In CCLS, the instructor lets the students themselves perform the clinical skills toward the simulated working scenario and construct knowledge via online discussion and video annotation. Instead of directly delivering the knowledge to students, the instructor supervises clinical skill performance via each uploaded video and guides the online annotation process in case there is a fundamental mistake. Via each performance video uploaded and annotated by student nurses, the instructor can directly assess each student’s understanding toward acquired knowledge and can predict his/her clinical performance in the future workplace. The online supervision helps the instructor to visualize the common misunderstanding student nurses have as well as the specific problems each student nurse has. The instructor is then able to provide tailored advice accordingly. In CCLS, student nurses are not only able to construct content knowledge through their practice on solving problems and annotating the practices from others, but also through the customized guidance that comes from the instructor. The process of co-construction on content knowledge is under the supervision of the instructor to ensure students are on the right track and the final solution to the proposed problem is effective in their future workplace practice. Therefore, the CCLS online platform creates a promising environment to promote personalized learning. Students can access the CCLS online platform via any digital device. Without the limitation of a physical learning space, the connectivity between students and the learning activities has been strengthened [71] students can plan and control their learning process on their path and embrace the new and expanded learning experiences [55]. In CCLS, via each uploaded video, the instructor can collect students’ data, such as their learning progress, understanding, strengths, or weaknesses, and then the instructor can provide individual feedback to students [55].
The present study investigates the effectiveness of applying to crowdsource to facilitate nursing students’ aquisition of professional skills and hoping to provide some evidence on pedagogy innovation and e-learning implementation in nursing education. It serves as a pilot project to investigate the effectiveness of CCLS as an alternative pedagogy to nursing education. Since the CCLS will be carried on an online platform and will be accessed via portable mobile devices, as a pilot, the study focuses on discussing the basic elements such as the effectiveness of the CCLS and user’s perception of using CCLS during the knowledge acquisition. The research questions are listed as follows:
Whether students in the CCLS condition outperform students in the traditional learning condition?
Whether students in CCLS condition perceive using CCLS online platform as helpful and easy to use?
Whether students in CCLS condition perceive using CCLS online platform as effective in facilitating knowledge acquisition?
3.2. Participants
The participants include 49 nursing students enrolled in the course Nursing Skills and Implementation at a university located in southern Taiwan. Participants were in their third year in college and their age ranged from 20 to 22 years old and they were required to take the nursing OSCE by the end of the semester.
After the class registration, an email was sent to all of the students who had enrolled in the course Nursing Skills and Implementation. Participants were invited to sign the attached consent form and to participate in the study voluntarily. Participants were informed that (1) the study would be conducted during the semester they had enrolled in; (2) all students were invited to participate in this study; (3) there were several suggestions on being a responsible participant in the study; (4) an extra course credit would be rewarded for participating in the study; (5) participants could drop-off the study at any time during the semester; (6) students who do not want to participate the study would still need to complete the tests and the OSCE as part of the course requirement, but their information would not be used in the study and they would not receive the extra credits for participating in the study.
3.3. Instrument
3.3.1. Class
The class Nursing Skills and Implementation is a required class for clinical skills training before a student nurse enters the nursing internship program as a senior. Due to the health regulation in the COVID period, one class is designed as a traditional face-to-face class, in which students had to physically attend the class under strict health guidelines, such as fixing the seats, measuring temperature, keeping social distance, wearing facial masks, and so on. Only when the COVID pandemic got worse in some specific locations in Taiwan, the campus was closed, and face-to-face classes converted to online instruction. The other class was delivered fully online for students who have health concerns or had to take health quarantine. The class has two separate course codes for class registration. Participants, based on their preference, registered for the course online via an educational administrative management system. The reality and the social distancing protocol during the pandemic offered a unique chance for this study to compare the differences between the traditional face-to-face pedagogy and a newly emerging online crowdsourcing intervention. Students were welcomed to select either course based on their preference and health concerns during the pandemic period. The course code differentiates the teaching method, which naturally divided two classes as the control group and the experimental group. Students in the control group received the traditional teaching method and had to complete the tasks and tests as they normally did. Whereas, students in the experimental group received the CCLS intervention and had to complete the tasks and tests as they normally did.
3.3.2. Instructor
The two classes were taught by the same instructor using the same course content to all participants.
3.3.3. Assessment
The Objective Structured Clinical Skills Examination (OSCE) was applied to assess the learning outcome. The Objective Structured Clinical Skills Examination (OSCE) was first established in the UK in 1975. Since then, the OSCE has been used as an effective objective repertoire of clinical evaluation to help healthcare professionals assess their clinical competence before they are allowed to practice the skills on patients in the workplace. According to the regulation, all student nurses are required to pass the OSCE before they are eligible to enter the nursing internship program.
3.3.4. CCLS
The CCLS was not a concrete instrument but was an alternative pedagogy delivered via an online platform. Participants can access the CCLS online platform either from the learning management system on a PC, or from the learning management app on any portable devices, such as an iPad, or a smartphone.
Questionnaire Two questionnaires were applied to investigate participants’ perception of their learning outcomes via the CCLS online platform (the Questionnaire on Learning Through the CCLS Online Platform) and the participants’ attitude toward using the CCLS online platform (the Questionnaire on Acceptance of Using the CCLS Online Platform). The questions in Questionnaire on Learning Through the CCLS Online Platform (Questionnaire 1) were modified from two published articles, the article Students’ Perceptions of Learning Outcomes from Group-based, Problem-Based Teaching and Learning Activities [71], addressing nursing students’ perception of group-based and problem-based pedagogy to help them to build a connection between textbook theory and actual practices, and A Manual for the Use of the Motivated Strategies for Learning Questionnaire [65] addressing students’ perception of their ability after knowledge acquisition. A Cronbach’s coefficient alpha was measured to ensure the reliability of the research instrument (Table 1). The Cronbach’s alpha value of the entire questionnaire was 0.812, which considered the questionnaire to be reliable.
The questions in the Questionnaire on Acceptance of Using the CCLS Online Platform (Questionnaire 2) were revised from the Technology Acceptance Model (TAM) and the questions were tested effective by Park [71] in the article, An Analysis of the Technology Acceptance Model in Understanding University Students’ Behavioral Intention to Use e-Learning. Furthermore, before implementing the two questionnaires, a Cronbach’s coefficient alpha was measured to ensure the reliability of the research instrument [71]. A Cronbach’s coefficient alpha for the revised TAM was measured (Table 2). The Cronbach’s alpha value of the entire questionnaire was 0.813, which considered the questionnaire to be reliable. The 5-point Likert scale was applied for the calculation on each indicator presented in the questionnaires.
3.4. Experimental Procedure
The study was a semester-long investigation lasting for 18 weeks (as shown in Figure 2 below). During week 1 to week 6, the instructor taught functional knowledge. To minimize the lecture differences between the groups, PowerPoint Slides were used to deliver functional information.
-
A simulated OSCE was held in week 7 to assess the clinical competence in both groups (Figure 3). All subjects gave their informed consent for inclusion before they participated in the study. The study was conducted in accordance with the Declaration of Helsinki, and the protocol was approved by the Ethics Committee of Tajen University (Project identification code: TAJEN-20200308172309-IRB). Participants in the experimental group were required to fill out their pre-test questionnaires (the Questionnaire on Learning Through the CCLS Online Platform). From week 8 to week 16, participants were required to practice the clinical skills with the simulated patient based on the OSCE rubric (Figure 4).
In the control group, the instructor performed each step physically with the simulated patient following the rubric and explained each clinical procedure in detail. Participants were then asked to practice the skills individually under the guidance of the instructor. The feedback to each clinical skill on participants’ performance returned to the participants within 2 weeks after the instructor scored, analyzed, and synthesized all the performances on that particular skill. In the experimental group, the instruction of clinical procedures was delivered through videos recorded to the instructor. Participants in the experimental group were then asked to make an appointment to practice the skills with the simulated patient. Participants had to practice the skills alone, but the practice was recorded and uploaded to the CCLS online platform. Once the practice was uploaded, the instructor and the people who are authorized to access the CCLS only platform (e.g., all participants in the experimental group) were able to view the uploaded video and annotated the video via the video annotation tools (Figure 5).
Likewise, to minimize the differences between groups, the instructor’s feedback to each clinical skill on participants’ performance returned to the participants within 2 weeks after the instructor scored, analyzed, and synthesized all the performances on that particular skill. However, participants were encouraged to annotate the videos before the instructor’s feedback was delivered. Participants were awarded extra points for annotating all five videos for each clinical skill. Week 17 to week 18 was for the OSCE. Participants in both groups had to take the OSCE. Participants in the experimental group were required to fill out their post-test questionnaires (the Questionnaire on Learning Through the CCLS Online Platform and Questionnaire on Acceptance of Using the CCLS Online Platform).
4. Results
4.1. Analysis of Effectiveness of CCLS
A t-test was carried out on assessing student nurses’ prior knowledge on clinical skills (Table 3) before the actual practice of those skills on the simulated patients. The results showed the mean score for the experimental group as 37.41 with a standard deviation of 6.15, and the control group is 39.40 with a standard deviation of 4.34. No statistical significance was found in the pre-test results between the two groups (t = −1.20, p > 0.05). Before the CCLS intervention, student nurses in both groups presented the same level of knowledge on clinical skills. An Analysis of Covariance (ANCOVA) was further tested on the post-test results between the two groups (Table 4). After excluding the influence of the pre-test results, a statistical significance of CCLS pedagogy was found (F = 4.25, p < 0.05). Compared with their counterparts in the control group (mean = 55.75, SD = 5.35), the mean score for the experimental group (mean = 72.78, SD = 5.91) showed a positive effect on CCLS pedagogy on helping student nurses to acquire knowledge. The effect size (d = 0.948) suggests a large effect brought by CCLS pedagogy.
4.2. Analysis of Student Nurses’ Perception of CCLS
Student nurses in the experimental group were also required to fill the Questionnaire on Learning Through the CCLS Online Platform twice and the Questionnaire on Acceptance of Using the CCLS Online Platform at the end of the semester to reveal their perception of using CCLS. Among 24 student nurses in the experimental group, two of them failed to complete the questionnaires and their incomplete responses were eliminated from the analysis. The Questionnaire on Learning Through the CCLS Online Platform was presented twice during the semester: (1) after knowledge instruction and before CCLS intervention; (2) after using the CCLS online platform and completing the final OSCE. The questionnaire was conducted to reveal the changes in student nurses’ perception of the effectiveness of learning through the CCLS online platform (Table 5) and to investigate student nurses’ self-perception on their clinical competence and their confidence level of performing clinical operations. A t-test was conducted to show the students’ perception difference on learning through the CCLS online platform. The reported self-perception on the cognition level of clinical skills and confidence of performing those skills showed a significant difference between pre-post questionnaires. Among all the eight questions, except question 6, I am confident that I can perform all practiced skills successfully on my patient during the intern period (t = −1.66, p > 0.05), a significance was reported from the other seven questions related to student nurses’ perception of improvement on their knowledge level of clinical skills, the confidence level of successfully performing the clinical skills in their future workplace, and the effectiveness of the CCLS online platform on facilitating knowledge acquisition. Even though question 6, I am confident that I can perform all practiced skills successfully on my patient during the intern period, was not reported as statistically significant, it does not meanthat students were not confident about their clinical competence, which a statistical significance that was found on question 5, I am confident with my clinical skills. It might be due to students’ anxiety about performing a clinical operation on real human patients. Besides, an overall large effect size was reported on the difference in student nurses’ perception of learning through the CCLS online platform. Student nurses believed CCLS helped them acquire knowledge. The large effect size was found on item 1 (d = 0.866), item 2 (d= 1.628), item 3 (d = 1.257), and item 4 (d = 0.97). Furthermore, student nurses were reported to feel more confident in their clinical skills and supported by the large effect size on item 5 (d = 0.84) item 7 (d = 0.985) and item 8 (d = 1.649).
Student nurses’ acceptance of the CCLS online platform was assessed through the Questionnaire on Acceptance of Using the CCLS Online Platform, which was revised from the Technology Acceptance Model (TAM), and the questions in TAM were tested as effective by Park [71]. The questionnaire in this study consisted of seven questions on the scale of usefulness and ease of use (Table 6). According to the five-point Likert scale, which ranged from 1 (strongly disagree) to 5 (strongly agree), student nurses reported positively to all questions concerning the usefulness and the ease of use CCLS online platform, and agreed that CCLS can effectively deliver course material (mean = 4.27), build an academic–practice connection (mean = 4.27), enhance actual clinical skills performance (mean = 4.45), and the CCLs was easy to access (mean = 4.64) and ease to manipulate (mean = 4.45) via portable devices.
5. Discussion
Clinical nursing training is an essential process for students to perform professional health services to future patients. However, the traditional nursing pedagogies with a teacher-centered lecture approach was questioned about the existence of academic–practice gap, which means it may not sufficient to help pre-service nurses fit in with their future workplace [17] which may create problems such as lack of self-efficacy and problem-solving skills during nursing clinical operations in a complex workplace context [17] Therefore, a call for the transformation from traditional teacher-centered pedagogy to a problem-based, collaborative and student-centered pedagogy has been aroused since the early 2000s [17] In light of such a call for nursing pedagogy transformation, the present study investigated the effectiveness of an alternative student-centered pedagogy. Besides receiving direct instruction from the teacher, student nurses in this study can construct knowledge through physically practicing clinical skills to minimize the academic–practice gap, sharing acquired knowledge with their classmates to rehearse the acquired knowledge, receiving feedback from both the teacher and the peer classmates, and working collaboratively to find the best solution to a particular problem. In general, students in the CCLS group outperformed students in traditional learning conditions at the end of the training semester, in which students in the CCLS group performed better than their counterparts in the final OSCE test. Through a pre–post OSCE analysis on student nurses’ clinical skill performance, it can be seen that the alternative CCLS pedagogy is more effective for improving the accuracy of performing clinical skills than the traditional teacher-centered pedagogy. In terms of student nurses’ self-perception on the effectiveness of the CCLS online platform, the pre–post results of the Questionnaire on Learning Through the CCLS Online Platform also revealed that student nurses felt reviewing their self-recorded performance video plus annotating performance videos from the others on the CCLS online platform was effective in helping them confirm the acquired knowledge, improving clinical skills, and being more confident in successfully performing the clinical skills in their future workplace. For example, student nurses confirmed that the CCLS online platform was effective in connecting textbook knowledge (e.g., question 1: I am familiar with basic clinical nursing skills after viewing my performance video and annotating videos from the others) to actual clinical practice (e.g., question 8: I can perform clinical skills much more accurate after performing the skills, annotate performance videos and review video annotations). Student nurses also showed continuing growth in their confidence level of performing clinical skills successfully after the semester-long skill training class, in which statistical difference was reported in questions such as I am confident with my clinical skills and I can recognize and manage risks in nursing clinical care. Moreover, the effectiveness of the CCLS online platform could be also verified from students’ perception in answering questions such as the instruction of the CCLS online learning platform is clear and is effective in guiding me understand the learning contents and combining the CCLS online learning platform and the actual clinical practice is helpful to acquire clinical skills. The finding is consistent with crowdsourcing, which claims the benefits of learning through collective intelligence or the wisdom of the crowd [42] that when students actively engaged in discussion, in articulating their understandings, in listening to different viewpoints, and defending their points, they are more likely to actively synthesize the acquired information, reframe ideas, and achieve a complete understanding [42] First, uploading onsite performance videos to the CCLS online platform enables student nurses to self-reflect their clinical performance from a different angle as a video reviewer, which provides an opportunity for student nurses to compare the performance along with the textbook knowledge and the OSCE rubric. Second, all the performance videos filmed on the same OSCE task create a video pool, representing different viewpoints toward the specific clinical problem. Through viewing, student nurses can compare and synthesize the information presented in those videos. By annotating those performance videos, student nurses can defend their performance, clarify their standpoints, and convince their peer classmates that their performance is following the OSCE rubric. With intensive interaction and collaboration on annotating performance videos and synthesizing information from annotations on the CCLS online platform, student nurses themselves can generate a comprehensive understanding of the specific clinical task rather than sitting passively and waiting for standardized feedback from the instructor. The role of the instructor in CCLS is not only to deliver information directly to student nurses but also supervises the process of online video annotation during the gap between completing the complex and high-labor cost OSCE scoring procedure and returning the feedback to student nurses. Researchers have argued that the traditional OSCE scoring procedure makes it impossible to provide onsite immediate feedback to students towards their performance [68]. Therefore, when the feedback is delayed, student nurses have to recall their performance on the test day, in which the recalled memory may distort and may not match their actual clinical competence [44] and hence the effectiveness of the instructor’s feedback has been diminished [44] Moreover, the OSCE test provides a very limited amount of time to score students’ onsite performance under high tension conditions. Researchers and examiners have reported the possibility of human error on scoring the OSCE due to time limitation and exhaustion [44] For CCLS, the onsite performance is recorded and uploaded right after student nurses perform the assigned clinical task. Both the student and the instructor are allowed to review and annotate the uploaded videos and leave comments towards the performance at any time and anywhere. The disadvantage of time limitation and exhaustion has vanished. The uploaded performance video does not only help student nurses to refresh and affirm their memory on their actual level of clinical competence on that particular clinical task and help student nurses to take advantage of professional feedback from the instructor, but also helps the instructor by reducing the possible errors made during the scoring procedure. The quality and effectiveness of the feedback have been maximized on the CCLS online platform.
To increase the level of ease to use and ease to access, instead of developing a new system, CCLS was merged in the established learning management system, in which students are used to checking in regularly to get e-learning credits according to university regulations. Embedding CCLS to the learning management system is user-friendly and will not create an extra workload for students to download, register, and accustom different operation panels. As for the results of the Questionnaire on Acceptance of Using the CCLS Online Platform showed, CCLS is successful on the level of easy to use and easy to access. Student nurses gave a high mark on all seven surveyed questions, which reveals students felt satisfied with their learning experience on the CCLS online platform. The recent trend in higher education has been to incorporate e-learning systems to facilitate learning [65] the level of easy-to-use and easy-to-access significantly affect students’ intention to use the e-learning system, and hence dramatically influence the helpfulness, usefulness, and effectiveness of the e-learning system during the practice [65] Therefore, before incorporating technology-related activities into the classroom to add twists to traditional nursing pedagogy [33] the level of easy-to-use and easy-to-access should be considered.
6. Limitations and Future Work
The present study has limitations: (1) participants are recruited from a single department of a single institution; (2) the scale of the crowd is relatively small; (3) the study is designed as a pilot to provide a test-view on the effectiveness of CCLS and it needs to be further polished in later attempts; (4) the dramatic differences between the control group and the experimental group may cause great differences and yields to the varied results. It is the chance to compare the difference between traditional pedagogy and an alternative pedagogy in nursing education. However, the notable difference existing in the experimental group may alter students’ motivation and learning behavior. It is possible that the improvement presented in this study is due to the attention given to the new intervention, which made students consciously learn and perform differently. Therefore, in the future, besides conducting it in other departments and other institutions, researchers are recommended to trace the theoretical support of CCLS (e.g., social cognitive theory, constructivism, collaborative learning, etc.) and try to define a promising scale of the crowd in school education and differentiate it with the vast scale of the crowd mentioned in learning via open resources and lifelong education.
7. Conclusions and Suggestions
The increasing threats of infectious disease further require a significant change in health care delivery and the preparation of pre-service nurses. Studies have shown that the traditional teacher-centered pedagogy is not sufficient to prepare preservice nurses to face the challenges in a complex workplace context. Nursing education has remained unchanged for over a half-century [45], and both academia and the industry are calling for a radical change of nursing pedagogy. The COVID-19 crisis brings both challenges and opportunities for traditional nursing education to embrace the benefits of modern technology. The present study takes the opportunity to provide an insight into how clinical training can be reshaped and combined with information technology during the pandemic. The proposed CCLS presents an alternative student-centered learning mechanism, which allows student nurses to actively construct functional knowledge on an online platform. The findings suggest the CCLS online platform can help students to revisit their clinical performance via the recorded videos, which not only facilitates student nurses’ self-reflection on their performance but also helps student nurses to minimize the academic–practice gap through comparing their performance with the textbook instruction and the OSCE rubric. Furthermore, the open online discussion and annotation toward the particular clinical task and its operation procedure leaves less pressure for student nurses to comment and argue on particular knowledge and helps strengthen their self-reflection toward acquired knowledge and skills. The instructor can provide more resources to target students’ strengths and weaknesses, and hence can provide more helpful instructions. The proposed CCLS has shown its effectiveness in helping student nurses improve their clinical skills performance. Meanwhile, from active contribution to comprehensive knowledge construction, student nurses are increasingly confident in their clinical competence.
Conceptualization, P.-S.H. and Y.G.; methodology, P.-S.H.; software, Y.-M.H.; validation, Y.-M.H.; formal analysis, Y.G.; investigation, Y.-M.H.; resources, P.-S.H.; data curation, Y.G.; writing—original draft preparation, P.-S.H.; writing—review and editing, Y.G.; visualization, Y.-M.H.; supervision, Y.-M.H.; project administration, Y.-M.H. All authors have read and agreed to the published version of the manuscript.
This research was funded by the Ministry of Science and Technology (MOST), grand number MOST 109-2637-H-127-001.
The study was conducted in accordance with the Declaration of Helsinki, and the protocol was approved by the Ethics Committee of Tajen University (Project identification code: TAJEN-20200308172309-IRB and been approved on 8 March 2020).
Informed consent was obtained from all subjects involved in the study. Written informed consent has been obtained from the student nurses and all participants to publish this paper.
The data presented in this study are available on request from the corresponding author. The data are not publicly available due to ethical considerations.
We would like to thank all teachers and student nurses that participated in the study.
The authors declare no conflict of interest.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Enlarge this image.Figure 1. The flow chart of the Crowdsourcing Collaborative Learning Strategy (CCLS).
Enlarge this image.Figure 2. A summary of the experimental duration and intervention. OSCE: Objective Structured Clinical Examination.
Reliability of the Questionnaire on Learning Through the CCLS Online Platform.
| Items | Content | Cronbach’s Alpha |
|---|---|---|
| Q1 | I am familiar with basic clinical nursing skills after viewing my performance video and annotating videos from the others. | 0.884 |
| Q2 | I am familiar with the important clinical operations. | 0.796 |
| Q3 | I am familiar with the process of assessing patients’ clinical needs. | 0.801 |
| Q4 | I am familiar with the process of evaluating patients’ psychological and social needs. | 0.825 |
| Q5 | I am confident with my clinical skills. | 0.814 |
| Q6 | I am confident that I can perform all practiced skills successfully on my patient during the intern period. | 0.803 |
| Q7 | I can recognize and manage risks in nursing clinical care. | 0.769 |
| Q8 | I can perform clinical skills much more accurately after performing the skills, annotate performance videos and review video annotations. | 0.810 |
Reliability of the Questionnaire on Acceptance of Using the CCLS Online Platform.
| Items | Cronbach’s Alpha | |
|---|---|---|
| Usefulness | (a) The instruction of the CCLS online learning platform is clear and effective to guide me to understand the learning contents. | 0.798 |
| (b) Combining the CCLS online learning platform and the actual clinical practice is helpful to acquire clinical skills. | 0.796 | |
| (c) My personal experience of learning via CCLS benefits my clinical performance on patients. | 0.763 | |
| Easiness | (d) It is easy to annotate the videos via CCLS online learning platform on mobile devices | 0.788 |
| (e) It is easy to read the annotate information on the CCLS online learning platform via mobile devices | 0.805 | |
| learning platform is well-matched with the learning progress on the site. | 0.801 | |
| (g) I think the interface of CCLS is easy to use. | 0.809 | |
T-test of the two groups on the pre-test results.
| Group | N | Mean | SD 1 | t (49) | d 2 |
|---|---|---|---|---|---|
| Experimental group | 24 | 37.41 | 6.15 | −1.20 | 0.376 |
| Control group | 25 | 39.40 | 4.34 |
1 SD represents Standard deviation; 2 d represents the effect size.
ANCOVA results of the two group on the post-test results.
| Group | N | Mean | SD 1 | F (1, 49) | d 2 |
|---|---|---|---|---|---|
| Experimental group | 24 | 72.78 | 5.91 | 4.248 * | 0.948 ** |
| Control group | 25 | 55.75 | 5.35 |
1 SD represents Standard deviation; 2 d represents the effect size; * represents p < 0.05; ** represents a large effect size (d > 0.8).
Student nurses’ perception of learning through the CCLS online platform.
| Item | Question | N | Mean | SD 1 | t | d 2 | |
|---|---|---|---|---|---|---|---|
| 1 | I am familiar with basic clinical nursing skills after viewing my performance video and annotating videos from the others. | Pre-test |
22 |
3.77 |
0.81 |
−3.13 * | 0.866 ** |
| 2 | I am familiar with the important clinical operations. |
Pre-test |
22 |
3.59 |
0.66 |
−5.70 * | 1.628 ** |
| 3 | I am familiar with the process of assessing patients’ clinical needs. | Pre-test |
22 |
3.73 |
0.63 |
−4.85 * | 1.256 ** |
| 4 | I am familiar with the process of evaluating patients’ psychological and social needs. | Pre-test |
22 |
3.59 |
0.85 |
−4.12 * | 0.97 ** |
| 5 | I am confident with my clinical skills. | Pre-test |
22 |
3.68 |
0.64 |
−3.77 * | 0.84 ** |
| 6 | I am confident that I can perform all practiced skills successfully on my patient during the intern period. | Pre-test |
22 |
3.50 |
0.59 |
−1.66 | 0.4 |
| 7 | I can recognize and manage risks in nursing clinical care. | Pre-test |
22 |
3.73 |
0.88 |
−4.46 * | 0.985 ** |
| 8 | I can perform clinical skills much more accurately after performing the skills, annotate performance videos and review video annotations. | Pre-test |
22 |
3.68 |
0.78 |
−4.90 * | 1.649 ** |
1 SD represents Standard deviation; 2 d represents the effect size; * represents p < 0.05; ** represents a large effect size (d > 0.8).
Usefulness and easiness of student perception.
| Scale | Questionnaire Item | Mean | SD 1 |
|---|---|---|---|
| Usefulness | (a) The instruction of the CCLS online learning platform is clear and is effective in guiding me to understand the learning contents. | 4.27 | 0.82 |
| (b) Combining the CCLS online learning platform and the actual clinical practice is helpful to acquire clinical skills. | 4.27 | 0.55 | |
| (c) My personal experience of learning via CCLS benefits my clinical performance on patients. | 4.45 | 0.80 | |
| Easiness | (d)It is easy to annotate the videos via CCLS online learning platform on mobile devices. | 4.64 | 0.49 |
| (e) It is easy to read the annotate information on the CCLS online learning platform via mobile devices | 4.05 | 0.48 | |
| (f) The response speed of the CCLS online learning platform is well-matched with the learning progress on the site. | 4.45 | 0.59 | |
| (g) I think the interface of CCLS is easy to use. | 4.36 | 0.49 |
1 SD represents Standard deviation.
References
1. World Health Organization. WHO Coronavirus Disease (COVID-19) Dashboard. 2020; Available online: https://covid19.who.int/ (accessed on 3 March 2021).
2. Clark, R.C.; Mayer, R.E.; Thalheimer, W. E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning. Perform. Improv.; 2003; 42, pp. 41-43. [DOI: https://dx.doi.org/10.1002/pfi.4930420510]
3. Basilaia, G.; Kvavadze, D. Transition to Online Education in Schools during a SARS-CoV-2 Coronavirus (COVID-19) Pandemic in Georgia. Pedagog. Res.; 2020; 5, em0060. [DOI: https://dx.doi.org/10.29333/pr/7937]
4. Ashby, S.E.; Snodgrass, S.H.; Rivett, D.A.; Russell, T. Factors shaping e-feedback utilization following electronic Objective Structured Clinical Examinations. Nurs. Health Sci.; 2016; 18, pp. 362-369. [DOI: https://dx.doi.org/10.1111/nhs.12279] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/27029015]
5. Howe, J. The rise of crowdsourcing. Wired Mag.; 2006; 14, pp. 1-4.
6. National League for Nursing. Position Statement: Innovation in Nursing Education: A Call to Reform. 2003; Available online: http://www.nln.org/newsroom/nln-position-documents/%20archived-position-statement (accessed on 23 June 2020).
7. McClure, R.M.; Walters, J.; Bietau, L.; Daws, D.; Grosvenor, L. None of Us is as Smart as All of Us. Educ. Assess.; 1993; 1, pp. 71-89. [DOI: https://dx.doi.org/10.1207/s15326977ea0101_5]
8. Kittur, A.; Nickerson, J.V.; Bernstein, M.; Gerber, E.; Shaw, A.; Zimmerman, J.; Lease, M.; Horton, J. The future of crowd work. Proceedings of the 2013 Conference on Computer Supported Cooperative Work; San Antonio, TX, USA, 23–27 February 2013; pp. 1301-1318.
9. Köse, U. A blended learning model supported with Web 2.0 technologies. Procedia Soc. Behav. Sci.; 2010; 2, pp. 2794-2802. [DOI: https://dx.doi.org/10.1016/j.sbspro.2010.03.417]
10. Chang, Y.-J.; Li, L.; Chou, S.-H.; Liu, M.-C.; Ruan, S. Xpress: Crowdsourcing Native Speakers to Learn Colloquial Expressions in a Second Language. CHI ’13 Extended Abstracts on Human Factors in Computing Systems on—CHI EA ’13; ACM: New York, NY, USA, 2013; pp. 2555-2560. [DOI: https://dx.doi.org/10.1145/2468356.2468829]
11. Shehab, S.; Rahim, R.E.A.; Daud, S. Knowledge sharing behavior of nursing supervisors in online healthcare communities. Int. J. Pharm. Res.; 2019; 11, pp. 1662-1678.
12. AlJeraisy, M.N.; Mohammad, H.; Fayyoumi, A.; Alrashideh, W. Web 2.0 in education: The impact of discussion board on student performance and satisfaction. Turk. Online J. Educ. Technol. TOJET; 2015; 14, pp. 247-258.
13. Weld, D.S.; Adar, E.; Chilton, L.B.; Hoffmann, R.; Horvitz, E.; Koch, M.; Landay, J.; Lin, C.H.; Mausam, M. Personalized Online Education-A Crowdsourcing Challenge. Proceedings of the Workshops at the Twenty-Sixth AAAI Conference on Artificial Intelligence; Toronto, ON, Canada, 22–23 July 2012.
14. Budge, E.J.; Tsoti, S.M.; Howgate, D.J.; Sivakumar, S.; Jalali, M. Collective intelligence for translational medicine: Crowdsourcing insights and innovation from an interdisciplinary biomedical research community. Ann. Med.; 2015; 47, pp. 570-575. [DOI: https://dx.doi.org/10.3109/07853890.2015.1091945]
15. Ghezzi, A.; Gabelloni, D.; Martini, A.; Natalicchio, A. Crowdsourcing: A Review and Suggestions for Future Research. Int. J. Manag. Rev.; 2017; 20, pp. 343-363. [DOI: https://dx.doi.org/10.1111/ijmr.12135]
16. Blasco-Arcas, L.; Buil, I.; Hernández-Ortega, B.; Sese, F.J. Using clickers in class. The role of interactivity, active collaborative learning and engagement in learning performance. Comput. Educ.; 2013; 62, pp. 102-110. [DOI: https://dx.doi.org/10.1016/j.compedu.2012.10.019]
17. Nulty, D.; Mitchell, M.; Jeffrey, C.; Henderson, A.; Groves, M. Best Practice Guidelines for use of OSCEs: Maximising value for student learning. Nurse Educ. Today; 2011; 31, pp. 145-151. [DOI: https://dx.doi.org/10.1016/j.nedt.2010.05.006] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/20573427]
18. El-Hussein, M.O.M.; Cronje, J.C. Defining mobile learning in the higher education landscape. J. Educ. Technol. Soc.; 2010; 13, pp. 12-21.
19. Abuatiq, A.; Fike, G.; Davis, C.; Boren, D.; Menke, R.; Alham, A.; Geraldine, F.; Claudia, D.; Denise, B.; Richell, M. E-learning in Nursing: Literature Review. Int. J. Nurs. Educ.; 2017; 9, 81. [DOI: https://dx.doi.org/10.5958/0974-9357.2017.00041.1]
20. Turner, J.L.; Dankoski, M.E. Objective structured clinical exams: A critical review. Fam. Med.; 2008; 40, pp. 574-578. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/18988044]
21. Crawford, J.; Butler-Henderson, K.; Rudolph, J.; Malkawi, B.; Glowatz, M.; Burton, R.; Magni, P.A.; Lam, S. COVID-19: 20 countries’ higher education intra-period digital pedagogy responses. Int. Perspect. Interactions Educ.; 2020; 3, pp. 9-27. [DOI: https://dx.doi.org/10.37074/jalt.2020.3.1.7]
22. Cahapay, M.B. Rethinking Education in the New Normal Post-COVID-19 Era: A Curriculum Studies Perspective. Aquademia Water Environ. Technol.; 2020; 4, ep20018. [DOI: https://dx.doi.org/10.29333/aquademia/8315]
23. Selim, A.A.; Ramadan, F.H.; El-Gueneidy, M.M.; Gaafer, M.M. Using Objective Structured Clinical Examination (OSCE) in undergraduate psychiatric nursing education: Is it reliable and valid?. Nurse Educ. Today; 2012; 32, pp. 283-288. [DOI: https://dx.doi.org/10.1016/j.nedt.2011.04.006]
24. Magno, C. The role of metacognitive skills in developing critical thinking. Metacognition Learn.; 2010; 5, pp. 137-156. [DOI: https://dx.doi.org/10.1007/s11409-010-9054-4]
25. Cheung, T.C.H.; Cheung, H.; Mark, K.P. A Study of the Impact of a Crowd Wisdom Online Learning Community Platform on Student Learning. Proceedings of the PACIS; Chengdu, China, 24–28 June 2014.
26. Institute of Medicine. Health Professions Education: A Bridge to Quality; National Academies Press: Washington, DC, USA, 2003.
27. Pedró, F. The New Millennium Learners: Challenging Our Views on ICT and Learning. 2006; Available online: http://www.oecd.org/education/ceri/38358359.pdf (accessed on 20 November 2020).
28. Ennis, R.H. Critical thinking assessment. Theory Into Pr.; 1993; 32, pp. 179-186. [DOI: https://dx.doi.org/10.1080/00405849309543594]
29. Créquit, P.; Mansouri, G.; Benchoufi, M.; Vivot, A.; Ravaud, P. Mapping of Crowdsourcing in Health: Systematic Review. J. Med. Internet Res.; 2018; 20, e187. [DOI: https://dx.doi.org/10.2196/jmir.9330]
30. Morin, K.H. Nursing education after COVID-19: Same or different?. J. Clin. Nurs.; 2020; 29, pp. 3117-3119. [DOI: https://dx.doi.org/10.1111/jocn.15322]
31. Rytivaara, A. Collaborative classroom management in a co-taught primary school classroom. Int. J. Educ. Res.; 2012; 53, pp. 182-191. [DOI: https://dx.doi.org/10.1016/j.ijer.2012.03.008]
32. Mo, Y.; Deng, L.; Zhang, L.; Lang, Q.; Liao, C.; Wang, N.; Qin, M.; Huang, H. Work stress among Chinese nurses to support Wuhan in fighting against COVID-19 epidemic. J. Nurs. Manag.; 2020; 28, pp. 1002-1009. [DOI: https://dx.doi.org/10.1111/jonm.13014]
33. Doyle, D.; Copeland, H.L.; Bush, D.; Stein, L.; Thompson, S. A course for nurses to handle difficult communication situations. A randomized controlled trial of impact on self-efficacy and performance. Patient Educ. Couns.; 2011; 82, pp. 100-109. [DOI: https://dx.doi.org/10.1016/j.pec.2010.02.013] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/20303230]
34. Vondrick, C.; Patterson, D.; Ramanan, D. Efficiently Scaling up Crowdsourced Video Annotation. Int. J. Comput. Vis.; 2013; 101, pp. 184-204. [DOI: https://dx.doi.org/10.1007/s11263-012-0564-1]
35. Alenezi, H.S.; Faisal, M.H. Utilizing crowdsourcing and machine learning in education: Literature review. Educ. Inf. Technol.; 2020; 25, pp. 2971-2986. [DOI: https://dx.doi.org/10.1007/s10639-020-10102-w]
36. Porcello, D.; Hsi, S. Crowdsourcing and Curating Online Education Resources. Science; 2013; 341, pp. 240-241. [DOI: https://dx.doi.org/10.1126/science.1234722]
37. U.S. Department of Education. Future Ready Learning: Reimagining the Role of Technology in Education; Office of Educational Technology: Washington, DC, USA, 2016; Available online: https://files.eric.ed.gov/fulltext/ED571884.pdf (accessed on 27 June 2020).
38. Institute of Medicine (IOM). The Future of Nursing: Leading Change, Advancing Health; National Academies Press: Washington, DC, USA, 2010.
39. Shaikh, U.U.; Karim, S.; Asif, Z. Re-Thinking Vygotsky: Applying Social Constructivism to Asynchronous Online Courses utilizing the Power of Crowdsourcing. Proceedings of the 21st Pacific Asia Conference on Information Systems; Langkawi, Malaysia, 16–20 July 2017.
40. Snodgrass, S.J.; Ashby, S.E.; Rivett, D.A.; Russell, T. Implementation of an electronic Objective Structured Clinical Exam for assessing practical skills in pre-professional physiotherapy and occupational therapy programs: Examiner and course coordinator perspectives. Australas. J. Educ. Technol.; 2014; 30, pp. 152-166. [DOI: https://dx.doi.org/10.14742/ajet.348]
41. Farasat, A.; Nikolaev, A.; Miller, S.; Gopalsamy, R. Crowdlearning: Towards collaborative problem-posing at scale. Proceedings of the Fourth ACM Conference on Learning Scale; ACM Press: Cambridge, MA, USA, 2017; pp. 221-224.
42. Razzak, N.A. Strategies for effective faculty involvement in online activities aimed at promoting critical thinking and deep learning. Educ. Inf. Technol.; 2016; 21, pp. 881-896. [DOI: https://dx.doi.org/10.1007/s10639-014-9359-z]
43. Machado, L.; Zanatta, A.; Marczack, S.; Prikladnicki, R. The Good, the Bad and the Ugly: An Onboard Journey in Software Crowdsourcing Competitive Model. Proceedings of the 2017 IEEE/ACM 4th International Workshop on Crowdsourcing in Software Engineering (CSI-SE); Buenos Aires, Argentina, 22 May 2017; pp. 2-8. [DOI: https://dx.doi.org/10.1109/csi-se.2017.6]
44. Raurell-Torredà, M.; Romero-Collado, A.; Bonmatí-Tomàs, A.; Olivet-Pujol, J.; Baltasar-Bagué, A.; Sola-Pola, M.; Mateu-Figueras, G. Objective Structured Clinical Examination: An Assessment Method for Academic-Practice Partnerships. Clin. Simul. Nurs.; 2018; 19, pp. 8-16. [DOI: https://dx.doi.org/10.1016/j.ecns.2017.11.001]
45. Ahlin, C.; Klang-Söderkvist, B.; Johansson, E.; Björkholm, M.; Löfmark, A. Assessing nursing students’ knowledge and skills in performing venepuncture and inserting peripheral venous catheters. Nurse Educ. Pr.; 2017; 23, pp. 8-14. [DOI: https://dx.doi.org/10.1016/j.nepr.2017.01.003] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28171853]
46. Bakator, M.; Radosav, D. Managing education in the COVID-19 era. BMJ; 2020; 370, [DOI: https://dx.doi.org/10.1136/bmj.m3349]
47. Baron, K.A. Changing to Concept-Based Curricula: The Process for Nurse Educators. Open Nurs. J.; 2017; 11, pp. 277-287. [DOI: https://dx.doi.org/10.2174/1874434601711010277] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/29399236]
48. Bartfay, W.J.; Rombough, R.; Howse, E.; LeBlanc, R. The OSCE approach in nursing education: Objective structured clinical examinations can be effective vehicles for nursing education and practice by promoting the mastery of clinical skills and decision-making in controlled and safe learning environments. Can. Nurse; 2004; 100, pp. 18-233.
49. Rushforth, H.E. Objective structured clinical examination (OSCE): Review of literature and implications for nursing education. Nurse Educ. Today; 2007; 27, pp. 481-490. [DOI: https://dx.doi.org/10.1016/j.nedt.2006.08.009]
50. Corneli, J.; Mikroyannidis, A. Crowdsourcing education on the Web: A role-based analysis of online learning communities. Collab. Learn. 2.0 Open Educ. Res.; 2012; pp. 272-286. [DOI: https://dx.doi.org/10.4018/978-1-4666-0300-4.ch015]
51. Jackson, D.; Bradbury-Jones, C.; Baptiste, D.; Gelling, L.; Morin, K.; Neville, S.; Smith, G.D. Life in the pandemic: Some reflections on nursing in the context of COVID-19. J. Clin. Nurs.; 2020; 29, pp. 2041-2043. [DOI: https://dx.doi.org/10.1111/jocn.15257]
52. Karataev, E.; Zadorozhny, V. Adaptive Social Learning Based on Crowdsourcing. IEEE Trans. Learn. Technol.; 2016; 10, pp. 128-139. [DOI: https://dx.doi.org/10.1109/TLT.2016.2515097]
53. Ghanizadeh, A. The interplay between reflective thinking, critical thinking, self-monitoring, and academic achievement in higher education. High. Educ.; 2017; 74, pp. 101-114. [DOI: https://dx.doi.org/10.1007/s10734-016-0031-y]
54. Zhang, J.; Cui, Q. Collaborative learning in higher nursing education: A systematic review. J. Prof. Nurs.; 2018; 34, pp. 378-388. [DOI: https://dx.doi.org/10.1016/j.profnurs.2018.07.007] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30243695]
55. Liu, C.C.; Lu, K.H.; Wu, L.Y.; Tsai, C.C. The impact of peer review on creative self-efficacy and learning performance in Web 2.0 learning activities. J. Educ. Technol. Soc.; 2016; 19, pp. 286-297.
56. Baumer, P.S.E. Reflective informatics: Conceptual dimensions for designing technologies of reflection. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems; Seoul, Korea, 18–23 April 2015; pp. 585-594.
57. Lu, J.; Xin, Y.; Zhang, Z.; Liu, X.; Li, K. Game-Theoretic Design of Optimal Two-Sided Rating Protocols for Service Exchange Dilemma in Crowdsourcing. IEEE Trans. Inf. Forensics Secur.; 2018; 13, pp. 2801-2815. [DOI: https://dx.doi.org/10.1109/TIFS.2018.2834318]
58. Wang, K.; Nickerson, J.; Sakamoto, Y. Crowdsourced idea generation: The effect of exposure to an original idea. Creat. Innov. Manag.; 2018; 27, pp. 196-208. [DOI: https://dx.doi.org/10.1111/caim.12264]
59. Prilla, M.; Blunk, O.; Chounta, I.-A. How Does Collaborative Reflection Unfold in Online Communities? An Analysis of Two Data Sets. Comput. Support. Coop. Work. (CSCW); 2020; 29, pp. 697-741. [DOI: https://dx.doi.org/10.1007/s10606-020-09382-0]
60. Dewart, G.; Corcoran, L.; Thirsk, L.; Petrovic, K. Nursing education in a pandemic: Academic challenges in response to COVID-19. Nurse Educ. Today; 2020; 92, 104471. [DOI: https://dx.doi.org/10.1016/j.nedt.2020.104471]
61. Forbes, M.; Hickey, M.T. Curriculum Reform in Baccalaureate Nursing Education: Review of the Literature. Int. J. Nurs. Educ. Sch.; 2009; 6, 27. [DOI: https://dx.doi.org/10.2202/1548-923X.1797]
62. Pintrich, P.R. A Manual for the Use of the Motivated Strategies for Learning Questionnaire (MSLQ). 1991; Available online: https://files.eric.ed.gov/fulltext/ED338122.pdf (accessed on 27 June 2020).
63. Park, S.Y. An analysis of the technology acceptance model in understanding university students’ behavioral intention to use e-learning. Educ. Technol. Soc.; 2009; 12, pp. 150-162.
64. Richardson, J.T.E. Methodological Issues in Questionnaire-Based Research on Student Learning in Higher Education. Educ. Psychol. Rev.; 2004; 16, pp. 347-358. [DOI: https://dx.doi.org/10.1007/s10648-004-0004-z]
65. Manoochehri, H.; Imani, E.; Atashzadeh-Shoorideh, F.; Alavi-Majd, A. Competence of novice nurses: Role of clinical work during studying. J. Med. Life; 2015; 8, pp. 32-38.
66. McLoughlin, C.; Lee, M.J.W. Personalised and self regulated learning in the Web 2.0 era: International exemplars of innovative pedagogy using social software. Australas. J. Educ. Technol.; 2010; 26, pp. 28-43. [DOI: https://dx.doi.org/10.14742/ajet.1100]
67. McWilliam, P.L.; Botwinski, C.A. Identifying strengths and weaknesses in the utilization of objective structured clinical examination (OSCE) in nursing program. Nurs. Educ. Perspect.; 2012; 33, pp. 35-39. [DOI: https://dx.doi.org/10.5480/1536-5026-33.1.35]
68. National League for Nursing. Accreditation Standards for Nursing Education Programs. 2016; Available online: http://www.nln.org/docs/default-source/accreditation-services/cnea-standards-final-february-201613f2bf5c78366c709642ff00005f0421.pdf?sfvrsn=12 (accessed on 27 June 2020).
69. Ng, Y.-M.; Or, P.L.P. Coronavirus disease (COVID-19) prevention: Virtual classroom education for hand hygiene. Nurse Educ. Pr.; 2020; 45, 102782. [DOI: https://dx.doi.org/10.1016/j.nepr.2020.102782] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32388117]
70. Pantazopoulos, I.; Tsoni, A.; Kouskouni, E.; Papadimitriou, L.; Johnson, E.O.; Xanthos, T. Factors influencing nurses’ decisions to activate medical emergency teams. J. Clin. Nurs.; 2012; 21, pp. 2668-2678. [DOI: https://dx.doi.org/10.1111/j.1365-2702.2012.04080.x] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/22889450]
71. Prilla, M.; Viktoria, P.; Birgit, K. Fostering collaborative redesign of work practice: Challenges for tools supporting reflection at work. Proceedings of the Computer Supported Cooperative Work; Springer: London, UK, 2013; pp. 249-268. [DOI: https://dx.doi.org/10.1007/978-1-4471-5346-7_13]
72. Weston, J. The clinical instructor program: Improving self-efficacy for nurse educators. Nurse Educ.; 2018; 43, pp. 158-161. [DOI: https://dx.doi.org/10.1097/NNE.0000000000000452]
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
1 Library and Information Center, Tajen University, Pingtung County 90741, Taiwan;
2 Department of Nursing, Tajen University, Pingtung County 90741, Taiwan;
3 Department of Engineering Science, National Cheng Kung University, Tainan City 701, Taiwan