An Analysis of the Demonstration of Five-Year-Long Creative ICT Education Based on a Hyper-Blended Practical Model in the Era of Intelligent Information Technologies

Abstract

In anticipation of the advanced information and communication technology (ICT) era’s profound impact on society and industry, the demand for strong technological comprehension and creativity is evident. Addressing this societal need, we have developed a hyper-blended educational model and program. It focuses on cultivating creative thinking and ICT understanding, tailored for elementary/middle school students, educators, administrators, and the public. Our program employs innovative pedagogical methods, encompassing subjects such as artificial intelligence (AI) ethics, blockchain, and cloud computing, providing forward-looking perspectives. With techniques such as Havruta dialogue and quantum learning, we promote engaging, convergent thinking. We blend artistic expression, physical activities, and technical education to encourage administrators’ creativity within educational settings. Over 2017–2021, involving 9596 participants, we observed consistently high satisfaction scores of around 4 on a 1–5 scale, with statistical significance. This underscores our program’s success in fostering creative thinking and ICT understanding across diverse domains. In conclusion, our study addresses the vital need for creativity and tech comprehension in an intricate technological landscape. Our hyper blended practical model (HPBM) serves diverse training groups, equipping them for future challenges. These findings guide future educational pursuits, emphasizing a seamless integration of creativity and tech understanding.

1. Introduction

Modern men and women living in the 21st century across the world agree that we have now entered the era of the Fourth Industrial Revolution, during which we need to coexist with AI [1]. Moreover, in the Republic of Korea, where 95% of the total population uses smartphones and various ICTs converge, the smartphone penetration rate ranks highest in a survey spanning 27 countries, including the United States, the United Kingdom, France, and Japan [2]. A significant number of individuals in the country engage with ICT for most of their day, starting from the moment they wake up until they go to bed in the evening, and even during sleep at night. Furthermore, the COVID-19 pandemic has brought many work routines, educational activities and art performances conducted offline in the past to online spheres, leading more and more industries to rely on digital technologies and accelerating the pace of digital transition across all industries. Of course, there are hundreds of countries on this planet, and not all countries are equal when it comes to digitization [2]. However, the fact of the matter is that all countries are moving along the same path and none of them will be immune to the influences of AI technology and cutting-edge ICTs. AI is growing ever stronger thanks to machine-learning and deep-learning technologies, and we will witness the emergence of so-called artificial general intelligence (AGI) capable of performing all intelligent tasks that can be performed by human beings up to now, beyond the constraints of the narrow AI solutions available to date, and perhaps even super AI that surpasses human intelligence [3].

To avoid being dominated by technology but to create or utilize it amid the rapidly changing technological landscape, we need to have creativity, in addition to technological understanding [4,5]. Creativity has been defined in various ways by experts on different subject matters. The studies on human creativity in the 1970s to 80s when modern research on the notion of creativity started, defined creativity as a process of transforming existing things into something unique. Outcomes of such a process can be tangible or intangible, and they must be unique, made by their creators. In addition, creative outcomes must serve the purpose and value specified by the creators [6]. When a digital transition is occurring in all industries, creative thinking competency is deemed an essential element that needs to be learned and taught [7]. In an intelligent information society characterized by complexity and diversity, jobs performing repetitive and straightforward routines will disappear. Therefore, we need to foster talents who can create new things by combining knowledge, information and technology with human sensibility, which machines cannot match.

Understanding technology is also a critical competency, next to creativity in preparing for a future society. Someone who can understand and utilize technologies in an intelligent information society can lead any technology-based industry [8]. Technical education (TE) has been defined differently in terms of its meaning and scope along the historical path of technological evolution. When technical art relied exclusively on human hands, manual arts were regarded as the subject of technical education, whereas as mankind entered the Industrial Age, technical education refers to industrial education that teaches about the technologies required for industrial development. Then, as the significance of science and technology became prominent after the launch of Russia’s artificial satellite, technical education covering science and technology became popularized [9]. The scope of technical education has been further expanded recently to cover ICTs, including AI. Technological understanding is required for specialists and engineers to be mobilized at industrial sites, but it is also appreciated as an element of the literacy required for the entire population, as ICT is now intertwined with all our daily routines.

Technical education operated in the public schooling program of Korea divides into ‘Practical Art’, ‘Technology & Home Economics’, and ‘Information Technology’. ‘Practical Art’ is taught in the 5 to 6th elementary school grades; ‘Technology & Home Economics’ and ‘Information Technology’ in middle schools; and ‘Information Technology’ in high schools. The practical art and technology & home economics classes teach the development of human beings, family, home economics and safety, resource management and self-sufficiency, and technical systems. ICT is taught as a sub-topic of communications technology in the technical systems. Information culture, data and information, problem solving and programing, and computing systems are taught in the information technology class. These concepts belong to common education courses, and more in-depth concepts can be learned in selective education courses [10]. However, the comparatively weaker focus on cutting-edge intelligent information technologies matters. In elementary schools, only 17 h out of 2 years are allocated to information technology education, and it 34 h in middle schools. In high schools, information technology is offered as one of the selective classes. Accordingly, many ICT education experts criticize the insufficient education hours, which are not enough to cover the needs of an intelligent information society and the AI era may compromise national competitiveness and result in a digital divide [11]. Let alone the issue of less-than-sufficient education hours, rather than simply encompassing ICT, technical education needs to be designed from now on to foster creativity [12]. In so doing, technical prowess centered around creativity as opposed to simple technical competency must be promoted, and therefore educational curricula need to be reshuffled to provide a creative perspective towards technologies, as enabled by critical and convergent thinking.

Creativity and technical education are essential elements for building the society, not just for today but also tomorrow. Nevertheless, most studies to date have addressed the two elements separately. Even where they were dealt with in combination, most studies just focused on so-called ed-tech, i.e., technologies employed in improving educational effectiveness. Even though STEM education converging science, technology, engineering and mathematics is conducted worldwide, it is not directly focused on cutting-edge technologies for the future, or on creativity. Therefore, education programs targeting not just students but also a more diverse spectrum of beneficiaries are not sufficiently studied. On top of that, the education system of Korea is not designed in a way to squeeze creativity education into the relatively few technical education hours. Hence, the Creative education center at Jeju National University has conducted relevant studies since 2017, thinking that technical competency-based creativity is an essential element for the capabilities required in the society of the future.

This paper describes the education program developed by the Creative education center at Jeju National University from 2017 to 2021 and contains the analysis results for the program’s execution. Section 1 provides the overview of the study, covering its purpose and necessity. Section 2 introduces other studies relevant to technical education and an education model for fostering creativity. Section 3 proposes the hyper-blended practical model and the contents of education programs based on the model, broken down into years and statuses. Section 4 describes how the developed education program is applied to the intended training targets and Section 5 provides analysis results for the execution of the education program. Section 6 contains a detailed description of the research results and the limitations of the study. Section 7 concludes the paper by presenting a plan and implications for further studies. All in all, the purpose of this paper is to propose topics and implementation strategies for a creative technical education model and education program and to analyze the execution results thereof at education sites, in a bid to contribute to fostering talents well versed in convergent technologies who can understand ICT and utilize it creatively in future society.

2. Related Background Research

2.1. Technical Education Fostering Creativity

The origin of creative technical education dates back to the United States of the 1940s. Industrial arts was added to the educational curriculum back then and ‘Mathematics, Science, and Technology’ (MTS) education was at the heart of the decision [13]. The US government that conceived the idea claimed that mathematics, science and technology needed to be taught as integrated in a multi-disciplinary way and believed that this integrated education could help students better adapt and respond to the complex environment of the 21st century. Then, as the focus of educational policy shifted toward academics in the 1960–1970s, science educators argued that the then-in-force education system, focused on academic knowledge, was detached from the need to employ science in addressing social challenges. Accordingly, ‘Science, Technology, and Social Studies’ (STS) education to creatively utilize scientific and technical knowledge in addressing social challenges was conducted [14]. STS education was designed to develop members of the society to be more capable of utilizing science and technology. Yet the MTS and STS education policies withered away in the midst of an educational trend emphasizing subject-centered and segmented knowledge and understanding [15]. Science, Technology, Engineering, and Mathematics (STEM) education that recently provided a foundation for convergent and creative technical education is deemed to have been inspired by the US economic crisis that erupted abruptly after the 2000s and the need to reduce anxiety over a potential lack of high-quality manpower. STEM combines existing MST education with the concept of ‘Engineering’. This seems to be an education policy reflective of the social landscape, built on an engineering-centric industrial structure emphasizing information technology, aerospace and robotics, etc. Similar to STEM education are the MINT education of Germany (Mathematik (Mathematics), Informatik (Computer Science), Naturwissenschaften (Natural Science), and Technik (Technology)). This German education policy was proposed to allow students to develop a variety of competencies in a bid to foster quality professionals equipped with the creativity and skillsets required by the German society of the future. Education, research, economic, political and industrial communities all join forces in actively supporting the MINT education and competent authorities provide a wide range of programs ranging from K-12 to field training [16]. STEAM (Science, Technology, Engineering, Arts, and Mathematics) education, where ‘Arts’ is added to STEM education, was born out of the criticism that STEM contributed to the intelligent growth of students, but did not help them develop emotional sensibility and social character. The purpose of STEAM education is to ensure that students attempt to approach social challenges with scientific, technical, engineering and mathematical knowledge and from the perspective of liberal arts, in order to address social issues and deliver humanistic and creative technical solutions [17].

The Republic of Korea (“Korea” hereinafter) also adopted STEAM as one of its education policies in 2010 at the initiative of the Ministry of Education and the Korea Foundation for the Advancement of Science and Creativity (KOFAC) to nurture creative and convergent talents who can spearhead Korea’s progressive development in science and technology [18]. The teaching and learning framework of the Korean STEAM education consists of three elements: ‘presentation of situation’ relating to daily life issues; ‘creative design’ that enables students to explore problem-solving directions; and ‘emotional experience’, including interest, focus, and challenging spirit, concerning experience of class topics by students during the learning process [19]. The Korean STEAM is based on science and technology learning and intended to encourage students to develop more interest in and understanding of science and technology, and to promote convergent thinking and competencies required for solving daily life issues. The KOFAC has selected pioneering schools and is deploying makerspaces where creative ideas can be implemented. Furthermore, the KOFAC provides training programs in STEAM education to schoolteachers and helps teachers to organize teachers’ research societies so that they can upgrade their teaching skills for creative and convergent technologies [20]. In addition, the KOFAC operates creative education centers in selected universities across the country. The centers study a wide range of creativity education programs and models, and distribute online, free of charge, a collection of best cases compiled from the creativity education programs based on such studies. Most creativity education programs are developed to utilize AI technologies or online platforms in education for each subject or deliver class teaching programs, adopting creative teaching methods [21]. However, no education program that teaches technology itself as educational content has been seen other than the product from the Creative education center at Jeju National University, as affiliation with prestigious schools still plays a significant role in Korean society and the prestige of a university is deemed to determine the future of its graduates. Accordingly, although the education system of Korea works hard incessantly to produce creative talents, education is dominated by focus on university admission tests, and education in technology and creativity not directly related to university admission tests is increasingly sidelined in the upper grades of high school [22]. Teachers at school point out that such an education policy is compromised by operational challenges. They emphasize that they do not have enough time to operate creative technology education programs and suffer excessive workload accordingly. Furthermore, some point out that creative technology education makes it hard to comply with the planned class teaching schedule [23]. Therefore, given the prevailing circumstances, creative technology education in Korea needs to be designed to require less time and effort from teachers and enable students to develop creativity and technical understanding more efficiently. In addition, a wide range of educational courses, including programs designed to nurture the creative technology teaching skills of teachers as well as students and informing the general public of the significance of improved creative and technical understanding, should be developed.

2.2. Education Model Relating to Creativity and Convergence

As technical education for creative and convergent thinking is becoming increasingly important, relevant education models are being studied and developed continuously. First of all, in the United States where STEM education was intensively promoted, the Support, Teaching, Efficacy, Materials (s.t.e.m) model for STEM education, containing considerations to be made in connection with integrated STEM education, was developed [24]. This model suggests support for STEM education, elements not to be missed in course design and classroom activities, as well as considerations to be made for improved efficacy. According to the model, to deliver STEM education in its truest form, collaboration with universities and research institutions is needed near the schools where STEM education is conducted. When designing a class, teachers must understand the competency of learners, teach with focus on learners, and emphasize the importance of problem-solving skills having direct bearing on real issues. Furthermore, it is desirable to reflect on applicable topics and encourage collaboration with classmates in class. Teacher’s commitment to STEM education is essential for its efficacy and fostering positive self-efficacy requires educational contents and pedagogical knowledge. Lastly, the model suggested the need for an extensive perspective toward technology, materials kits for class activities, and physical space for such activities.

As for the STEAM education model, where the perspective of liberal arts is added to STEM education, the STEAM Pyramid developed in the United States is an iconic example [25]. This model is pyramid-shaped and consists of of five different sections. The lowest section is ‘Content Specific’, where science, technology, engineering, mathematics, and liberal arts are studied in depth. Students are supposed to study in a team of peers engaged in the same field or collaborate with other teams studying different fields. This section is suitable for tertiary education where strong focus is required. Another section is ‘Discipline Specific’, where fields to be intensively learned are specified, but other fields can also be learned. These other fields are utilized as assistive subjects to help with the understanding of the focused field. This section is deemed to be most appropriate for secondary education. Above this section is the ‘Multidisciplinary’ section, where learners are taught how specific academic fields selected by them interface with other fields. In this section, teachers propose that students learn on the basis of real issues, and warn that their connection with the reality may be compromised unless they learn about liberal arts and other fields in combination, implying the importance of integrating education with liberal arts,. This section is suitable for middle school education. The fourth section is ‘Integrative’ where learning encompasses all fields. Teachers in charge of this section need to collaborate with experts in education for each academic discipline and learners should try to become familiar with inter-disciplinary relations. This section is proposed to be applicable to K-12 education and all other education levels. The last section is ‘Life-long Holistic’, linked with a holistic perspective. In this section, science, technology, engineering, mathematics and arts are not viewed from different perspectives, respectively, but approached from a holistic perspective. As each learner is likely to have a different perspective on integrated concepts in the last section, all learners may not understand it uniformly.

Meanwhile, as IT has significantly evolved into an academic discipline of its own, a standard IT education model curriculum has been developed [26]. This model breaks down IT education into Information Systems Fundamentals, Information Systems Theory and Practice, Information Technology, Information System Development, and Information Systems Development and Management Process. In the realm of IT education, a distinction can be drawn between fields that utilize Information Technology (IT) as a tool or medium within education and those that directly teach IT as a subject matter. The former encompasses instances where technologies such as blockchain or artificial intelligence are integrated into educational contexts [27,28,29], while the latter involves educating students about the principles underlying technologies, such as blockchain or artificial intelligence [30,31]. This paper primarily addresses the latter form of IT education. In addition, the significance of creativity is revealed in various forms within the education curriculum proposed by this model. Patience, curiosity, creativity, risk-taking, and perseverance are suggested as qualities to be developed by students taking the education curriculum. It is emphasized that not only their understanding of IT and relevant knowledge but also communication and teamwork with peers in learning are important. As when E-learning and online learning started, a blended learning model that allows IT education to be conducted in both on and offline class environment was also suggested [32]. The model developers pointed out the issues of traditional unilateral lecture-style teaching practices dictated by teachers and predicted that a curriculum system utilizing multimedia or 3D animation would enable interactive education free from spatial constraints. As IT knowledge to be covered in this education, basic IT theories and culture, trends in IT development, digital image processing, programming and problem-solving, database technologies and methods were named, and it was proposed that culture, applied technologies and innovations in various IT technologies be learned creatively through diverse systems. However, the two education models for IT education [26,32] were developed to target students in undergraduate degree programs. With IT education becoming increasingly important, needs for developing IT and ICT competencies from childhood were recently pointed out, which led to the development of education models designed to teach elementary students ICT competencies [33]. These models are designed to contain six different procedures (problem statement, relevant information retrieval, solution design, prototype development, pilot test, and presentation) to help students to develop such competencies as information access, data gathering, information visualization, software use, and product creation. These models use social media favored by students, such as Facebook or Line, as class tools to allow students to engage in classes more attentively. The education model regarded as the progenitor of creative ICT education models for elementary/secondary students is the Creative Plugged Education Model (CPEM). The CPEM marks the beginning of a creative education program development model using ICT and it was developed with a view towards developing future-oriented talents, possessing not only ICT competencies but also sensibility and creativity [34]. This model is focused on class innovation and strategies to develop a wide range of educational contents that can be designed in classes.

3. Hyper Blended Practical Model

3.1. Development Background

As a variety of information technologies, including Internet of Things for data collection, cloud for data accumulation, big data for data analytics, and mobile network for data transfer, to name a few, has developed, artificial intelligence (AI) is now interwoven with information, and ICT has driven the Fourth Industrial Revolution [35]. Accordingly, we determined that it is important for not only students but also many other stakeholders, including teachers, school administrators, and the general public, to understand ICT and develop creativity. To be more specific, to ensure that elementary and secondary students become familiar with cutting-edge ICT trends and principles not covered in the public schooling system and to develop creativity at the same time, not only teachers, but also school administrators, parents, and the general public must all be educated. In addition, a blended education program that supports both on- and offline education in response to changes in educational landscape is required. Yet preceding studies focused more on specific training targets or education curriculums rather than an integrated practical model encompassing a variety of educational audiences and environments [24,25,26,27,28,29,30,31,32]. The CPEM was developed for various learner groups, including students and faculty members, but was not designed to cater for an online education environment, which necessitated an additional teaching strategy for a blended learning environment [34]. Therefore, the HBPM is designed to combine the CPEM and the Blended and Distributed Learning Approach (BDLA) of Hyper Island [35]. The BDLA is an approach to blended learning proposed by a marketing school in Sweden known as the ‘Digital Harvard’. They focused on the experience of students in line with constructivism and encouraged online team collaboration activities. To that end, the BDLA supports secure communication tools and allowing various data to be stored in a trusted archive [36]. This enables interactive communication between students, and between students and teachers, in combination with various activities, rather than delivering unilateral online classes. The HBPM is an integrated creative technical education model based on the class innovation strategy of the CPEM and combined with the BDLA.

3.2. Configuration of the Proposed Education Model

The HBPM is a technical education model intended to help learners develop creativity [35]. This model is designed to allow for an integrated and practical strategy to provide innovative technical education to various educational audiences beyond class education. In particular, it is configured to focus on on-site applicability to ensure that creative technical education can take root effectively at education sites. This model consists of the vision, objective, tasks, major features, and practical strategy.

First of all, the vision of the model is to foster creative technical education at national level and its corresponding objective focuses on developing creative and convergent technical talents in the era of intelligence and information. Tasks roughly defined are research, contents development, faculty competency reinforcement, and performance diffusion. Continuous research is called for in developing creative technical talents and it is important to develop educational content that can be utilized in classes. Contents to be developed need to be designed to deliver more accurate information and timely content in research partnership with specialized entities in Korea and elsewhere around the world. In addition, education programs and materials for faculty members need to be available so that they can utilize the developed educational content in classes more effectively. Furthermore, education programs in the form of ‘talk concerts’ or forums targeting the general public should be configured to ensure that research outcomes can be communicated to wider audiences. The major features of this model include flexible adaptability to on- and offline class environments and emphasis on realistic media utilization coupled with dynamic activities. In addition, the model is noted for hyper-personalized customizable learning utilizing big data-based AI technology and developing and innovating an assessment system for nurturing creative thinking. It is also proposed that classes be designed to follow a cyclical structure of ‘Ready-Set-Go-Review’ adapted from the SAM (Successive Approximations Mode), developed to overcome the drawbacks of the ADDIE (Analyze-Design-Develop-Implement-Evaluate) model frequently used in class designs. As a strategy to implement the model, it is suggested that creative teaching methods and technical education contents be studied and textbooks developed based on such studies. Developed programs can be applied on a pilot basis in creative factories operated in partnership with faculty members and the feasibility of the education program can be analyzed accordingly. Faculty members and school administrators will also be trained in the developed educational contents to facilitate their adoption at education sites. This will be made possible by partnership networking with relevant authorities. Accordingly, this study has developed a creative technical education program based on the HBPM, which is one of the representative creative technical education models. Figure 1 shows the visualization of the HBPM.

3.3. Development of New Creative ICT Education Program Based on HBPM

The creative technical education program proposed herein has been developed to help students to improve understanding of ICT and creativity in the era of the Fourth Industrial Revolution. The program proposed herein covers education for elementary/secondary students, faculty members, school administrators, and the general public in terms of these targeted educational audiences. The education program targeted at such audiences was conducted every year from 2017 to 2021. The education program is based on the basic framework of the hyper blended practical model, but its sub-programs may vary, depending on the development schedule or prevailing social conditions. All education programs were developed in collaboration with doctoral students in computer education and computer engineering departments, teachers, education experts within competent authorities, and 10 to 15 technology experts. If the number of participating experts is 10, applicable educational content can be deemed to be feasible only when the Content Validity Ratio (CVR) is at or above the threshold of 0.62. Therefore, to be more conservative, 0.70 was specified as the minimum threshold and educational content whose CVR value was at or below 0.70 was either modified or excluded [37].

Topics of the education program developed for students in 2017 were four in total: ‘human body communications technology’, ‘cutting-edge realistic media’, ‘AI of the future and ethics’, and ‘cloud of the future and security’. The program consisted of six lesson hours in total and textbooks were made for each topic and distributed to teachers who would participate in the program. The faculty member training program covered ‘creative skills in the knowledge and information era of the 21st century’, ‘key ICT principles and creativity education in the era of intelligence and information’, ‘reconfiguration of education curriculum through innovative and creative teaching/learning methods and creative technical education strategy’. The training program for school administrators consists of ‘creative personality education via Gotjawal tour in Jeju’, ‘ICT-enabled innovation-type creative technical education’, and ‘best cases of creative technical education’. Meanwhile, the education program for the general public encompasses an extensive range of topics, including creative ICT with interface to ecology education and pursuit of happiness in school education while preparing for the intelligence and information society, etc.

In 2018, an education program targeting elementary/secondary students was designed, covering ‘principles of blockchain’, ‘big data/AI liberal arts’, ‘principles of information security’, and ‘space radiation disaster and safety’. As specific contents of the faculty member training, key competencies of innovative creative technical education for the Fourth Industrial Revolution, a creative technical education class model and case studies were covered. Specific contents of the program for school administrators included ‘AI and creativity of the future’, ‘future hyper-connectivity technology education applying Havruta teaching method’ and ‘creative technical education in international baccalaureate (IB) education’, etc. Furthermore, the program for the general public covered changing education policies and implementation strategies, ranging from AI education taught via visual programming to understanding of the principles underpinning ICT.

In 2019, ‘blockchain application’, ‘liberal arts of big data/AI liberal arts application’, ‘hyper-trust of the future, authentication security’, and ‘risk communication’ were selected as topics for the education of elementary/secondary students. For faculty members, IB education courses and evaluation methods were taught to suggest an innovative evaluation paradigm for creative technical education and foster understanding of the creative technology class design approach, creative teaching techniques, and community collaboration for effective deployment of creative technical education at education sites. The training for school administrators covered key ICT principles and concepts in detail to help trainees better understand ICT and wake up to its importance. For the general public, IT-enabled career development courses were conducted along with camp programs providing various activities in AI and cybersecurity, etc., to make these seemingly strange technologies more accessible to the general public.

In 2020, as the topics for the elementary/secondary students education program, ‘cutting-edge technologies in our life’, ‘creative technical education converging digital technologies and liberal arts’, ‘history of information security techniques’, and ‘mega science’ were selected. For faulty members, several best cases of technical education that could be referenced in classes were provided, and for school administrators, creative intelligence and information education befitting the New-Normal era following the outbreak of COVID-19 and unplugged and online worksheet activities focused on understanding of AI concepts including neural network, guided learning, etc., and blockchain forgery/alteration prevention principles were conducted. In addition, for the general public, programs to foster understanding of technical education aligned with the Digital New Deal policy announced by the Korean government in 2020 were provided in combination with interesting case studies in online creative technical education.

In 2021, ‘AI humanities’, ‘forensic science’, and ‘digital therapeutics’ were selected as overarching topics of the education program for elementary/secondary students and a comprehensive creative technical education program encompassing 14 sub-topics was developed. Faulty members were trained in how to effectively adapt creative teaching techniques to the education program for elementary/secondary students at education sites. School administrators were taught how to provide creative technical education in schools, using a metaverse platform, and to utilize AI learning management systems so that they could deploy ed-tech in schools. Last, but not least, the program for the general public taught what constitutes a genuine creative ICT education in a digital era based on data, networks and AI (DNA), and how creative technical education is conducted in the United States and Laos, in order to ensure that they understand the criticality of technical education. Table 1 lists the major contents of the creative technical education program proposed herein.

4. Methodology

4.1. Research Procedure

This paper proposes an education model, develops an education program based on the model and analyzes the application results of the program to identify relevant implications. The development and performance of the education program was studied in three phases: firstly, in the ‘Design’ phase, literature on creativity, creativity education, technical education, creative technical education, education model, etc., were studied comprehensively. Then, training targets were selected. The education program was conceived in accordance with the HBPM and a variety of teaching/learning methods were considered in designing the program in a way to best nurture the creativity of students. Education experts and ICT specialists were engaged in developing the education program.

In the second ‘Development & Application’ phase, education programs appropriate for selected training targets were developed. Based on the education contents primarily planned in the ‘Design’ phase, education programs were developed in consideration of the level of training targets, educational background, environment, lesson hours, etc. Feasibility of the programs then developed was studied by experts from relevant entities identified in the partnership network. After the feasibility study, education content associated with low CVI values was deleted to further upgrade the education programs. Then, the developed programs were conducted continuously on training targets from 2017 to 2021 and satisfaction level survey was conducted after the program implementation.

In the third ‘Application Results Analysis’, the results of the satisfaction level surveys conducted for five years were comparatively analyzed. First of all, mean satisfaction levels per year across sub elements of the training targets were calculated, equated to compare satisfaction scores for multiple years on the same basis, and compared across years and training target statuses. Further implications were identified from the analysis of the results across the five years.

The first and second phases are covered in Section 2 and Section 3, whereas the third phase is addressed in Section 5. The procedures of this study in these three phases are as shown in Figure 2.

4.2. Study Tools

This study was conducted as funded by the Korean Ministry of Education and the KOFAC. Therefore, satisfaction survey tools uniformly distributed to centers by the KOFAC were used in this study [38]. We adapted the distributed tools to applicable training targets and programs. The study tools were broken down into tools for students and adults (faculty members, administrators, general public). In acceptance of expert feedback that the tool specific to students that was used in 2017 contained rather too many and duplicated questions, which compromised its applicability, questions in the survey tool were streamlined in 2018 and the same tool was used from 2018 to 2020. However, in 2021, the KOFAC released a revised version of the previous satisfaction survey tool for students, and we used this revised satisfaction survey tool. Meanwhile, the same survey tool for adults was used from 2017 to 2021. Therefore, four types of survey tools were used in this study.

Firstly, the satisfaction survey tool used for students in 2017 covered ‘overall satisfaction with lessons’, ‘level of interest’, ‘level of engagement’, ‘appropriateness of contents’, ‘difference from existing lessons’, ‘challenging points’, ‘commitment to further education’, ‘detailed satisfaction level with lessons’, ‘interesting topic or activity’, ‘activity or contents desired to be added’, and ‘pros and cons of the lessons’. The satisfaction level then surveyed back overlapped somewhat with the awareness survey, and this study used only ‘overall satisfaction’, ‘level of interest’, ‘appropriateness of contents’, ‘commitment to further education’, and ‘detailed satisfaction level with lessons’ in the analysis. These elements were designed to be evaluated on a five-point Likert scale. The education program satisfaction level survey tool used for students in 2017 is shown in Appendix A.

Secondly, the satisfaction survey tool used for students from 2018 to 2020 contained fewer questions than that used in 2017, facilitating the survey progress. The survey was broken down into five elements, all evaluated on a five-point Likert scale. The elements covered by the tool included ‘overall satisfaction’ with lessons, ‘level of interest’, ‘level of engagement’’ ‘appropriateness of contents’, and ‘commitment to further education’. This survey tool is shown in Appendix B.

Thirdly, the satisfaction survey tool used in 2021 covered seven elements, including ‘overall satisfaction’, ‘level of interest’, ‘depth of learning’, ‘teacher’s guidance’, ‘opportunity for communication between teachers and students’, ‘improvement of problem-solving skills following education’, and ‘commitment to further education’. This survey tool is described in Appendix C.

Fourthly, the satisfaction survey tool for education programs designed for adults, including faculty members, school administrators, and the general public from 2017 to 2021 was designed to cover three elements: ‘satisfaction with education contents’, ‘satisfaction with education operation’, and ‘satisfaction with detailed contents’. The number of survey questions varied across years, as the satisfaction with detailed contents contained a different number of questions for applicable contents of the education programs. Questions contained in the survey tool are presented in Appendix D.

4.3. Study Participants

This study covers the creative technical education programs and application results thereby developed by the Creative education center of Jeju National University, as funded by the Ministry of Education and the KOFAC from 2017 to 2021. The education programs targeted not only elementary and secondary students but also faculty members responsible for effective delivery of education, school administrators including principals and vice principals authorized to control in part the school atmosphere and educational orientation, and the general public, including parents, to improve their understanding of creative technical education. Accordingly, in 2017, the developed education programs were applied to 1139 persons in total, including 581 elementary/secondary students, 308 faculty members, 40 school administrators, and 210 members of the general public. In 2018, the programs were conducted with 1711 persons in total including 325 elementary/secondary students, 452 faculty members, 90 school administrators, and 844 members of the general public. In 2019, 2807 persons in total were covered in the programs, including 538 elementary/secondary students, 897 faculty members, 99 school administrators, and 1273 members of the general public. In 2020, due to the suspension of offline education and enforcement of social distancing policies, etc. in the aftermath of the COVID-19 pandemic, 1902 persons in total, fewer than those covered in the previous years, including 794 elementary/secondary students, 351 faculty members, 495 school administrators, and 262 members of the general public, were covered by the programs. In 2021, as education infrastructure that could support both on and offline education was implemented, 2037 persons in total including 596 elementary/secondary students, 79 faculty members, 33 school administrators, and 1329 members of the general public were covered by the programs. A total of 9596 persons engaged in the education programs during the five years. The breakdown of those covered in this study per year and status is as shown in Table 2.

4.4. Analysis Methods

As the various education programs developed were conducted with training targets from 2017 to 2021 and application results thereof were analyzed, yearly data of the sub-elements of elementary/secondary students’ survey results and time-series analysis on the yearly data of survey results for elementary/secondary students, faculty members, school administrators and the general public were performed. In the yearly analysis of the sub-elements, means and standard deviations were calculated, using technical statistics, and in the yearly time-series data analysis, one-way ANOVA was conducted on the equated survey results to compare across different groups. In this study, the Welch’s test conducted in the case of heteroskedasticity was conducted and the Games–Howell post-doc test was performed on the Welch’s test results. The IBM SPSS 24.0 program was used as a test tool. To compare the survey results across five years, the survey results were equated first. As survey questions varied across the years, units of parameters in common questions were unified before equating the survey results. Accordingly, Scale Transformation (ST) was used to calculate a scale transformation constant [39]. To calculate the constant, coefficients calculated by the Stocking–Lord method were used [40]. Therefore, we performed scale transformation on the parameters of survey questions used from 2018 to 2020 and in 2021 into units used in 2017. After the scale transformation, Item Response Theory (IRT) true score equating was performed by PIE program, and scores produced by equating satisfaction level scores across the five years in the 2017 unit were calculated [41]. Secondly, based on the survey equating, survey scores from 2017 to 2021 were transformed into a cross-comparable scale. The levels specified in 2017 were linked to survey scores from 2018 to 2021. In these processes, trends of changes in the means of satisfaction scores across the years were analyzed and the trends of score changes across different statuses per year were also analyzed. As the same study tool for adults was used throughout the five years, One-way ANOVA was conducted without equating the survey results.

5. Time-Series Analysis of Application Results of ICT-Based Education Program

The education programs developed herein were applied to each training target group for five years from 2017. Figure 3 shows that the developed programs are being applied to a variety of training target groups, including students, teachers, school administrators and the general public.

After the application of the programs, a satisfaction level survey was conducted for all training targets. Satisfaction survey and results analysis were intended to understand how effective the education programs were and to predict how likely it is to satisfy subsequent educators by such creative technical education programs. Application effects of the creative technical education analyzed and satisfaction scores predicted in this study will provide multiple implications for efforts to develop education strategies and policies designed to foster creativity and understanding of technologies.

The analysis results include those of satisfaction survey results specific to sub-elements per year and the means of survey results per year. In the analysis of the means of survey results per year, only the survey results for elementary/secondary students were equated, based on the fact that their satisfaction survey questionnaires contained different sub-questions across the years. Time-series analysis is a statistical technique that deals with time-series data and trend analysis. Time-series data is measured at regular time intervals or collected at specific periodic intervals. In essence, a time-series consists of a sequence of data points arranged over time, and time-series analysis is the process of understanding this data. As a time-series analysis is effective in identifying changing trends across years only when results of common factors are to be analyzed, survey results were equated in a way to remove unduplicated factors in the survey results and then analyzed. In other words, only common questions were extracted from the survey results of elementary/secondary students for a time-series analysis and un-equated data was used for the survey results of adults, including faculty members, school administrators and the general public.

5.1. Analysis of Satisfaction Based on Sub Elements of Education Participants by Year

First of all, for the survey results of elementary/secondary students, 2571 responses out of 2834 in total were used in the analysis, excluding insincere responses. The sub-elements of the 2017 survey included overall satisfaction level, level of interest, level of engagement, appropriateness of educational content, commitment to further education and detailed satisfaction level. Among them, the mean of overall satisfaction level was the highest at 4.89 whereas the appropriateness of educational contents was the lowest at 4.14. From 2018 to 2020, only detailed satisfaction level was excluded from the satisfaction questions used in 2017, and the rest of the questions were used as they were for the surveys. As mentioned before, the mean for overall satisfaction was the highest at 4.29 and the mean of appropriateness of educational contents was the lowest at 3.41. In 2019, the mean of level of engagement was the highest at 4.59, and the mean of appropriateness of educational contents was the lowest at 3.98. In 2020, the mean of level of interest was the highest at 4.41 whereas the mean of appropriateness of educational contents was the lowest at 3.97. In 2021, survey elements were changed from the previous year to contain overall satisfaction, level of interest, depth of learning, teachers’ guidance, opportunities for communication between teachers and students, improvement of problem-solving skills following education, and commitment to further education. Among them, the mean of overall satisfaction was the highest at 4.20, whereas the mean of improvement of problem-solving skills following education was the lowest at 3.85. Generally speaking, the mean of overall satisfaction was the highest in most cases and the mean of appropriateness of educational contents was the lowest. Detailed results are presented in Table 3.

Secondly, the sub-elements in the satisfaction survey for the education programs for faculty members provided for five years were analyzed. The 2087 faculty members participated throughout the study period, and 1996 sincere responses to the survey questions were utilized in the analysis. The sub-elements in the satisfaction survey tool for faculty members included educational content, education operation and detailed contents, and the same set of questions were used for five years to gather data. Speaking of the satisfaction survey results for sub-questions, the means of educational contents and detailed contents were the highest equally at 4.92 in 2017 and the mean of education operation turned out to be 4.68. In 2018, the mean of satisfaction with educational contents was the highest at 4.83 and the mean of detailed contents was 4.54. In 2019, satisfaction with education operation was the highest at 4.63 and the mean of educational contents was the lowest at 4.48. In 2020, the mean of detailed contents was the highest at 4.74 and the mean of education operation was the lowest at 4.71. In 2021, the mean of educational contents was the highest at 4.67 and the mean of education operation was the lowest at 4.57. All in all, excluding 2019, satisfaction with educational contents remained high, whereas satisfaction with detailed contents was comparatively lower than other elements for the same year. Detailed results are presented in Table 4.

Thirdly, out of 757 responses from the participants in the school administrator education programs for five years from 2017, only 671 responses were used in the analysis of the results. As is the case with the faculty member survey tool, the school administrator survey tool included such sub-elements as educational contents, education operation and detailed contents. In the 2017 survey, educational contents hit the highest satisfaction level at 4.94, almost equal to the perfect score, whereas the mean of detailed contents was 4.78. In 2018, the mean of education operation was the highest at 4.86 and that of detailed contents was the lowest at 4.80 on average. In 2019, the mean of detailed contents was 4.10 and the mean of satisfaction with educational contents hit only the 3.xx-range at 3.81 during the five years. In 2020, the mean of detailed contents was the highest at 4.5 and the mean of education operation was the lowest at 4.46. In 2021, satisfaction level with education operation was the highest at 4.79 and the mean of detailed contents was 4.44. All in all, satisfaction level with education operation remained high except for 2019. Table 5 shows the results of school administrator satisfaction survey specific to each sub-element.

Fourthly, 3918 persons participated in the education programs for the general public, constituting the biggest participant group, and 3525 responses were utilized in the analysis, excluding incomplete ones. The sub-elements of the survey were the same as those used for faculty members and school administrators. In the 2017 survey of the satisfaction level of the general public, the mean of educational contents was the highest at 4.79 and the mean of education operation was the lowest at 4.59. In 2018, the mean of education operation was 4.37 and the mean of educational contents was the lowest at 4.20. In 2019, the mean of education operation was overwhelmingly high at 4.85, but the mean of detailed contents was relatively low at 4.42. In 2020 and 2021, the mean values were relatively lower than in other years, with the highest mean in 2020 being for educational contents at 4.24 and the lowest being for detailed contents at 4.00. In 2021, the mean of satisfaction level with education operation was 4.28 and the mean of educational contents was 4.11. Satisfaction level in the education program for the general public, where the participating cases outnumbered other participant groups, tended to be somewhat low after the outbreak of COVID-19. Analysis results of the detailed elements are shown in Table 6.

5.2. Analysis of Satisfaction Regarding the Yearly Averages of Education Participants

First of all, to perform the time-series analysis of the survey results for elementary/secondary students, 2017 was specified as the base year for the survey scores, and survey scores from 2017 to 2020 were equated to the 2021 survey scores. As a result, differences in the scores for the remaining years were not significant against the 2021 reference scores and, when rounded up, these equated scores were equal to the 2021 reference scores. In conclusion, scores for overall satisfaction, interest level and commitment to further education which are common across the five-year-long survey results were extracted. In these processes, the means extracted by equating survey scores from 2017 to 2021 were calculated and compared. As the test of variance homogeneity featured a p value lower than 0.05, indicating heteroskedasticity of variances, the Welch’s test was performed. Furthermore, as the samples showed heteroskedasticity of variances and featured different sizes, the Games–Howell post-hoc test was performed to control error rates. Given the means across the five years as tested, satisfaction level with the education programs in 2017 was the highest, which was followed by 2020, 2018, 2021, and 2019. In addition, significant differences were found in the means of satisfaction of elementary/secondary students across the years (p < 0.001). Meanwhile, in the post-hoc test to identify the variation of mean values across the years, the mean of satisfaction level in 2017 was higher than for other years with statistical significance, and the results in 2018 were significantly higher than those of 2019 and 2020. Furthermore, the participants in 2020 showed a significantly higher level than those in 2019 and 2021, whereas the participants in 2021 showed a higher satisfaction level than those in 2019, with significant differences. However, the difference between the means of satisfaction in 2018 and 2021 was insignificant. Table 7 shows the comparison of yearly means of the equated survey scores for elementary/secondary students for the five years. Figure 4 visualizes the satisfaction scores of elementary/secondary scores across the five years in a box-and-whisker plot, with a trend line added. The trend line confirms that the mean of satisfaction level in 2019 fell remarkably. In addition, the data distribution is found to be wide.

As faculty members, school administrators, and the general public, except for elementary/secondary students, were surveyed on the same set of elements for five years, their time-series analysis was performed on un-equated data. The mean of satisfaction level of faculty members was the highest in 2017, as is the case with the mean of satisfaction level of elementary/secondary students (Mean = 4.82). After 2017, the means in 2018 and 2020 were the same at 4.72, which was followed by 2021 and 2018. As the lowest mean value was 4.61, faculty members who participated in the programs showed a high satisfaction level in general. Differences between the means of satisfaction across the years were statistically significant (p < 0.001), with the mean of satisfaction level in 2017 significantly higher than those of 2018 and 2019. The analysis results of the means of satisfaction level of faculty members across five years from 2017 are shown in detail in Table 8. Figure 5 shows the data in a box-and-whisker plot with a trend line added, and this trend line is smoother than that featured in Figure 4. Furthermore, except for 2020 and 2021, the data is distributed between 4 and 5 points.

Speaking of the mean of satisfaction level of school administrators with the education programs across the five years, the satisfaction level in 2017 was the highest, as is the case with the satisfaction levels of elementary/secondary students and faculty members (Mean = 4.86). By a slight margin, the mean of satisfaction level in 2018 was 4.84, followed by 2021, 2020, and 2019. The satisfaction level results of school administrators with the education programs across the years was statistically significant (p < 0.001). In the post-hoc test, the means of satisfaction level in 2017 and 2018 were significantly higher than those of 2017 and 2018, whereas the means of satisfaction level in 2020 and 2021 were more significant than that of 2019, a year which features more significant results than the means of satisfaction in all the other years for the past five years, excluding 2019. Results of detailed analysis of the means of satisfaction level of school administrators across the five years are shown in Table 9. The visualization chart in Figure 6 shows a flat trend, excluding 2019 which is the only year when the mean dropped to the 3-point range. 2019 also shows somewhat wider data distribution than the other years.

When the means of satisfaction level of the general public with the education programs across the five years from 2017 to 2021 were analyzed, the mean of satisfaction level in 2017 was the highest at 4.69 as is the case with all the other participant groups. Then, the mean in 2019 showed the second-highest satisfaction level at 4.64, which was followed by 2018, 2020 and 2021. The means of satisfaction of the general public analyzed per year for five years from 2017 showed statistically significant results (p < 0.001). Meanwhile, the post-hoc test showed that the high satisfaction level in 2017 was statistically significantly higher than the means of satisfaction in 2018, 2020, and 2021. Furthermore, the mean of satisfaction level in 2019 showed statistically more significant results than those of 2018 and 2021. Table 10 shows the analysis results of the means of satisfaction level of the general public across the five years, whereas Figure 7 presents a visualized chart of such analysis results. Given the data distribution and trend line in the chart, the means of satisfaction level in all years exceeded 4 points, indicating that the members of the general public who participated in the education programs remained satisfied across the five years.

6. Discussion

This paper proposes an education model that can develop creative and technical talents and improve the accessibility of creative technologies for the general public and describes how various creative technical education programs were developed, based on the model and applied to education sites for five years from 2017.

All education programs were planned early each year in order to be based on appropriate topics for each training target status, including elementary/secondary students, faculty members, school administrators and the general public. The education programs for elementary/secondary students focused on teaching the underpinning principles of ICT. In addition, not only such technologies as are already used in our daily life but also future technologies under development, such as human body communications technology enabling data transfer via human body or hyper-trust information security technologies, were introduced and their underpinning principles were taught. We designed the education programs in a way not just to encourage students to have interest in and understand ICT but also to adopt creative teaching techniques, including Havruta dialogue or quantum learning, in a bid to foster their convergent thinking. The education programs for faculty members provided more theoretical contents and teaching strategies readily workable on site. As the educational curriculum revised in 2015 was applied on site in Korea during the period of this study, the educational focus was shifted to promoting competency in preparation for the emergence of an intelligence and information society [42]. Accordingly, we provided textbooks to teachers busy implementing the new educational curriculum on site and suggested detailed educational contents and practical strategies usable in classes to help them. The education programs for school administrators included a tour program in Jeju Island and emphasized the importance of harmony between creative personality and ICT and needs for schools to spearhead changes. Meanwhile, as an offline tour as a part of the education programs was impacted by the outbreak of COVID-19, we utilized a metaverse platform to provide the training targets with unique educational experience. These educational strategies enabled us to realize how important ever-changing ICTs are to our life in the 21st century. Additionally, we integrated collage art and physical exercise into technical education to encourage administrators’ creative development. The education programs for the general public included more specific topics than those covered in education programs for other training target groups to ensure that more people can relate to the importance of creative technical education and practical strategy. Therefore, liberal art-oriented topics, including storytelling technique for environmental and ecological stories, ethics of AI, and relevance of creative technical education to happiness, were adopted to render ICTs more accessible to the general public. We implemented camping and ‘talk concert’ programs, offering educational content through artistic performances, such as music, dance, and sand art, departing from traditional lecture-oriented approaches. However, the program topics posed challenges for lower-grade elementary students due to their limited alignment with the formal educational curriculum. Consequently, faculty members or school administrators might face difficulties in accommodating these education programs within the constraints of the existing curriculum. Therefore, faculty members or school administrators may find it difficult to allocate time for the education programs proposed herein out of the total time available for the current educational curriculum. Moreover, this education approach did not consider variations in students’ performance levels. Since the education programs were designed based on the average comprehension level of the training targets, they might not be optimal for learners who already possess a substantial grasp of technology or for groups with differing performance levels.

The programs developed herein were applied to various training targets from 2017 to 2021, and their satisfaction level was surveyed afterward to gather and analyze feedback from them on the education programs. Firstly, satisfaction survey results of the training targets were analyzed per sub-element. Most of the students who took the education programs were found to be high in scores of overall satisfaction with the programs, interest level and level of engagement. The students became interested in the programs delivering new educational contents in combination with fun-oriented learning activities and focused more than classes of conventional subjects. Meanwhile, they were found to be somewhat reserved when responding to a question asking whether the level of education was appropriate for them. In fact, many students found the educational contents challenging. Furthermore, they did not whole-heartedly agree when responding to questions designed to verify problem-solving competency after education. This indicates that the programs intended for elementary/secondary students were not considerably conducive to the problem-solving competency of the learners. Speaking of the satisfaction survey results per sub-element of faculty members, they showed the highest satisfaction level with the educational contents in most cases each year. In particular, in the satisfaction survey results for the faculty members who participated in the programs, the lowest mean of the sub-elements was 4.48, implying that the education programs sufficiently fulfilled the expectations of the faculty members for five years. However, the mean of detailed contents was somewhat lower than the means of other elements, which indicates that the overall theme of the programs was good, but detailed contents could have been designed to be more logical. In the analysis of the sub-elements of satisfaction level of school administrators, they showed the highest mean when it came to educational contents in the same manner as for faculty members and revealed a uniform level of satisfaction across all sub-elements. The Jeju tour program incorporated into the education programs for administrators seems to have played a critical role in evoking a high satisfaction level from them. In 2019, the satisfaction level dropped somewhat across all sub-elements. In particular, satisfaction level with educational content fell to the lowest at 3 points, which is deemed to suggest that the educational contents did not fully address the need for ICT awareness from the school administrators’ perspective. The education programs for the general public covered the biggest number of participants, namely 3918 persons, and showed a high satisfaction level across all sub-elements, in spite of the fact that so many training targets were accommodated. However, in 2020 and 2021 after the outbreak of COVID-19, the satisfaction level tended to fall across all sub-elements. This seems to indicate that the fall in satisfaction level was not attributable to flaws in the educational contents, but to the fact that a large number of training targets had to receive education online rather than in person, leading to training targets relating to the contents less wholeheartedly.

Secondly, total score means were comparatively analyzed on the time-series data compiled across the five years, and the means of satisfaction level for all training targets across the years were verified to be different with statistical significance (p < 0.001). As for elementary/secondary students, their satisfaction level in 2019 was the lowest, but it was at or above 4 points in all other years. Meanwhile, the satisfaction level in 2018 was not significantly higher than that of 2021, but the means of satisfaction level were in the order of 2017, 2020, 2018, 2021, and 2019. The fall in satisfaction level among the students of 2019 is deemed to be attributable to difference in their performance level, which indicates the drawbacks of the education programs not differentiating contents level according to students’ performance level. Unlike other groups, total satisfaction scores of faculty members remained relatively uniform across the five years. However, the satisfaction level was the highest in 2017, and higher than those of 2018 and 2019 at significant level. It is deemed that providing diverse educational materials each year directly contributed to improving the satisfaction level of faculty members. The mean of satisfaction with the education for school administrators was also the highest in 2017, and the lowest in 2019. In addition, the satisfaction level in both 2017 and 2018 was significantly higher than those of 2019 and 2020. This is believed to be attributable to the same cause that led scores in all sub-elements to fall in 2019. The mean of satisfaction with the education programs for the general public across the years remained high at or over 4 points each year, with the satisfaction level in 2017 significantly higher than those of 2018, 2020, 2021 and the satisfaction level in 2019 higher than those of 2018 and 2020. In conclusion, the satisfaction level with all education programs, excluding those for the general public, was lower in 2019 than in any other years, but as the underlying cause was identified and the programs were improved, the satisfaction level rose up in 2020 and 2021. In addition, as the mean of satisfaction level remained above 4 points in many cases for five years, the education programs that introduced possibly relatively unfamiliar concepts and principles of ICT to the training targets but adopted creative techniques are deemed to have fulfilled the expectations of the training targets.

7. Conclusions

The findings of this study have significantly contributed to addressing the critical need for nurturing creativity and enhancing technical comprehension through education. While the importance of these aspects has been widely acknowledged, there has been a dearth of comprehensive studies that intricately intertwine both aspects [43,44]. This research was initiated by recognizing the necessity to bridge this gap and establish innovative educational paradigms. Notably, the educational models and programs developed in this study were influenced by the HBPM, a recognized pedagogical framework for fostering creativity. To augment the effectiveness of these models, we incorporated cutting-edge instructional methodologies, such as Havruta, quantum learning, and AI solutions. Additionally, we innovatively integrated offline Jeju tours into select programs, enriching the learning experience and nurturing convergent thinking skills. The adaptation of these techniques aimed to provide tailored instruction on ICT principles and educational content, catering to a diverse array of participants. Over the span of five years, from 2017 to 2021, more than 9000 individuals engaged with the proposed education programs. The consistently high satisfaction levels, often surpassing 4 points, underscore the efficacy and relevance of the creative technical education programs across a broad spectrum of participants. This success reaffirms the timeliness and adaptability of the developed educational initiatives. Furthermore, the significance of this study extends beyond its immediate impact. The educational topics and outcomes documented herein carry implications for future endeavors in creative technical studies. The multifaceted nature of our approach, which harmonizes with diverse educational settings—ranging from students to faculty members, school administrators, and the general public—paves the way for integrated creativity and technical understanding. As we move forward, our focus will remain on refining educational content that aligns seamlessly with school curricula and devising strategies for the effective implementation of the current educational landscape. The culmination of this study’s efforts, encapsulated within these findings, stands as a valuable reference point for subsequent explorations in the realm of creativity and technical education. This not only enhances the understanding of these critical domains but also charts a course for their cohesive development in the future.

Author Contributions

Conceptualization, N.P.; methodology, J.K.; validation, E.C., J.K. and N.P.; formal analysis, E.C. and J.K.; investigation, J.K.; resources, N.P.; data curation, E.C.; writing—original draft preparation, E.C.; writing—review and editing, E.C.; visualization, J.K.; supervision, N.P.; project administration, N.P.; funding acquisition, N.P. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to a Confidentiality Agreement.

Conflicts of Interest

The authors declare no conflict of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1. Hyper Blended Practical Model (HBPM).

Figure 2. Three steps process for research methodology.

Figure 3. Education program being applied.

Figure 4. Box-and-whisker plot of the satisfaction of elementary/secondary students by year.

Figure 5. Box-and-whisker plot of the satisfaction of faculty members by year.

Figure 6. Box-and-whisker plot of the satisfaction of school administrators by year.

Figure 7. Box-and-whisker plot of the satisfaction of general public by year.

Table 1

Educational contents.

	Status
	Elementary/Secondary Students	Faculty Members	School Administrators	General Public
2017	Human body communications technology	Creative skills for the era of knowledge & information in the 21st century	Creative personality education via Jeju Gotjawal tour	Technical education via ecology story told with a stork
	Cutting-edge realistic media	Key ICT principles & creative education in the era of intelligence & information	IIT-enabled innovative creativetechnical education	Decision-making competency & AI’s judgment capability
	AI of the future & ethics	Reconfiguration of educational curriculum via innovative creative teaching/learning methods & creative technical education strategy	Case studies ofcreative technical education	Transformed creative technical education
	Cloud of the future & security			Pursuit of happiness in creative technical education preparing for intelligence & information society of the future
				Creative technical education practical strategy
				Technical education dreamed by all
2018	Blockchain Principles	Key competencies in innovative creative technical education for the 4th Industrial Revolution	AI & creativity of the future	How to understand key AI principles using visual thinking teaching method
	Big data/AI liberal arts		Future hyper-connectivity technical education using Havruta teaching method	Key competencies of creative and convergent talents of the future in the era of intelligence & information
	Information security principles	Creative technical education class model	Creative technical education in International Baccalaureate education	Creative technical education done together, changed classroom atmosphere
	Space radiation disasters & safety	Best cases of creative technical education		Changing education policies, how to engage in education processes
2019	Blockchain application	IB education courses & evaluation methods	Social-emotional learning & creative technical education	Technical intelligence required of future talents
	Big data liberal arts/AI liberal arts application	Creative technical class design & sharing	Future AI image learning	IT convergence career-AI-based autonomous drivingForensic expertSpace environment expertInformation security expert
	Hyper trust of the future, Authentication informationsecurity	Creative teaching methods & implementation of technical education in the field	Machine learning neural network principles	AI future classroom
	Riskcommunication	Importance of creative technical education community collaboration	Cryptocurrency & blockchain	Cyber security camp
2020	Cutting-edge technologies in our life	Upcycling education using storytellingtechniques	Future education of creativity, intelligence & information in the New-Normal era	AI creativity talent education in the post-COVID era
	Creative technical education converging digital & liberal arts	Use of smart devices in classroom	Deep learning/machine learning-convolutional neural networks	Online creative technical education
	History of information security techniques	Engineering ethics education	Blockchain forgery/alteration prevention principles	Technical education befitting the Digital New Deal policy
	Mega science	Problem-solving skills for information society of the future	AI-guided learning-K-Nearest Neighbor	Maker lesson using husky lens
2021	AI humanities - Support vector machine- Convolutional neural network- AI ethics	AI humanities education using blended learning with focus on online interactions	Field application of creative technical education using metaverse platform	DNA-based creative information technology education in the intelligent information era
	Forensic science- Forensic science principles- Fingerprint identification technologies- Finding remains of Korean War soldiers- Forgery & alteration- Hacking & information security- Digital footprints- Anti-forensic- Virtual ecosystem implementation	Forensic science education using quantum learning with emphasis on dynamic learning activities	Mobile face recognition security program & blockchain	Creative technical education in the US/Laos on on/offline hybrid education platform
		AI-enabled digital therapeutics education	AI learning management system utilization method	Creative technical education & customizable lesson evaluation
	Digital therapeutics- COVID-blue digital therapeutics- ADHD digital therapeutics- Cyber abuse digital therapeutics	Creative technical mathematicsRobustness of rectangular frame structure	IT humanities education contents	Future creative technical education practical strategy at national level

Table 2

Research participants.

		Frequency	Ratio (%)
Year	2017	1139	11.87
	2018	1711	17.83
	2019	2807	29.25
	2020	1902	19.82
	2021	2037	21.23
Total		9596	100
Status	Elementary/secondary students	2834	29.53
	Faculty members	2087	21.75
	School administrators	757	7.89
	General public	3918	40.83
Total		9596	100

Table 3

Survey results of elementary and secondary school students’ satisfaction based on sub-elements by year.

	No. of Cases	Elements	Mean	Standard Deviation
2017	522	Overall satisfaction	4.89	0.45
		Level of interest	4.71	0.57
		Level of engagement	4.58	0.57
		Appropriateness of educational contents	4.14	0.93
		Commitment to further education	4.45	0.68
		Detailed satisfaction level	4.80	0.31
2018	284	Overall satisfaction	4.29	0.82
		Level of interest	4.25	0.86
		Level of engagement	4.09	0.88
		Appropriateness of educational contents	3.41	0.85
		Commitment to further education	3.86	1.03
2019	505	Overall satisfaction	4.25	0.59
		Level of interest	4.34	0.64
		Level of engagement	4.59	0.66
		Appropriateness of educational contents	3.98	0.82
		Commitment to further education	4.14	0.84
2020	769	Overall satisfaction	4.39	0.83
		Level of interest	4.41	0.84
		Level of engagement	4.33	0.84
		Appropriateness of educational contents	3.97	0.95
		Commitment to further education	4.18	0.90
2021	491	Overall satisfaction	4.20	0.85
		level of interest	4.09	0.89
		Depth of learning	4.02	0.91
		Teachers’ guidance	4.11	0.90
		Opportunities for communication between teachers and students	4.18	0.91
		Improvement of problem-solving skills following education	3.85	0.87
		Commitment to further education	4.03	0.99

Table 4

Survey results of faculty member satisfaction based on sub-elements by year.

	No. of Cases	Elements	Mean	Standard Deviation
2017	293	Educational contents	4.92	0.19
		Education operation	4.68	0.35
		Detailed contents	4.92	0.26
2018	438	Educational contents	4.83	0.25
		Education operation	4.79	0.26
		Detailed contents	4.54	0.46
2019	855	Educational contents	4.48	0.41
		Education operation	4.63	0.33
		Detailed contents	4.50	0.41
2020	339	Educational contents	4.73	0.47
		Education operation	4.71	0.39
		Detailed contents	4.74	0.44
2021	71	Educational contents	4.67	0.57
		Education operation	4.57	0.67
		Detailed contents	4.60	0.57

Table 5

Survey results of school administrator satisfaction based on sub-elements by year.

	No. of Cases	Elements	Mean	Standard Deviation
2017	38	Educational contents	4.94	0.15
		Education operation	4.91	0.18
		Detailed contents	4.78	0.24
2018	64	Educational contents	4.86	0.32
		Education operation	4.93	0.14
		Detailed contents	4.80	0.36
2019	93	Educational contents	3.81	0.89
		Education operation	4.02	0.78
		Detailed contents	4.10	0.64
2020	451	Educational contents	4.47	0.62
		Education operation	4.46	0.62
		Detailed contents	4.56	0.54
2021	25	Educational contents	4.71	0.61
		Education operation	4.79	0.41
		Detailed contents	4.44	0.68

Table 6

Survey results of general public satisfaction based on sub-elements by year.

	No. of Cases	Elements	Mean	Standard Deviation
2017	195	Educational contents	4.79	0.42
		Education operation	4.59	0.51
		Detailed contents	4.74	0.39
2018	1018	Educational contents	4.20	0.57
		Education operation	4.37	0.44
		Detailed contents	4.25	0.84
2019	998	Educational contents	4.62	0.85
		Education operation	4.85	0.36
		Detailed contents	4.42	0.97
2020	235	Educational contents	4.24	0.69
		Education operation	4.21	0.77
		Detailed contents	4.00	0.78
2021	1079	Educational contents	4.11	0.67
		Education operation	4.28	0.65
		Detailed contents	4.18	0.66

Table 7

2017~2021 means elementary/secondary students satisfaction level.

	No. of Cases	Mean	Standard Deviation	Welch F	p	Post-Hoc(Games–Howell)
2017 (a)	522	4.54	0.60	89.304 ***	0.000	a > b,c,d,eb > c,dd > c,ee > c
2018 (b)	284	4.14	0.78
2019 (c)	505	3.30	0.80
2020 (d)	769	4.32	0.71
2021 (e)	491	4.10	0.82

*** p < 0.001.

Table 8

2017~2021 means of faculty members’ satisfaction level.

	No. of Cases	Mean	Standard Deviation	Welch F	p	Post-Hoc(Games–Howell)
2017 (a)	293	4.82	0.18	9.097 ***	0.000	a > b,c
2018 (b)	438	4.72	0.30
2019 (c)	855	4.54	0.33
2020 (d)	339	4.72	0.35
2021 (e)	71	4.61	0.57

*** p < 0.001.

Table 9

2017~2021 means school administrators’ satisfaction level.

	No. of Cases	Mean	Standard Deviation	Welch F	p	Post-Hoc(Games–Howell)
2017 (a)	38	4.86	0.17	33.471 ***	0.000	a > c,db > c,dd > ce > c
2018 (b)	64	4.84	0.23
2019 (c)	93	3.99	0.69
2020 (d)	451	4.50	0.54
2021 (e)	25	4.67	0.61

*** p < 0.001.

Table 10

2017~2021 means of general public satisfaction level.

	No. of Cases	Mean	Standard Deviation	Welch F	p	Post-Hoc(Games–Howell)
2017 (a)	195	4.69	0.36	32.081 ***	0.000	a > b,d,ec > b,e
2018 (b)	1018	4.23	0.50
2019 (c)	998	4.64	0.66
2020 (d)	235	4.18	0.70
2021 (e)	1079	4.18	0.62

*** p < 0.001.

Appendix A. 2017 Creative Technical Education Program Satisfaction Level Survey Tool (for Students)

Mark how much you agree to the question presented in the left. (Most agreed 5, Not agreed 1)

1. Overall satisfaction	1	2	3	4	5
1. Are you satisfied with the education program?
2. Interest level	1	2	3	4	5
2. Was the education program interesting?
3. Level of engagement	1	2	3	4	5
3. Did you engage actively in the education program?
4. Appropriateness of educational contents	1	2	3	4	5
4. Was the level of the education program appropriate for you?
5. Commitment to further education	1	2	3	4	5
5. Would you like to engage in a similar creative technical education program in the future?
6. Detailed satisfaction	1	2	3	4	5
6.1. I engaged in the program actively and briskly.
6.2. The contents of the education program were interesting to me.
6.3. I liked the education program more than regular school classes.
6.4. The teacher guided me well regarding my questions.
6.5. The teacher worked hard to provide guidance.
6.6. I was given enough opportunities to ask questions in class.
6.7. I learned the topics of the program in depth.
6.8. While engaging in the program, I was able to freely communicate with teachers or other students.
6.9. While I was engaging the program, the teacher allowed students to come up with diverse answers freely.
6.10. While I was engaging the program, the teacher did not just throw in a right answer, but gave students enough time to explore diverse solutions.
6.11. While engaging the program, I participated in the program activities attentively.
6.12. After the program, I began to try to solve problems from a different perspective from others.
6.13. After the program, I got to think on my own to solve problems.
6.14. After the program, I got to finish various learning activities to the end.
6.15. After the program, I began to approach a problem from multiple perspectives.
6.16. After the program, I tried to link what I had learned with my daily life.
6.17. After the program, I tried to apply knowledge learned under various subjects concurrently to solving problems.
6.18. After the program, I was encouraged to participate in creative technical education programs covering more extensive topics, if available.
6.19. After the program, I realized how important it is to cooperate with others.

Appendix B. 2018–2020 Creative Technical Education Program Satisfaction Level Survey Tool (for Students)

Mark how much you agree to the question presented in the left. (Most agreed 5, Not agreed 1)

1. Overall satisfaction	1	2	3	4	5
1. Are you satisfied with the education program?
2. Interest level	1	2	3	4	5
2. Was the education program interesting?
3. Level of engagement	1	2	3	4	5
3. Did you engage actively in the education program?
4. Appropriateness of educational contents	1	2	3	4	5
4. Was the level of the education program appropriate for you?
5. Commitment to further education	1	2	3	4	5
5. Would you like to engage in creative technical education programs in the future?

Appendix C. 2021 Creative Technical Education Program Satisfaction Level Survey Tool (for Students)

Mark how much you agree to the question presented in the left. (Most agreed 5, Not agreed 1)

1. Overall satisfaction	1	2	3	4	5
1. Are you satisfied with the education program?
2. Interest level	1	2	3	4	5
2. Was the education program interesting?
3. Depth of learning	1	2	3	4	5
3. I learned the topics of the program in depth.
4. Teachers’ guidance	1	2	3	4	5
4. The teacher guided me well regarding my questions.
5. Communication with teachers or other students	1	2	3	4	5
5. While engaging in the program, I was able to freely communicate with teachers or other students.
6. Improved problem-solving competency after education	1	2	3	4	5
6.1. After the program, I got to think on my own to solve problems.
6.2. After the program, I began to approach a problem from multiple perspectives.
6.3. After the program, I tried to apply knowledge learned under various subjects concurrently to solving problems.
6.4. After the program, I tried to link what I had learned with my daily life.
7. Commitment to further education	1	2	3	4	5
7. After the program, I was encouraged to participate in creative technical education programs covering more extensive topics, if available.

Appendix D. 2017–2021 Creative Technical Education Program Satisfaction Level Survey Tool (for Adults)

Mark how much you agree to the question presented in the left. (Most agreed 5, Not agreed 1)

1. Educational contents		1	2	3	4	5
1.1. Did the education program help you understand the overall direction of future-oriented creative technical education in the era of the 4th Industrial Revolution?
1.2. Were you satisfied with the configuration and contents of the education program in general?
1.3. Was the education program staffed with appropriate instructors in general?
1.4. Do you think this education needs to be maintained and developed way forward?
1.5. Did you engage in the training sincerely?
2. Educational operation		1	2	3	4	5
2.1. Was the administrative procedure for admission to the program (application process, written notification, etc.) appropriate?
2.2. Was the educational infrastructure (education venue, location, etc.) appropriate?
2.3. Were logistics for the program (meals, supplies, digital devices, etc.) appropriate?
2.4. Was the education program appropriately scheduled and planned?
3. Detailed contents (indicative)	Instructor	1	2	3	4	5
3.1. The 4th Industrial Revolution & Key Competencies	Park
3.2. Creative Pioneering Education Program Workshop Utilizing Intelligent and Information Technologies	Lee
3.3. Observation of Schools Implementing Creative Technical Education Programs	Kim

References

1. French, A.; Shim, J.; Risius, M.; Larsen, K.R.; Jain, H. The 4th industrial revolution powered by the integration of AI, blockchain, and 5G. CAIS; 2021; 49, pp. 266-286. [DOI: https://dx.doi.org/10.17705/1CAIS.04910]

2. Pew Research Center. Available online: https://www.pewresearch.org/global/2019/02/05/smartphone-ownership-is-growing-rapidly-around-the-world-but-not-always-equally (accessed on 21 February 2021).

3. Bundy, A. Preparing for the future of artificial intelligence. AI Soc.; 2017; 32, pp. 285-287. [DOI: https://dx.doi.org/10.1007/s00146-016-0685-0]

4. Cropley, A. Creativity-focused technology education in the age of industry 4.0. Create. Res. J.; 2019; 32, pp. 184-191. [DOI: https://dx.doi.org/10.1080/10400419.2020.1751546]

5. Choi, E.; Park, N. Development and demonstration of creative and convergence textbooks using creative storytelling techniques. J. Korea. Conv. Soc.; 2021; 12, pp. 143-151. [DOI: https://dx.doi.org/10.33645/cnc.2021.08.43.8.143]

6. Iandoli, C.C. A synopsis of theories of creativity since 1950. JOTS; 1994; 20, pp. 8-15.

7. Fowler, Z. Creativity and education futures: Learning in a digital age. Br. Educ. Res. J.; 2011; 38, pp. 349-352. [DOI: https://dx.doi.org/10.1080/01411926.2011.605110]

8. Henriksen, D.; Henderson, M.; Creely, E.; Ceretkova, S.; Černochová, M.; Sendova, E.; Sointu, E.T.; Tienken, C.H. Creativity and technology in education: An international perspective. Technol. Knowl. Learn.; 2018; 23, pp. 409-424. [DOI: https://dx.doi.org/10.1007/s10758-018-9380-1]

9. Choi, Y.; Lim, Y.; Kim, S.H.; Lee, K.-J. Perceptions of Korean technology education experts on the future technology education directions in relation to current technology education in the United States and Japan: Focusing on “STEL” in the United States and “Technology Education in the 21st Century” in Japan. J. Korean Inst. Ind. Educ.; 2021; 46, pp. 40-68. [DOI: https://dx.doi.org/10.35140/kiiedu.2021.46.2.40]

10. Ministry of Education Republic of Korea. Available online: https://www.moe.go.kr/boardCnts/view.do?boardID=141&boardSeq=60747&lev=0&searchType=null&statusYN=W&page=20&s=moe&m=040401&opType=N (accessed on 5 March 2022).

11. Jeong, Y.; Yu, J.; Kim, C. Establishment of elementary information curriculum to solve the learning gap between students. JKAIE; 2021; 25, pp. 33-40. [DOI: https://dx.doi.org/10.14352/jkaie.2021.25.1.33]

12. Choi, E.; Choi, Y.; Park, N. Blockchain-centered educational program embodies and advances 2030 sustainable development goals. Sustainability; 2022; 14, 3761. [DOI: https://dx.doi.org/10.3390/su14073761]

13. Sorenson, R.L. Electrical Manufacturing Industry: A resource Study with Implications for Industrial Arts Curriculum Development. Ph.D. Dissertation; The Ohio State University: Columbus, OH, USA, 1964.

14. Hofstein, A. Discussion over at the fourth IOSTE symposium. Int. J. Sci. Educ.; 1988; 10, pp. 357-366. [DOI: https://dx.doi.org/10.1080/0950069880100403]

15. Kim, M. A Comparative Study on the STEM Education Policy in the United States and the STEAM Education Policy in Korea. Master’s Thesis; Chonnam National University: Gwangju, Republic of Korea, 2013.

16. Fraser, D.M.; Avis, M.; Mallik, M. The MINT project—An evaluation of the impact of midwife teachers on the outcomes of pre-registration midwifery education in the UK. Midwifery; 2013; 29, pp. 86-94. [DOI: https://dx.doi.org/10.1016/j.midw.2011.07.010]

17. Radziwill, N.M.; Benton, M.C.; Moellers, C. From STEM to STEAM: Reframing what it means to learn. STEAM J.; 2015; 2, 3. [DOI: https://dx.doi.org/10.5642/steam.20150201.3]

18. Baek, Y.S.; Park, H.; Kim, Y.; Noh, S.G.; Park, J.-Y.; Lee, J.; Jeong, J.-S.; Choi, Y.H.; Han, H. STEAM Education in Korea. JLCCI; 2011; 11, pp. 149-171.

19. Park, H.; Kim, Y.; Noh, S.G.; Lee, J.; Jeong, J.-S.; Choi, Y.H.; Han, H.; Baek, Y.S. Components of 4C-STEAM education and a checklist for the instructional design. JLCCI; 2012; 12, pp. 533-557.

20. Kim, Y.; Lee, W.; Kim, J. Analysis of the operation status of STEAM education. Proceedings of the 2022 KACE; Jeju-si, Republic of Korea, 21 January 2022.

21. Korea Foundation for the Advancement of Science and Creativity. Available online: http://www.crezone.net/education/creativeeducationcenter/referenceRoom/detail.do?sid=2633&pageIndex=1&search= (accessed on 18 June 2022).

22. Kim, J. Education system reshuffle for balanced development of national human resources: 2022 college entrance policy and credit system in high school. KPPAR; 2019; 18, pp. 207-234.

23. Kim, H.-H.; Han, K.-S. Exploring elementary teachers’ perceptions on creativity-personality education using concept mapping. J. Korean. Soc. G T; 2011; 10, pp. 49-72.

24. Stohlmann, M.; Moore, T.J.; Roehrig, G.H. Considerations for teaching integrated STEM education. J-PEER; 2012; 2, 4. [DOI: https://dx.doi.org/10.5703/1288284314653]

25. Yakman, G. ST∑@ M education: An overview of creating a model of integrative education. Pupil’s Attitudes Towar. Technol.; 2008; 19, pp. 335-358.

26. Gorgone, J.; Davis, G.B.; Valacich, J.S.; Topi, H.; Feinstein, D.L. IS 2002 model curriculum and guidelines for undergraduate degree programs in information systems. Commun. Assoc. Inf.; 2003; 11, pp. 1-53. [DOI: https://dx.doi.org/10.17705/1CAIS.01101]

27. Chinnasamy, P.; Ramani, D.R.; Ayyasamy, R.K.; Jebamani, B.J.A.; Dhanasekaran, S.; Praveena, V. Applications of Blockchain Technology in Modern Education System—Systematic Review. Proceedings of the 2023 International Conference on Computer Communication and Informatics (ICCCI); Coimbatore, India, 23–25 January 2023; [DOI: https://dx.doi.org/10.1109/ICCCI56745.2023.10128381]

28. Chinnasamy, P.; Elumalai, A.; Ayyasamy, R.K.; Kavya, S.P.; Dhanasekaran, S.; Kiran, A. BookChain: A Secure Library Book Storing and Sharing in Academic Institutions using Blockchain Technology. Proceedings of the 2023 International Conference on Computer Communication and Informatics (ICCCI); Coimbatore, India, 23–25 January 2023; [DOI: https://dx.doi.org/10.1109/ICCCI56745.2023.10128571]

29. Lee, D.; Kim, S.; Park, N. The blockchain-based online learning platform for the untact education environment in the post-COVID-19 era. J. KIIT; 2020; 18, pp. 109-121. [DOI: https://dx.doi.org/10.14801/jkiit.2020.18.11.109]

30. Choi, E.; Choi, Y.; Park, N. Development of blockchain learning game-themed education program targeting elementary students based on ASSURE model. Sustainability; 2022; 14, 3771. [DOI: https://dx.doi.org/10.3390/su14073771]

31. Chiu, T.K.F. A holistic approach to the design of artificial intelligence (AI) education for K-12 schools. Techtrends; 2021; 65, pp. 796-807. [DOI: https://dx.doi.org/10.1007/s11528-021-00637-1]

32. Mei, L.; Qi, L.; Zhang, Y. Blended learning models for information technology education as general course for college students. Proceedings of the 2015 International Symposium on Educational Technology; Wuhan, China, 29 July 2015; [DOI: https://dx.doi.org/10.1109/ISET.2015.18]

33. Changpetch, S.; Seechaliao, T. The propose of an instructional model based on STEM education approach for enhancing the information and communication technology skills for elementary students in Thailand. Int. Educ. Stud.; 2020; 13, pp. 69-75. [DOI: https://dx.doi.org/10.5539/ies.v13n1p69]

34. Choi, E.; Park, N. Demonstration and effect analysis of creative plugged education model using intelligent information technology. J. KIIT; 2021; 19, pp. 95-103. [DOI: https://dx.doi.org/10.14801/jkiit.2021.19.4.95]

35. Choi, E.; Park, N. Application methods and development assessment tools for creative convergence education programs for elementary and secondary schools based on hyper blended practical model. JCCT; 2022; 8, pp. 117-129. [DOI: https://dx.doi.org/10.17703/JCCT.2022.8.2.117]

36. Hyper Island. Available online: https://www.hyperisland.com/blog/hyper-islands-blended-and-distributed-learning-approach-responds-to-our-learners-unique-needs (accessed on 13 July 2016).

37. Ayre, C.; Scally, A.J. Critical values for Lawshe’s content validity ratio: Revisiting the original methods of calculation. Meas. Eval. Couns. Dev.; 2014; 47, pp. 79-86. [DOI: https://dx.doi.org/10.1177/0748175613513808]

38. Han, S.; Kim, H.; Lee, C. Development and application of creative education learning program using creative thinking methods. J. Korean Soc. Earth Sci. Edu.; 2020; 13, pp. 162-174. [DOI: https://dx.doi.org/10.15523/JKSESE.2020.13.2.162]

39. He, Y.; Cui, Z. Evaluating robust scale transformation methods with multiple outlying common items under IRT true score equating. Appl. Psychol. Meas.; 2019; 44, pp. 296-310. [DOI: https://dx.doi.org/10.1177/0146621619886050]

40. Kolen, M.J.; Brennan, R.L. Test equating, scaling, and linking methods and practices. Statistics for Social and Behavioral Sciences; 3rd ed. Springer: New York, NY, USA, 2014; pp. 1-536.

41. Kim, D.-I.; Brennan, R.; Kolen, M. A comparison of IRT equating and beta 4 equating. J. Educ. Meas.; 2005; 42, pp. 77-99. [DOI: https://dx.doi.org/10.1111/j.0022-0655.2005.00005.x]

42. Lee, J.Y.; Lee, K.-H.; Lee, B.C.; Ka, E.-A. Case analysis of competency-based school curriculum design and implementation: Focused on curriculum research schools. JCE; 2017; 20, pp. 1-30. [DOI: https://dx.doi.org/10.22799/jce.2017.20.1.001]

43. Dufva, T.; Dufva, M. Grasping the future of the digital society. Futures; 2019; 107, pp. 17-28. [DOI: https://dx.doi.org/10.1016/j.futures.2018.11.001]

44. Santos, A.I.; Serpa, S. The importance of promoting digital literacy in higher education. Int’l J. Soc. Sci. Stud.; 2017; 5, pp. 90-93. [DOI: https://dx.doi.org/10.11114/ijsss.v5i6.2330]