1. Introduction
Integrating environmental sustainability concepts into different sectors is crucial in addressing challenges such as climate change, energy efficiency, food insecurity, biodiversity loss, and global pollution. These challenges demand systemic solutions that involve both societal and individual efforts. Integrating these topics into formal and non-formal training education programmes enables individuals to develop the skills and knowledge to cope with complex sustainability challenges. As Elegbede et al. pointed out, such educational involvement encourages individuals to engage in sustainable practices actively. These efforts contribute to the United Nation’s Sustainable Development Goals (SDGs), promoting a more resilient and equitable society [1].
Furthermore, integrating sustainability topics into formal education programmes and informal training aligns with education for sustainable development (ESD), which prepares individuals to tackle complex sustainability issues [2]. These issues comprise environmental protection, social equity, economic sustainability, and promoting an environmentally ethical society. These aspects also align with the sustainable learning and education (SLE) concept reported by Hays and Reinders [3]. The SLE concept aims to equip individuals with the skills to thrive in complex contexts and contribute to a better world, aligning with sustainable curricula and learning methods. In addition, the authors of the SLE emphasise systemic thinking and self-sufficiency, suggesting a synergistic approach to promoting sustainability in education and professional development.
Over the last decade, studies and literature reviews revealed a growing interest in integrating sustainability competencies into various study programmes, including teacher education [4,5]. Although many studies focus on assessing sustainability competencies [6] and their integration into pedagogical approaches [7], there is still a lack of a unified framework for integrating and evaluating these competencies at the global and national levels. Self-directed learning courses remain a common pedagogical approach. Studies indicate that students are partially prepared for sustainable learning, especially in digital competencies, which are key to self-directed learning [8]. However, they face difficulties setting goals, managing time, managing stress, and preparing for online lessons.
Our study within the DECIDE project entitled “Career choices competencies for the post-pandemic future using multicriteria decision-making”, initially presented at the 19th conference on sustainable development of energy, water, and environment systems (SDEWES) [9], underscores the importance of sustainable development for society’s future. We aim to explore how digital tools, such as an optimised web application and an e-guide, develop young people’s career competencies. Therefore, our key research question is how these tools support self-directed learning and enable young people to identify and fill gaps in their competencies. In this respect, we include digital strategies and tools to support the development of sustainability competencies in this paper. Our research optimised a previously developed web-based application, whose functionality and testing results are described in detail elsewhere [10,11]. For the DECIDE project, the optimised version of the application allows young people to monitor the development of their career competencies to tackle sustainability challenges in workplaces and society. This study contributes to understanding and addressing sustainability challenges by integrating digital tools with holistic approaches. Tools such as the DECIDE e-guide empower individuals to make informed decisions and foster sustainability-driven decision-making in education and society.
2. Contextual Overview of Sustainability and Digitalisation in Education
2.1. The Importance of Integrating Environmental Sustainability into Educational Programmes
As the environment and society continue to change, understanding the impact of individuals’ decisions on society and the economy also evolves. Integrating sustainability into education enhances people’s awareness of how their actions affect the environment and society. This focus remains critical for future societal and economic development in the face of current global crises. However, it is essential to acknowledge that the future may not align perfectly with today’s sustainability expectations. Achieving sustainable development requires a collective and holistic approach [12] involving education, research, business, policy, and society at local, regional, national, and international levels [13]. Inclusive education must also address the guidelines, laws, rules, and policy frameworks targeting environmental issues. It should encourage active participation, empowering individuals to contribute to collective changes towards environmental responsibility. Education institutions play a pivotal role by implementing actions such as constructing eco-friendly buildings or adopting waste reduction measures [1]. Learning should incorporate sustainability principles to protect natural resources and ensure human well-being, respecting ecological limits. By integrating environmental sustainability into education, individuals are better prepared for a future where sustainability considerations are key. At the same time, they gain competencies to make informed decisions in the face of ongoing sustainability challenges.
The importance of embedding sustainability in higher education (HE) has been recognised by Obrecht et al. Their investigation into study programmes at different levels (Bachelor of Science, Master of Science, and Doctor of Philosophy) found that moderate levels of sustainability are often included in topics such as environmental protection, ecology, and ‘green’ practices [14]. However, there are significant differences between specific study programmes and fields of study, reflecting students’ unequal level of education for future management challenges.
Awareness of sustainable development extends beyond formal education, particularly through citizen science [15,16]. Citizen science fosters collaboration between researchers and volunteers, integrating local knowledge and community values into sustainable policy agendas. Citizen science promotes concepts of environmental sustainability in society and education. In this way, it helps make policies more context-sensitive and responsive to societal and economic needs. Certoma et al. emphasised that urban environments are extremely important for applying crowdsourcing in managing urban sustainable development. They recommend that policymakers allocate more resources to ensure crowdsourcing platforms are open, transparent, interoperable, and scalable [17]. By actively engaging participants in hands-on research and collaborative projects, citizen science contributes to the evolving landscape of education that emphasises digital tools and platforms for fostering environmental consciousness.
2.2. Digitalisation and Tools to Support Sustainable Development in Education
Jackman et al. highlight the need for inclusive digitalisation in education to support sustainable development and the digital economy [18]. Developing digital skills is critical for individuals to thrive in the digital age. Digital education strategies for sustainable development are becoming increasingly important, particularly in light of initiatives such as the United Nations Sustainable Development Goals, especially Goal 4, which emphasises inclusive and quality education for all [19]. ESD recognises the transformative role of digital technology in pedagogy and organisational practices. Leveraging these technologies fosters innovative teaching methods for sustainability education.
One emerging strategy is using massive open online courses (MOOCs) to promote sustainable development education. Gómez Zermeño’s research highlights that challenge-based learning in MOOCs strengthens participants’ skills that can be applied in real-world scenarios [20]. Future studies should examine how big data can predict digital skills needed to enhance MOOCs’ effectiveness in promoting education for sustainable development. Similarly, visualisation tools such as concept maps are valuable for addressing complex sustainability challenges. Concept maps have been increasingly used for educational purposes in recent years, involving technological developments’ impact on education’s sustainability [21]. Liu et al. found that concept maps clarify hierarchical knowledge structures, aiding content organisation and navigation [22]. Moreover, concept maps can serve as a common knowledge base that promotes collaboration and exchange between students and teachers from different countries or fields of study. It was even shown in [23] that using concept maps in the learning process leads to acquiring new knowledge, skills, and critical thinking. This approach facilitates the transfer of knowledge and skills from educational contexts to real-world applications, underlining the importance of collaborative and personalised learning environments. Embedding environmental sustainability in society and education requires a holistic approach, with room for digital tools and technologies [24]. These technologies can help address challenges holistically and support decision-making on sustainability in education and society.
3. Methods
The research focuses on addressing competencies for career choice in a post-pandemic future using multi-criteria decision-making through adaptive self-directed learning. This study focused on young people aged between 18 and 25. During this period, young people usually make career and further education decisions that can lead to long-term consequences for their lives and employment. This is also the period when young people are coming up against various life milestones, such as finishing secondary school, choosing a higher education study field, having their first interviews with employers and entering the labour market. Young people are exploring their interests, identities, strengths, skills, abilities, and values, which are crucial for making further decisions. These are also the results of different social influences in the real and virtual world. In addition, young people at this time are often confronted with pressures and expectations from society, family, and themselves regarding their success and career, as well as their educational and professional paths. It is, therefore, crucial at this stage to provide supportive tools and opportunities to develop competencies that will enable young people to make more informed decisions and successfully navigate their career paths in sustainable-oriented world scenarios.
Recognising the importance of this crucial period, we stress the need to provide them with accessible tools for self-assessment and competency analysis tailored to their individual needs. Previous studies have shown that young people, for example, in cultures such as South Korea, often face career decision difficulties due to a lack of self-understanding and career self-identity, highlighting the importance of providing tailored career counselling programmes that establish self-concept and identity [25]. This approach aims to empower them to make informed career choices and to open opportunities for them to participate actively in green transition and sustainable development initiatives. By developing and testing the DECIDE e-guide, which contains five self-learning modules and a multi-criteria competency self-assessment tool, and by optimising the online application for monitoring the development of career competencies, we aim to equip young individuals with the tools needed to identify missing competencies that are essential for pursuing sustainable and innovative career paths. Such initiatives not only support young people during a transformative period in their lives, but also provide fundamental data for constructing effective career guidance systems, which could have broader implications for national career development and education policies.
3.1. Testing Location
Training students from different countries and validating the tools developed was conducted in Święta Katarzyna, Poland, over five working days, from 6 to 10 November 2023. The event programme included several activities that encouraged networking, communication, cooperation, teamwork, problem-solving, and acquiring new knowledge and competencies to facilitate young people’s future career and educational decisions.
3.2. Target Group
The key target group for the training and tool assessment was young people aged 18–25 who were motivated to participate and learn new skills to develop their career paths and decision-making. The training was also attended by volunteers from different organisations and other project participants motivated to impact developing career decision-making among the youth. The distribution of DECIDE participants by gender and nationality is shown in Figure 1. The project and study involved 11 students or young individuals. However, we did not track data regarding their specific university degree levels or their average age.
The participant selection process for the training in Poland was carried out in several steps. Each project partner in their respective country contacted various non-governmental organisations working with youth and secondary schools and universities to identify potential candidates willing to participate. Partners conducted inquiries within these organisations to find suitable candidates based on motivation, interest in skill development, ability to collaborate in an international setting, and willingness to travel. The project covered travel and accommodation costs for all selected participants. After the selection, participants received confirmation of their participation.
3.3. Developing E-Guide Modules and Tools for Self-Assessment of Competencies
One of the project activities includes the development of an e-guide with presentations of future job profiles based on the current post-pandemic and labour market requirements about European Union (EU) legal frameworks such as the EU Green Deal (e.g., green jobs, interdisciplinarity and digital literacy…). The document aims to promote a better understanding of long-term career choices among young people. In addition, at the end of the DECIDE e-guide, there is a tool for self-assessment of multi-criteria competencies for young people’s career choices through an adaptive learning approach with personalised results for each user based on their competencies input and interests. The methodology for their development follows a user-centred and iterative approach, combining instructional design principles, adaptive learning techniques, and best practices in e-learning development. First, needs are assessed based on previous research, and then content is created that aligns with EU legal frameworks and the post-pandemic labour market. The e-guide is structured around clear learning objectives, theory segments, and interactive activities such as quizzes and exercises designed to engage users and reinforce learning.
The content of the e-guide is also integrated with the DECIDE web application for competencies development monitoring. The e-guide is available online at
3.4. Online App Development
The DECIDE project is developing a web application that will allow young people to follow the development of their career competencies while learning online through the DECIDE e-guide. The application is based on the result-oriented engagement system for performance optimisation (RESPO) application developed in the RESPO X project [10], entitled “Revolution of E-Skills with Participatory Online eXpert system”. For the DECIDE project, the RESPO application was optimised in terms of database development and functionality, which was significantly simplified. The app allows user registration, where the individual can see the charts for competency development during the training by module.
When pre-test and post-test scores are available, there are several possible approaches to show knowledge and competency development progress in the web application. One option is to show it with a graphical representation of progress with a timeline. The x-axis should show time (pre-test, post-test), and the y-axis should show the number of points achieved. For example, a separate bar graph shows the results before and after the training, giving a clear overview of the changes in knowledge. It is important to use dynamic graphics where the pre-test results are shown in one colour and the post-test results in another. This allows a clear distinction between results before and after training. The graphs should also include trend lines showing the change in the group’s average scores before and after the test.
3.5. Testing Procedure
As part of the training, we tested the content of an e-guide for young people on developing career competencies needed for future jobs and the labour market. For the youth training in Kielce, we used an existing online solution, the Socrative platform, to assess the competencies acquired during self-learning with the e-guide. We found the selected online tool to have several advantages when used for self-learning. The digital tool is always free for students to use. Students can install and use it on digital platforms (e.g., smartphones, tablets, laptops, and computers). For lecturers, it allows automatic real-time scores and instant feedback from participants. In this online tool, we prepared a pre-test and post-test for each e-guide module to check the participants’ knowledge before and after the lecture for each module except for Module 5. Before the content of each module was discussed during the training, each participant completed an online pre-test with questions on the module topics. The questions were of two types: multiple choice and “fill in the blank space” questions. They then listened to a short lecture and were engaged in interactive exercises. At the end of each training session, students completed a post-test with the same questions as those asked in the pre-test. The respective project partner responsible for developing each module prepared the test questions. Each module included ten multiple-choice questions with one correct answer, where participants had to select the correct answer from four options. Some modules (Modules 2 and 3) also included short answers, requiring participants to fill in 10 missing words or phrases. All questions are publicly available on the project website as part of the e-guide.
The online tests were accessed by scanning a quick response (QR) code that opened a learning room on the Socrative platform. The room with questions was only active during the completion of each test and allowed the lecturer to monitor the students’ results in real time. Based on the answers entered, the application calculated the scores. The scores were converted into the levels of career competencies expressed by percent before and after learning the content of the individual modules and transferred into the DECIDE app database. The percentage scores were used to visualise the progress of competency development for each participant and the group.
3.6. Question Types in Pre-Tests and Post-Tests
When developing and testing a web application that measures progress in the development of knowledge and competencies, it is very important to measure knowledge before and after training or self-directed learning. It is desirable that the method of knowledge assessment is as objective and automated as possible and easy to implement [26]. The following types of questions are often used for automatic assessment of answers, where subjective evaluation is avoided:
Short-answer questions (SAQ), where participants write a brief answer in the text box. The system then compares their answers with predefined keywords or phrases.
True/false questions, where participants select whether the statement is true or false. The system can automatically check whether they have chosen the correct answer.
Multiple-choice questions (MCQ), where participants are offered a choice to select the correct answer. The system then checks the option chosen.
Word matching questions, where participants link pairs of words or expressions; for example, link a definition to the correct expression. The system checks that the pairs are correctly connected.
Numerical questions, where participants enter a numerical answer and if the answer is within a specific range, the system marks it as correct.
Fill in the blank space, where participants fill in the missing parts of the sentence with the appropriate word or expression. The system checks to ensure the options are matched. This type is very similar to short-answer questions.
Correct order, where participants sort the items or answers into the proper order. The system checks that the order is correct.
Two types of questions (multiple-choice questions and fill in the blank space) illustrated in Figure 2 were selected for the DECIDE assessing approach to evaluate knowledge, contributing to an efficient self-evaluation without needing a subjective assessment. Additionally, other studies found that using pre-tests and post-tests as multiple-choice questions can improve teaching and learning performance in online courses [27]. They also found that students were more confident of the correct answers after the post-tests and were better at identifying the correct answers than in the pre-tests. In addition, such a pre-test helped students structure and focus their learning, making it easier for them to identify areas of uncertainty and gaps in their knowledge. This suggests that using these methods can help improve the delivery of online course content and identify areas for improvement in teaching, content delivery, and the design of test questions.
4. Results
4.1. Assessing Competencies with Digital Tools
The Socrative app allows users to view live test results. Lecturers can monitor students’ progress and see how they are doing as they answer the quiz. They can track a student’s progress as a percentage of how many questions they answered. Results are also saved and can be exported for further analysis. Special buttons allow anonymity of the students and their answers, for example, if you broadcast the results page to the whole group of participants. From Figure 3, it is possible to see how many students answered each question correctly in Module 4.
Figure 4 shows the progress in developing career competencies for the participants for all modules. Moreover, on average, the group progressed in competencies development in all modules (indicated by circles for class scoring). The highest progress is observed in Module 3 with the title “Potential of circular & green entrepreneurship”, where the initial state in terms of knowledge level was also the lowest. We can see that for Module 3, there is a difference of more than 30% in the average progress in the development of knowledge and competencies for the whole group, and for Module 4, there is a difference of more than 20%. For Module 1 and Module 2, the difference is 12%. Due to the small sample size of 11 participants, statistical analysis of the differences between the pre-test and post-test results was not performed, as the statistical power of such tests would be insufficient to provide meaningful or reliable conclusions. This limitation should be considered when interpreting the results of the study. Despite the lack of statistical significance, the qualitative feedback and observed trends from the participants still provide valuable insights into the tools’ effectiveness.
The accuracy of assessing students’ competencies using pre-test and post-test results can be limited by various factors. Higher scores on a post-test are not necessarily an indicator of students’ progress in knowledge and competency development, as these scores may be related to other effects. It is possible that students simply learnt certain information or concepts but not their application in practice. Testing under controlled conditions can result in students learning material for testing purposes only, but not for sustainable use. Additionally, certain students may be more proficient with the tests or have better memory skills, which may affect test results. When assessing students’ competencies, it is important to consider several factors and different sources of assessment, such as teachers’ evaluations, observations of students during practical activities, mentors’ feedback, and students’ self-evaluation. More on this topic is explained in Section 5 Discussion.
4.2. Determining Future Job Profiles with a Self-Assessment Tool
The future labour market will likely follow current trends and require job profiles beyond the technical skills that young people acquire in formal education. Employers will demand from young people at the end of their educational career transversal and specific competencies related to critical thinking, systemic decision-making, adaptability, digital skills, advances in materials science and technology development, leadership, and active participation. Young professionals will need to explore and understand dynamic labour markets and continuously learn and upskill to meet the changing demands of employers. In relation to these requirements, we surveyed four European countries (Slovenia, Poland, Greece, and Bulgaria) to identify the five most in-demand occupational profiles in the labour market that the world will need most in the post-pandemic age. The selected areas of employability are key to building a more sustainable society:
Information technology (IT) enables efficient management of resources, optimisation of processes, and digitisation and automation of systems, contributing to waste reduction, energy efficiency, and improved environmental standards.
Digital marketing enables businesses to promote sustainable products and services, raise consumer awareness of environmental issues and encourage responsible consumption, which can lead to increased demand for sustainable products and services and stimulate sustainable consumer behaviour.
Energy and environment are key to the development and implementation of renewable energy, innovative green technologies for remote sensing (e.g., terahertz spectroscopy, light detection and ranging (LIDAR), multispectral and hyperspectral cameras, nuclear magnetic resonance spectroscopy, etc.), and measures to reduce environmental pollution and climate changes.
Data sciences play an important role in analysing large volumes of data on the environment, climate change, energy efficiency, and sustainable practices, as well as facilitating the interpretation of analyses through various visual and imaging methods.
Biotechnology enables sustainable food production, the treatment of environmental problems, the improvement of human and animal health, and the development of biodegradable materials.
The most valuable skills and competencies in each job profile, as identified in the project study, are shown in Figure 5. The job profiles are concerned not with one specific job title, but with a professional area of expertise. Thus, Figure 5 summarises several job profiles identified in the project. The figure lists examples of professions for each selected area, such as digital marketing, energy and environment, etc. It highlights the key competencies required for these professions or the competencies that employers expect from candidates seeking employment in these fields.
In Module 5, participants answered a variety of questions about their interests and future career aspirations. Among the 11 participants, almost two-thirds answered in a way that matched their future job profile as a digital marketer. A total of 18% matched the job description for energy and environment specialist and 18% for data scientist. The distribution by future job profiles, based on their interests, educational backgrounds, and career aspirations, is given in Figure 6. The graph displays the results for the students who participated in the training. Two job profiles, i.e., biotechnology expert and IT expert, were not covered at all by the participants’ answers. This matching of participants with job profiles may be because the profession of digital marketer is more visible or more popular among young participants. If they are more familiar with the profession, they are more likely to express an interest in it. Participants may also be unfamiliar with the content or requirements of other professions, such as energy and environment specialists or data scientists. Participants may have been more confident in expressing interests and thinking related to communication and marketing (digital marketing) while feeling less confident or less familiar with other areas. Perhaps participants already gained knowledge or experience in their education related to digital marketing, which makes them more inclined towards this area. However, there is a possibility that participants may simply not be interested in certain profiles, such as biotechnology experts and IT experts, because of their personal preferences and goals.
As mentioned by other researchers, self-assessment brings some benefits, but there are doubts about its value and accuracy [28]. Although self-assessment produces consistent results in a variety of situations, the information generated by self-assessment may only partially correspond to the information usually generated by teachers. Nevertheless, self-assessment contributes to higher student achievement and improved student behaviour, as it is an important part of monitoring one’s learning processes to make adjustments that deepen learning and improve performance. Although it can also be used as a final assessment method, some research has shown that self-assessment is most useful for self-regulated learning supported by training [29]. Nevertheless, further research on the cognitive and affective mechanisms of self-assessment is important to better understand this process and its usefulness in pedagogical contexts.
The distribution of results in our research certainly raises the question of the reliability of such self-assessments without deeper psychological profiling of the individual. While some participants may match their future career profiles based on self-assessment, it is important to remember that self-assessment may not capture all the relevant aspects of an individual’s personality, interests, skills, and abilities that are crucial in career choice. Without additional psychological profiling, including a deeper analysis of personality traits, values, motivations, goals, and other factors, self-assessment may lead to misinterpretation and unreliable results. In addition, it should be kept in mind that an individual’s career choice may also be influenced by external factors such as social norms, environmental influences, job opportunities and personal experiences, which are not necessarily captured in the self-assessment. Therefore, when interpreting the results of questionnaires such as the one in Module 5, caution should be exercised, and potential limitations of self-assessment in the career decision-making process should be considered.
Such self-evaluation questionnaires can serve as a starting point to guide young people through a systematic and critical reflection on their interests, aspirations, values, and educational and career goals. The results obtained from the questionnaires can form the background for exploring different academic and career options for the individual to understand their career preferences in discussions and consultations with mentors or career counsellors. Higher education institutions can use the results of such questionnaires to design and adapt educational programmes that better match the needs and interests of individuals or to offer additional non-formal training that develops in individuals those competencies that they cannot acquire in formal studies but are required by the future labour market. However, when using these results, it is important to be aware of the limitations of such questionnaires and simultaneously use additional methods and tools for career guidance rather than using them as the sole criterion for decision-making.
In using such a self-assessment tool to identify an individual’s most optimal future job profiles, it is important to ensure that the questions asked are as objective, clear, and specific as possible. Asking questions that focus on a variety of topics helps to gain a more comprehensive insight into an individual’s desires, skills, and personal qualities, allowing for a more accurate assessment of which job profile is most suitable for them. In the further optimisation of the self-assessment tool, the project will try to complement the questions by covering different areas such as skills and interests, work preferences, experience and knowledge, personal qualities, goals and motivation, working environment and conditions, global mindset, work–life balance, complexity of tasks, etc. We will also complement the instructions by indicating whether only one or more answers are possible for each question.
5. Discussion
5.1. Empowering Youth Through Self-Directed Learning and Digital Tools
Self-directed learning (SDL) with digital tools is becoming increasingly popular among young people, especially in the context of not in education, employment, or training (NEET) youth. The article by Kõiv and Saks underscores the necessity for SDL competencies as essential for lifelong learning and personal development, especially for youth disconnected from traditional educational or employment pathways [30]. Moreover, the NEET youth are often characterised by being low in motivation and skills and, therefore, require a holistic approach to develop SDL capabilities. NEET youth do not usually participate in education or work-related activities. Thus, SDL for them should be designed to support the development of positive attitudes, self-motivation, and key employability competencies crucial to overcoming their socio-economic challenges.
Based on the results of our study, we can say that the DECIDE project developed an e-guide that helps young people, including NEET, explore different career options by identifying their interests. The content of the DECIDE e-guide enables young people to deepen their knowledge and skills through self-learning, which can help them overcome the challenges of employment in the transition from education to the labour market while building a successful personal and professional career. Kõiv and Saks mentioned that the content of SDLs should be related to the characteristics, unique personalities, and needs of young people [30]. Motivational factors and elements, such as encouraging self-regulation, goal-setting, and problem-solving skills, are essential to help young people overcome the challenges of a rapidly evolving labour market. This finding complements our study’s recommendation that the development of competencies should be supported by focusing on academic knowledge and soft skills such as emotional intelligence, resilience, and adaptability. In this way, young people take part of the responsibility for their career development at the start, which is no longer solely dependent on the prior formal knowledge provided by educational institutions. It also helps individuals to carry out a kind of strengths, weaknesses, opportunities, and threats (SWOT) analysis of oneself, i.e., to discover one’s strengths, weaknesses, opportunities, and limits to success in life.
Furthermore, self-learning through self-evaluation makes it easier for young people to adapt more quickly to constantly changing jobs that require new skills, especially in the context of rapid technological change. Today, where skills are becoming essential, it is important to foster young people’s commitment to lifelong learning. The readiness of individuals to continuously improve their knowledge and skills is a key factor for career progress. During the DECIDE youth training, the participants also obtained an insight into entrepreneurial skills, encouraging young individuals to explore innovative solutions and potentially pursue entrepreneurial ventures. Furthermore, global perspectives and cross-cultural communication in international teams prepare individuals for careers that involve working with diverse teams or in international settings. Participants benefit from personal development through various interactive activities that prompt self-reflection, goal setting, and developing a personal career plan.
5.2. Assessing Career Competencies Development Through Pre-Test and Post-Test Evaluations
By assessing knowledge before and after each e-guide module, the project gained insight into developing career competencies in the individuals who participated in the training. In all four modules, students successfully developed module-specific career competencies. This progress has been more significant for some students, while for others, it remained at the same level as before the training. There is no negative trend suggesting that the content is inappropriate or too complex for the students.
Module 2 and Module 3 also included “fill in the blank space” questions. Students recognised that this type of question is more difficult to answer because they need to know the exact answers. This difficulty is reflected in the results (Figure 4) of the pre-test and post-tests, where, overall, the level of knowledge both before and after the training was lower compared to Module 1 and Module 4, which included only MCQ. The difficulties students encounter with “fill in the blank space” questions compared to multiple-choice questions can be attributed to several factors. For instance, short-answer questions often require the learner to understand the content in-depth.
In contrast, multiple-choice questions may require the learner to identify the correct answer from all the answers given or randomly select the correct answer. Wijk et al. also found that SAQ or “fill in the blank space” questions are more discriminatory than MCQ, which is consistent with the level of difficulty we found in our study [31]. The need for students to generate their answers rather than identify the correct ones may explain why students perform less well on short-answer questions or “fill in the blank space” questions, as they require more profound understanding and recall, which is more difficult. The authors pointed out that MCQs allowing guessing are less reliable in discriminating between students’ proficiency levels. We also found that in multiple-choice tests, learners may randomly select the correct answer, which undermines the depth of the assessment.
Moreover, short-answer questions typically demand critical thinking and the ability to recall information from memory, which can be more challenging than simply recognising the correct answer among given options in multiple-choice questions. One study showed that both short-answer and multiple-choice quizzes can improve retention if the quizzes are followed by feedback [32]. This suggests that both types of questions and timely feedback can still be very effective for long-term retention.
Students must express their answers accurately and demonstrate a higher level of language and communication skills when answering questions requiring a written response. As a result, even a tiny misspelling can lead to a loss of points. Additionally, sometimes, a missing field in the text may also correspond to different words that are correct but were not expected by the person composing the questionnaire when answering the questions. Furthermore, “fill in the blank space” questions are time-consuming, especially if students find it difficult to recall information or express their thoughts concisely. This challenge can negatively affect their overall performance on the assessment and the time taken to complete the quizzes themselves, as was also evident in the implemented test. Participants took at least twice as long to formulate their answers than to select multiple-choice answers.
Moreover, some students may experience test anxiety, which can be heightened when faced with open-ended questions. The fear of not providing a correct or complete answer may impact their performance. Another important factor is the design of short-answer or “fill in the blank space” questions, including the clarity of instructions and the specificity of what is expected, which can influence how well students perform. Unclear questions can confuse the learner when formulating an answer. Finally, students may have been more familiar and comfortable with the format of multiple-choice questions due to its prevalence in standardised testing. Thus, a shift to short-answer questions or “fill in the blank space” questions might require an adjustment in their approach.
The results suggest that including short answers in pre-tests and post-tests may pose challenges for test designers. In any case, such tests need to provide additional explicit instruction on answering “fill in the blank space” questions, providing opportunities for practice, and emphasising critical thinking and knowledge application in all self-learning modules. The choice of the type of questions to be used for the pre-test and post-tests in the self-assessment depends on several factors, such as the learning objectives, the competencies addressed, the type of learning material, the method of knowledge transfer, the learning outcomes (individual seminar, group exercises, practical project, etc.) and the characteristics of the target group involved in the training. Considering that different types of questions have their benefits and limitations, it is most beneficial to include a combination of various kinds of questions in one test to ensure a balanced assessment, stimulate different aspects of learning, and address the different learning styles of the participants. It is also important to provide clear instructions and, where necessary, to add explanations of the correct answers, regardless of the type of question is chosen.
The module content and complexity may influence the observed differences in progress among the modules. For example, the module with the most progress may have contained more relevant content to the participants or may have been presented in a way that allowed for better understanding. It is also possible that the content in that module may have been less complex or more interesting. Additionally, Module 3 started with the lowest initial knowledge level. As participants had more room for improvement in this module, there was a higher percentage of progress compared to modules with higher baseline knowledge.
Furthermore, Module 3 may have addressed topics relevant to the participants’ career goals or personal interests, motivating them to invest more effort and time in learning. This increased motivation likely contributed to their higher levels of engagement, which contributed to better learning outcomes. Participants may be more motivated to overcome challenges and actively participate in some modules’ learning processes and activities.
Moreover, individual learning styles and preferences may differ among trainees; therefore, learning methods must be adapted accordingly. It is important to note that a learning method used in one module may not be equally effective in another. Additionally, the effectiveness of the lecturer in each module can play an important role in the delivery of knowledge, as a more experienced lecturer can engage learners more effectively in the classroom, communicate the material more effectively, and improve learning outcomes. Another factor that could have influenced the results is the timing and sequencing of modules. Participants may have built upon that knowledge more effectively if one module addressed foundational concepts or skills necessary for success in subsequent modules. Furthermore, ongoing feedback and assessment during the training programme and adjustments made based on feedback from Module 1 and Module 2 could have positively impacted the design and delivery of Module 3, which was presented to the participants as the final one. Finally, the approach to assessing competencies varied between modules (e.g., different question types), which may have also influenced the effectiveness of each module.
5.3. Visualising Competency Development Progress
Two examples of competency development progress visualisation in the developed application are presented in Figure 7. In Figure 7A, the pre-test result is marked with a blue line, and the post-test result is marked with a green line. The line graph has the x-axis representing each student by identification (ID) number, and the y-axis shows the competency score as a percentage from 0 to 100. In Figure 7B, the pre-test result is marked in blue, and the post-test result is in orange for the selected module. The concentric circles in the pie chart indicate competency development values from 0% to 100%. Each slice of the pie chart represents the result for an individual student, indicated by the student’s ID number. To enhance clarity, the line graph in Figure 7A allows easy comparison of individual progress before and after the module, visually showing each student’s advancement. Meanwhile, the pie chart in Figure 7B offers a holistic view of the cohort’s results, helping to identify overall trends in competency development across students.
Other visualisation approaches can be used in the online tool to show, for example, how the results are distributed in different categories before and after testing, such as a pie chart showing the proportions of correct answers in different competencies or learning modules. When visualising results, graphics with icons or symbols representing progress (e.g., up arrows for improvement, down arrows for deterioration) can make the results easier and faster to understand. The results can also be displayed as an e-portfolio, where each participant can access a personal profile showing progress in the development of competencies, where they can monitor their results and possible improvements in the areas in which they have been trained.
The progress report allows individuals to reflect on their strengths and areas for improvement and where they can focus on further growth. The progress report serves as positive reinforcement, encouraging participants to stay engaged in their learning journey and continue developing their skills. To this end, the DECIDE project also prepared an individual progress report for each participant, which was sent to their e-mail after the training. Such a report, detailing the development of competencies that the individual developed during their various training courses, can be a valuable addition to a young person’s curriculum vitae (CV). For example, the automated report that an online tool can generate from such self-evaluations of self-learning can be attached to job applications, presented to recruiters at job interviews, or used to find further training opportunities. Such a report provides concrete evidence of the development of competencies and knowledge acquired and the individual’s commitment to lifelong learning. All this provides additional positives when looking for a first job and makes individuals more attractive to potential employers.
Based on the feedback given by the participants, it can be concluded that most participants had a positive experience using the e-guide and the web application to evaluate the progress of their knowledge and skills during self-directed learning. The feedback highlights several strengths and areas for improvement, offering valuable insights into the tools’ effectiveness. Many participants found the process engaging, educational, and interactive, with several emphasising the usefulness of pre-tests and post-tests and the ability to explore new topics. Positive comments included mentions of the appealing design, the opportunity to deepen knowledge, and the enjoyable learning experience. Most participants expressed satisfaction with the application, especially with the results displayed and the possibility of checking pre-tests and post-tests. Participants mentioned that they had learnt a lot and that it had been interesting to participate in the testing. Some participants expressed that they had learned things they had not known before, which shows the programme’s educational value. Some stressed that the topic was interesting, that they enjoyed the learning process, and that they gained new information. Most agreed that the tool worked well, but some users suggested improvements, such as reducing the amount of information presented, adding interactive elements such as games or puzzles to enhance engagement, and clarifying the app’s functionality for first-time users. Some expressed that there was perhaps too much information, suggesting that a more straightforward presentation of the content could increase the effectiveness of the learning. Overall, the experience seems to have been positive, with some suggestions for improvement, especially regarding greater interactivity and clarity of content. This feedback underscores the tools’ potential while providing concrete directions for refining user experience, enhancing interactivity, and ensuring accessibility for diverse user groups.
5.4. The Future Job Sector of Youth
Work values influence young people’s career choices, guiding their attitudes, motivation, and preferences towards future careers [33]. For this reason, career counsellors and educational institutions should include work and personal values in career development programmes to help students align their personal values with their career aspirations. Some research also shows significant differences in career choice-making among students by gender [33]. For example, female students are more likely to face specific challenges such as hesitation, lower self-understanding, and regretting decisions, which may be the result of gender stereotypes and narrower career options compared to their male peers. It is undoubtedly important to consider these challenges in career counselling and to present and provide female students with a broader range of career options than they see for themselves. In addition, self-esteem and self-efficacy are important for decision-making, as students with higher self-esteem and higher self-efficacy can better overcome difficulties in making decisions about their future [34,35]. This is particularly important for students training to become classroom teachers, who play a key role in shaping future generations’ sustainable knowledge, values, and skills. A recent study of Finnish classroom teachers found that improving self-efficacy beliefs related to teaching ethics, values, and systems thinking can significantly improve their sustainability competence, which is transferred to their students [36]. Overall, in career counselling, it is crucial to consider a student’s social background, including their political, economic, and environmental situation, as these factors influence their psychological well-being and ability to make career decisions.
The preference for digital marketing over other job profiles in our study, as reflected in the answers of the training participants, may be influenced by several factors, particularly current trends in the labour market, work values, and the growing importance of digital technologies. Digital marketing is becoming an increasingly important component of business strategies due to the continuous shift towards online platforms and digital consumer behaviour. Younger generations, particularly Generation Z and Millennials, who drive online media, find roles in digital marketing to be exciting opportunities for dynamic future careers. The attraction may also stem from the visibility and accessibility of the profession, with many students more familiar with social media and content creation, which are central to digital marketing. The rise in platforms that blend entertainment, social interaction, and commerce also increases the need for professionals who can navigate these complex ecosystems.
Many articles indicate that digital marketing is an attractive career choice for young people due to its visibility and role in modern economies and alignment with entrepreneurial and creative aspirations. For instance, Makrydakis discusses how digital marketing significantly influences the attraction of students to higher education, making it a field young people recognise as impactful and aligned with their aspirations [37]. Furthermore, Masenya highlights the role of digital marketing as a pillar for job creation and economic growth, especially in the digital economy, making it an appealing career for youth aiming to tackle unemployment [38]. Albab and Munandar explore the involvement of youth in digital marketing to manage community enterprises, illustrating how the integration of technology and marketing appeals to entrepreneurial-minded young individuals while also being supported by government initiatives [39]. Policymakers and educators can use these insights to further develop curricula and training programmes, thus ensuring that young people have the skills and competencies to succeed in this field.
To develop the necessary competencies for less popular occupational profiles such as biotechnology or IT, targeted educational measures should focus on raising awareness of these fields and their relevance to future challenges. In biotechnology, training could include practical modules on sustainable food production, environmental solutions for waste reduction, and biomedical innovations that align with global sustainability goals. In IT, programmes could emphasise advanced coding, cybersecurity, and artificial intelligence applications and highlight their transformative impact on industries. Collaboration with industry leaders, researchers, and universities can provide insights into market needs, and career advice can clarify possible career paths in these fields.
6. Conclusions
Considering the results of the e-guide and web-based application testing, combined with participants’ feedback, this study highlights several positive aspects of the developed DECIDE digital tool. Participants consistently emphasised the clarity of the presentation of the result, demonstrating the application’s effectiveness in visualising competency development progress and learning improvements. They also gained valuable new knowledge, confirming that the self-learning programme has an educational potential for career competency development. Furthermore, participants described the testing process as engaging and enjoyable, underscoring the value of an interactive and game-based learning approach. Notably, the relevance and attractiveness of the tool content were evident from participants’ interest in career choices.
Despite these successes, the study identified areas for improvement to optimise the web-based application and e-guide content. At the beginning of each module, the objectives and expectations should be clearly defined, and participants should clearly understand those expectations. Adjusting the level of complexity, providing clear and simple content, and breaking information into smaller, precise units could address the issue of too much information. Clear and simple instructions to help with initial navigation through the e-guide and app, including short video instructions, could improve the user experience. Increasing interactivity through additional games and integrating gamification features could further motivate participants and enrich the user experience. Additionally, incorporating post-module feedback mechanisms and detailed explanations for incorrect answers in post-tests would provide more valuable learning opportunities. These improvements could further enhance the effectiveness of self-learning and self-assessment and improve the user experience.
The DECIDE tool is particularly beneficial for young individuals between 18 and 25 navigating critical career decisions, considering their skills, interests, and the demands of a changing labour market. Educational institutions can utilise such tools to support students in selecting study fields and professions, while career counsellors can integrate them into guidance frameworks. Using such tools is also appropriate for individuals who want to explore different options in the labour market and those who want to understand their career prospects better. Furthermore, the tool’s value lies in promoting self-directed learning and self-assessment, empowering individuals to take responsibility for their professional and personal growth. It helps individuals carry out a personal SWOT analysis of themselves, i.e., discover one’s strengths, weaknesses, opportunities, and barriers to success. Self-directed learning through self-evaluation is especially crucial in preparing young people for rapid technological changes that redefine the skills in modern workplaces. Lifelong learning is essential, and tools such as DECIDE help instil a mindset of continuous self-improvement, which is a key factor for career progress. In addition to all its advantages, when using a tool such as the DECIDE, it is important to consider its limitations and the impact of various factors on interpreting the results.
Further research should focus on refining the competency development monitoring tool to better support the monitoring of sustainability competency development. Within ongoing Erasmus+ projects (IGNITE, Greenlead), efforts will include integrating sustainability and entrepreneurship into science, technology, engineering, and mathematics (STEM) study programmes, developing learning modules for platforms popular with young people such as TikTok and YouTube, and optimising the tool for tracking competencies such as green leadership. Enhancements such as gamified elements and expanded interactivity will further align the tool with the needs of young users navigating dynamic job markets. Such innovations will enhance self-directed learning and contribute to creating more effective career development tools, enabling educational institutions to serve as key drivers of sustainability and lifelong learning.
Conceptualisation, A.A. and A.Z.; methodology, A.A., M.P., D.K. and V.K.; validation, A.A. and A.Z.; formal analysis, A.A. and A.Z.; investigation, A.A., M.P. and V.K.; writing—original draft preparation, A.A.; writing—review and editing, A.Z., M.P., D.K. and V.K.; visualisation, A.A.; supervision, A.Z. All authors have read and agreed to the published version of the manuscript.
Not applicable.
Not applicable.
All presented data are available at Jožef Stefan International Postgraduate School, which has the project partner role in the DECIDE project. Participation in the survey for training participants was completely voluntary and guaranteed complete anonymity. The data collected was processed and presented at the aggregated DECIDE project level, without the possibility of identifying individual respondents. All data collected (responses) are recorded exclusively in the DECIDE database.
Author Marika Prucnal participated in the DECIDE project under the organization Stowarzyszenie IMPAKT from Kielce, Poland. She is not employed by the organization but collaborates with them under a different type of contract. Stowarzyszenie IMPAKT, which is a non-governmental organization, also served as the project coordinator. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Footnotes
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Enlarge this image.Figure 2. Two examples of questions in pre-tests and post-tests are multiple-choice questions (a) and fill in the blank space (b).
Enlarge this image.Figure 3. Visualisation of the real-time pre and post-test results using the Socrative application.
Enlarge this image.Figure 4. The average score achieved by the whole group of participants in the training, by module, in the pre-tests and post-tests.
Enlarge this image.Figure 6. Distribution of participants according to match with future job profiles.
Enlarge this image.Figure 7. Visualisation of competency development progress in the DECIDE app for different students shown with a line graph (A) and a pie chart (B).
Competencies developed by participants in the training modules were assessed using different methods.
| Self-Learning Module Title | Competency | Assessing Method |
|---|---|---|
| Post-pandemic and green transition of Europe: tackling youth unemployment | Post-pandemic green transition awareness | Pre-test and post-test |
| Future labour market demands: competencies for new job profiles | Career readiness and competency identification | Pre-test and post-test |
| The potential of circular and green entrepreneurship | Circular and green entrepreneurship awareness | Pre-test and post-test |
| Inspirative best practices to follow | Learning to learn | Pre-test and post-test |
| Self-evaluating multi-criteria tool guidelines | Adapting learning | Self-assessment tool |
References
1. Elegbede, I.; Matti-Sanni, R.; Moriam, O.; Emily Osa, I. Sustainability Education and Environmental Awareness. Encyclopedia of Sustainable Management; Idowu, S.; Schmidpeter, R.; Capaldi, N.; Zu, L.; Del Baldo, M.; Abreu, R. Springer International Publishing: Cham, Switzerland, 2020; pp. 1-9. ISBN 978-3-030-02006-4
2. Paul, I. Education for Sustainable Development and Environmental Ethics. Discourses of Globalisation, Ideology, Education and Policy Reforms; Zajda, J.; Majhanovich, S. Globalisation, Comparative Education and Policy Research; Springer International Publishing: Cham, Switzerland, 2022; pp. 159-173. ISBN 978-3-030-71583-0
3. Hays, J.; Reinders, H. Sustainable Learning and Education: A Curriculum for the Future. Int. Rev. Educ.; 2020; 66, pp. 29-52. [DOI: https://dx.doi.org/10.1007/s11159-020-09820-7]
4. Imara, K.; Altinay, F. Integrating Education for Sustainable Development Competencies in Teacher Education. Sustainability; 2021; 13, 12555. [DOI: https://dx.doi.org/10.3390/su132212555]
5. Joint Research Centre (European Commission) Bianchi, G. Sustainability Competences: A Systematic Literature Review; Publications Office of the European Union: Luxembourg, 2020; ISBN 978-92-76-28408-6
6. Nakad, M.; Gardelle, L.; Abboud, R.J. A Systematic Review of the Different Methods Assessing Sustainability Integration in Engineering Curricula. Sustainability; 2024; 16, 4549. [DOI: https://dx.doi.org/10.3390/su16114549]
7. Ahmadov, T.; Karimov, A.; Durst, S.; Saarela, M.; Gerstlberger, W.; Wahl, M.F.; Karkkainen, T. A Two-Phase Systematic Literature Review on the Use of Serious Games for Sustainable Environmental Education. Interact. Learn. Environ.; 2024; pp. 1-22. [DOI: https://dx.doi.org/10.1080/10494820.2024.2414429]
8. Burkšaitienė, N.; Lesčinskij, R.; Suchanova, J.; Šliogerienė, J. Self-Directedness for Sustainable Learning in University Studies: Lithuanian Students’ Perspective. Sustainability; 2021; 13, 9467. [DOI: https://dx.doi.org/10.3390/su13169467]
9. Abina, A.; Darko, K.; Marika, P.; Vaia, K.; Rebeka, K.L.; Aleksander, Z. Online Tools for Developing Young People’s Career Competencies through Personalised Adaptive Learning and Self-Assessment. Proceedings of the 19th Conference on Sustainable Development of Energy, Water and Environment Systems; Rome, Italy, 8 September 2024.
10. Abina, A.; Cestnik, B.; Kovačič Lukman, R.; Zavernik, S.; Ogrinc, M.; Zidanšek, A. Transformation of the RESPO Decision Support System to Higher Education for Monitoring Sustainability-Related Competencies. Sustainability; 2023; 15, 3477. [DOI: https://dx.doi.org/10.3390/su15043477]
11. Abina, A.; Temeljotov Salaj, A.; Cestnik, B.; Karalič, A.; Ogrinc, M.; Kovačič Lukman, R.; Zidanšek, A. Challenging 21st-Century Competencies for STEM Students: Companies’ Vision in Slovenia and Norway in the Light of Global Initiatives for Competencies Development. Sustainability; 2024; 16, 1295. [DOI: https://dx.doi.org/10.3390/su16031295]
12. Finnveden, G.; Egan, E.-D.; Sandberg, T.; Strömberg, E. A Holistic Approach for Integration of Sustainable Development in Education, Research, Collaboration and Operations. Handbook of Theory and Practice of Sustainable Development in Higher Education; Leal Filho, W.; Skanavis, C.; do Paço, A.; Rogers, J.; Kuznetsova, O.; Castro, P. Springer International Publishing: Cham, Switzerland, 2017; Volume 2, pp. 287-303. ISBN 978-3-319-47889-0
13. Moallemi, E.A.; Malekpour, S.; Hadjikakou, M.; Raven, R.; Szetey, K.; Ningrum, D.; Dhiaulhaq, A.; Bryan, B.A. Achieving the Sustainable Development Goals Requires Transdisciplinary Innovation at the Local Scale. One Earth; 2020; 3, pp. 300-313. [DOI: https://dx.doi.org/10.1016/j.oneear.2020.08.006]
14. Obrecht, M.; Feodorova, Z.; Rosi, M. Assessment of Environmental Sustainability Integration into Higher Education for Future Experts and Leaders. J. Environ. Manag.; 2022; 316, 115223. [DOI: https://dx.doi.org/10.1016/j.jenvman.2022.115223]
15. Kılkış, Ş. Sustainable Development of Energy, Water and Environment Systems (SDEWES) Index for Policy Learning in Cities. IJISD; 2018; 12, 87. [DOI: https://dx.doi.org/10.1504/IJISD.2018.089263]
16. Criscuolo, L.; L’Astorina, A.; van der Wal, R.; Gray, L.C. Recent Contributions of Citizen Science on Sustainability Policies: A Critical Review. Curr. Opin. Environ. Sci. Health; 2023; 31, 100423. [DOI: https://dx.doi.org/10.1016/j.coesh.2022.100423]
17. Certomà, C.; Corsini, F.; Rizzi, F. Crowdsourcing Urban Sustainability. Data, People and Technologies in Participatory Governance. Futures; 2015; 74, pp. 93-106. [DOI: https://dx.doi.org/10.1016/j.futures.2014.11.006]
18. Jackman, J.A.; Gentile, D.A.; Cho, N.-J.; Park, Y. Addressing the Digital Skills Gap for Future Education. Nat. Hum. Behav.; 2021; 5, pp. 542-545. [DOI: https://dx.doi.org/10.1038/s41562-021-01074-z] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33707657]
19. Ricard, M.; Zachariou, A.; Burgos, D. Digital Education, Information and Communication Technology, and Education for Sustainable Development. Radical Solutions and eLearning: Practical Innovations and Online Educational Technology; Burgos, D. Lecture Notes in Educational Technology; Springer: Singapore, 2020; pp. 27-39. ISBN 978-981-15-4952-6
20. Gómez-Zermeño, M.G. Massive Open Online Courses as a Digital Learning Strategy of Education for Sustainable Development. J. Sustain. Dev. Energy Water Environ. Syst.; 2020; 8, pp. 577-589. [DOI: https://dx.doi.org/10.13044/j.sdewes.d7.0311]
21. Almulla, M.A.; Alamri, M.M. Using Conceptual Mapping for Learning to Affect Students’ Motivation and Academic Achievement. Sustainability; 2021; 13, 4029. [DOI: https://dx.doi.org/10.3390/su13074029]
22. Liu, S.; Guo, D.; Sun, J.; Yu, J.; Zhou, D. MapOnLearn: The Use of Maps in Online Learning Systems for Education Sustainability. Sustainability; 2020; 12, 7018. [DOI: https://dx.doi.org/10.3390/su12177018]
23. Lukman Kovačič, R.; Virtič, P. Developing Energy Concept Maps—An Innovative Educational Tool for Energy Planning. J. Sustain. Dev. Energy Water Environ. Syst.; 2018; 6, pp. 742-754. [DOI: https://dx.doi.org/10.13044/j.sdewes.d6.0219]
24. Mulder, K.; Desha, C.; Hargroves, K. Sustainable Development as a Meta-Context for Engineering Education. J. Sustain. Dev. Energy Water Environ. Syst.; 2013; 1, pp. 304-310. [DOI: https://dx.doi.org/10.13044/j.sdewes.2013.01.0023]
25. Lee, S.; Jung, J.; Baek, S.; Lee, S. The Relationship between Career Decision-Making Self-Efficacy, Career Preparation Behaviour and Career Decision Difficulties among South Korean College Students. Sustainability; 2022; 14, 14384. [DOI: https://dx.doi.org/10.3390/su142114384]
26. Schiekirka, S.; Reinhardt, D.; Beißbarth, T.; Anders, S.; Pukrop, T.; Raupach, T. Estimating Learning Outcomes from Pre- and Posttest Student Self-Assessments: A Longitudinal Study. Acad. Med.; 2013; 88, pp. 369-375. [DOI: https://dx.doi.org/10.1097/ACM.0b013e318280a6f6]
27. Luetsch, K.; Burrows, J. Certainty Rating in Pre-and Post-Tests of Study Modules in an Online Clinical Pharmacy Course—A Pilot Study to Evaluate Teaching and Learning. BMC Med. Educ.; 2016; 16, 267. [DOI: https://dx.doi.org/10.1186/s12909-016-0783-1]
28. Ross, J.A. The Reliability, Validity, and Utility of Self-Assessment. Pract. Assess. Res. Eval.; 2006; 11, 10. [DOI: https://dx.doi.org/10.7275/9wph-vv65]
29. Andrade, H.L. A Critical Review of Research on Student Self-Assessment. Front. Educ.; 2019; 4, [DOI: https://dx.doi.org/10.3389/feduc.2019.00087]
30. Kõiv, K.; Saks, K. The Role of Self-Directed Learning While Supporting NEET-Youth: Theoretical Model Based on Systematic Literature Review. Int. J. Adolesc. Youth; 2023; 28, 2242446. [DOI: https://dx.doi.org/10.1080/02673843.2023.2242446]
31. van Wijk, E.V.; Janse, R.J.; Ruijter, B.N.; Rohling, J.H.T.; van der Kraan, J.; Crobach, S.; de Jonge, M.; de Beaufort, A.J.; Dekker, F.W.; Langers, A.M.J. Use of Very Short Answer Questions Compared to Multiple Choice Questions in Undergraduate Medical Students: An External Validation Study. PLoS ONE; 2023; 18, e0288558. [DOI: https://dx.doi.org/10.1371/journal.pone.0288558]
32. McDermott, K.B.; Agarwal, P.K.; D’Antonio, L.; Roediger III, H.L.; McDaniel, M.A. Both Multiple-Choice and Short-Answer Quizzes Enhance Later Exam Performance in Middle and High School Classes. J. Exp. Psychol. Appl.; 2014; 20, pp. 3-21. [DOI: https://dx.doi.org/10.1037/xap0000004]
33. Wang, X.-H.; Wang, H.-P.; Lai, W.-Y. Sustainable Career Development for College Students: An Inquiry into SCCT-Based Career Decision-Making. Sustainability; 2023; 15, 426. [DOI: https://dx.doi.org/10.3390/su15010426]
34. Pignault, A.; Rastoder, M.; Houssemand, C. The Relationship between Self-Esteem, Self-Efficacy, and Career Decision-Making Difficulties: Psychological Flourishing as a Mediator. Eur. J. Investig. Health Psychol. Educ.; 2023; 13, pp. 1553-1568. [DOI: https://dx.doi.org/10.3390/ejihpe13090113] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/37754452]
35. Park, J.-H.; Kang, S.-W. College Life Adjustment, Life Stress, Career Barriers, and Career Decision-Making Self-Efficacy of College Students Who Have Experienced COVID-19 in South Korea. Healthcare; 2022; 10, 705. [DOI: https://dx.doi.org/10.3390/healthcare10040705]
36. Karvonen, R.; Ratinen, I.; Kemi, U. Promoting Sustainability Competency and Self-Efficacy in Class Teacher Education. Front. Sustain.; 2023; 4, 1205680. [DOI: https://dx.doi.org/10.3389/frsus.2023.1205680]
37. Makrydakis, N.S. The Role of Digital Marketing in Public Higher Education Organizations in Attracting Younger Generations. Expert J. Mark.; 2021; 9, pp. 28-38.
38. Masenya, T.M. Promotion of Digital Entrepreneurship for Youth Employability in the Post Covid-19 Era. Handbook of Research on Sustaining SMEs and Entrepreneurial Innovation in the Post-COVID-19 Era; IGI Global Scientific Publishing: Hershey, PA, USA, 2021; pp. 122-142. ISBN 978-1-7998-6632-9
39. Albab, B.; Munandar, A.I. The Role of Youth in Managing Village-Owned Enterprises (BUMDES) through Digital Marketing. Managing Learning Organization in Industry 4.0; Routledge: London, UK, 2020; ISBN 978-1-003-01081-4
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Kovačič, Darko 2 ; Prucnal, Marika 3 ; Kiratzouli, Vaia 4 ; Zidanšek, Aleksander 5
1 Jožef Stefan International Postgraduate School, Jamova 39, 1000 Ljubljana, Slovenia;
2 International Institute for the Implementation of Sustainable Development (MIITR), 2000 Maribor, Slovenia
3 Stowarzyszenie IMPAKT, 25-001 Kielce, Poland
4 Institute of Entrepreneurship Development (iED), 412 22 Larissa, Greece
5 Jožef Stefan International Postgraduate School, Jamova 39, 1000 Ljubljana, Slovenia;