1. Introduction
Currently, technology has increased exponentially and is accelerating the ability of companies to be more productive with proper management of resources and obtaining better results. New technological trends include the internet of things and cloud computing. Artificial intelligence and data mining handle large amounts of information in real time about the needs and preferences of customers, which allows companies to make decisions when creating new products and services [1]. However, the exponential increase in the use of these technologies and the amount of information they generate has led to more computer attacks, which threaten the confidentiality, integrity, and availability of companies around the world. According to a Deloitte study, four out of ten organizations in Ecuador, the country where this work is carried out, have suffered security incidents, and 70% of the organizations state that they are not certain about the effectiveness of their response process to cybersecurity incidents [2].
This is because even though many companies have the necessary capital to invest in cybersecurity, there is a great shortage of professionals in it. According to a report published by (ISC)² in 2019, the demand for cybersecurity specialists increased to 4.07 million worldwide [3]. Likewise, the COVID-19 pandemic has reflected a large increase in the number of cyberattacks, which no longer only target people and small companies, but have also been carried out against large companies, governments, and critical infrastructure. An INTERPOL study revealed that in the first four months of 2020, the private sector had approximately 907,000 spam messages, 737 incidents, and 48,000 malicious URLs related to COVID-19. One solution to protect information systems is to have professionals trained in the area. To train them, one method is the use of gamification, interactive activities, and simulations, which allows for improving the performance of the students, the commitment, and the motivation of learning [4,5]. In the area of computer security, a common hands-on learning method is Capture the Flag (CTF) competitions, which consist of teams competing to see who can solve the most security problems within a certain time limit [6]. These types of competitions are held around the world and many platforms are used for this purpose.
This work proposes the development and implementation of a CTF platform that contributes to learning cybersecurity, using theoretical–practical teaching simply and understandably, without the need for prior knowledge. This platform is adjusted to the needs of the population that participates in this study, since most of these tools, being international, imply certain criteria that bias their use, for example, the language, difficulty of the challenges, difficulty understanding the documentation, and paid subscriptions, among others. This problem occurs mainly in those who are beginning to be interested in the subject and would like to acquire a basic knowledge of it. The CTF platform is designed for the improvement of information security skills with practice and challenge; by facing complex and realistic challenges, participants can develop technical and strategic skills that are relevant to the field of information security [7]. In addition, by training the participant in the process of discovering vulnerabilities in different environments, the participants can help identify possible security gaps in real systems. Similarly, the challenges in a CTF platform have often been found to be too difficult for a single participant to solve. As a result, the participants can work as a team to solve the challenges.
This work proposes the design of a training and competition tool in which the participants must solve a series of challenges related to computer security. The novelty in this work focuses on its design and functionalities adapted to the needs of an organization. As innovative features, the CTF platform has a user interface that is easy to use and navigate, which significantly improves the user experience and facilitates the participation of people with different levels of knowledge in computer security. Furthermore, it allows organizers to customize challenges and create new ones in real time, making the competition more interesting and challenging. Another innovative aspect is that the platform can integrate emerging technologies such as blockchain, artificial intelligence, and virtual reality, among others, and offers a unique and attractive experience for participants and organizers. Finally, the available discussion forums, online chats, and collaboration tools help create an active and participatory community [8].
This article is organized as follows: Section 2 reviews the works similar to this proposal, as well as the concepts used, and describes the proposed method; Section 3 presents the results of the investigation and comments on the results obtained; Section 4 makes a comparison between the results obtained in this proposal and the methods proposed in other works; and, finally, Section 5 presents the conclusions.
2. Materials and Methods
For the development of this method, several fundamental parameters are considered that are aligned with the objectives and hypotheses for the design of the CTF platform. The parameters may vary depending on the context in which the platform is used and the needs of the organization. As objectives, the platform must be able to improve the computer security skills of the participants, as well as evaluate the computer security skills that they acquire. Another fundamental aspect is that the platform must promote collaboration and teamwork among the participants and encourage interest in information security and related careers [9]. As a hypothesis, the method establishes that the participants should improve their computer security skills after participating in the CTF platform. In addition, the scores obtained with the CTF will be correlated with the level of competence of the participants in computer security. The challenges in the CTF platform will be realistic enough to identify vulnerabilities in computer systems. The participants can work as a team to solve the challenges in the CTF platform.
In addition, several concepts are used that serve as a basis for the development of the platform. Therefore, an analysis of the most used platforms for CTF competitions is established. This analysis shows its characteristics in terms of design, capacity, technologies used, and user interface, among others [10]. For this study, five training and competence platforms on cybersecurity issues were selected; the platforms considered are FBCTF, CTFd, HackTheBox, PicoCTF, and TryHackMe. These are evaluated using a combination of research criteria, experience acquired in contests in which the authors have participated, and the documentation of each platform.
-
FbCTF is a platform created by the developers of Facebook, to host a Jeopardy or King of the Hill-type Capture the Flag competition. The FBCTF platform is designed considering flexibility and adaptability with different types of facilities, depending on the needs of the end user.
-
CTFd is a platform that is considered one of the oldest and largest CTF in the world called CSAW. This is designed to facilitate the use of both administrators and users. In addition, it has several functions that allow you to carry out a competition successfully. Among its advantages are that it is open source and easy to install and modify [10].
-
HackTheBox allows the generation of a large-scale online cybersecurity training model that allows individuals, companies, universities, and all kinds of organizations to improve their hacking skills. It has a learning platform called HTB academy for CTF and a platform to practice using challenges in controlled environments with vulnerabilities [11].
-
PicoCTF is a free educational platform where young people learn basic concepts of computer security. PicoCTF offers an original and creative way to solve CTF challenges, both for training and competing. This platform was developed by experts in computer security and software from Carnegie Mellon University. The main categories it has are steganography, web, cryptography, reversing, etc.
-
TryHackMe is a platform that teaches cyber security using short labs replicated from the real world. It has content for both beginners and experienced computer scientists. In addition, it has built-in guides and challenges to satisfy different learning styles.
2.1. Literature Review
As reviewed by [12], it was found that many studies have focused on the design and evaluation of CTF platforms. These studies have evaluated the effectiveness of CTF platforms in computer security training and competition, as well as the effectiveness of different features of the platform. For example, one study found that customizing challenges and feedback effectiveness are important factors for improving the learning and motivation of participants in a CTF platform. Other studies have focused on creating challenges suitable for different levels of computer security skills and knowledge. In addition, they have explored how the difficulty levels of the challenges can affect the motivation and participation of the participants in the competition. In recent years, the use of machine learning and artificial intelligence (IA) techniques in CTF platforms has been investigated [13]. These studies have explored how these techniques can improve the challenge creation and security assessment of CTF platforms.
At the educational level, some works [14,15] have explored the use of CTF platforms for computer security education at different levels, from secondary education to higher education. These studies have evaluated how CTF platforms can improve learning and understanding of computer security concepts. Research in this field is very active, and some of these works, such as [16], have investigated how to evaluate the abilities of the participants in CTF competitions. These studies have explored how to measure participant competence, how to design tests of skills, and how to use metrics to assess participant performance. In [17], the authors conducted a systematic review of the existing literature on CTF competitions. The authors identified 52 relevant scientific papers and found that most studies focused on the design and evaluation of CTF competencies as well as the safety of CTF platforms. The authors also pointed out the need for more studies on the impact of CTF competencies in the training of information security professionals.
In works such as [18], the authors evaluated the effectiveness of a CTF competition in learning the fundamentals of computer security. The authors found that the CTF competency significantly improved participants’ knowledge of computer security and recommended that CTF competencies be used as effective training tools in this area. In [19], the authors presented a network security education platform based on CTF competencies. The authors evaluated the effectiveness of the platform on student learning and found that the platform significantly improved students’ computer security knowledge and skills. In [20], the authors explored the use of CTF competencies as a pedagogical approach in cybersecurity education. The authors found that CTF competencies can significantly improve student learning and motivation in this domain.
A detailed review of the strategies used in CTF competitions is presented in [21], as well as a taxonomy to classify the different categories of challenges that can be found on CTF platforms. For its part, [22] describes the implementation of a CTF competition platform for information security education and presents a set of challenges that cover topics from the security of networks to application security. A detailed analysis of the implementation of a CTF competition platform and its use in training computer security students is presented in [23]. The authors also discuss the limitations of CTF competencies and propose some recommendations to improve their effectiveness as training tools. Finally, of the articles considered, [7] describes the implementation of a CTF competition platform for information security education. The authors discuss the platform’s features, including challenge selection, evaluation, and feedback, and present the results of a study evaluating the platform’s effectiveness as a training tool.
2.2. Criteria
Each of the criteria chosen for the analysis of the different CTF platforms is detailed below.
2.2.1. Functionality
This criterion measures the operations and processes of the tool, the accessibility in the use of the platform, and the learning environment. The attributes with the strengths and weaknesses of the platforms are presented in Table 1.
2.2.2. Teaching Facilitators
This criterion is important for the development of this proposal since the objective that arises in the development of this work is to generate learning using a CTF. Therefore, several evaluated characteristics serve as a basis for improving the environment and the learning process [24]. Particularly, this category includes options that help improve learning processes and the presentation of their challenges. The criteria considered and the responses are shown in Table 2.
2.2.3. Challenge Administration
This criterion evaluates how the platforms handle the flags in the competition. The criteria are detailed in Table 3.
2.2.4. Categories
This criterion refers to the categories of challenges and learning of a platform. It is important to highlight that the field of cybersecurity is very broad, and it is necessary to know at least the fundamentals of all existing categories [25]. Table 4 details the existing criteria in each of the platforms.
All the evaluation criteria have been considered to define the functional requirements in the design of a CTF platform, with the capabilities to educate on cybersecurity issues using a competency element such as a Capture the Flag.
2.3. Functional Requirements
Within the functional requirements, it is established that the system can allow the creation of two types of users, the student, and the administrator. Registration to the platform is open with a guided challenge [26]. The system includes a section for CTF skills and another for learning and integrates a repository of writeups. The system allows you to upload learning modules and challenges from the categories of steganography, cryptography, OSINT, Pwn, web, fundamentals, reversing, programming, mobile, miscellaneous, forensics, and trivia. The section for CTF competitions is composed of challenges, scores, graphs, equipment, instructions, and clues. In the cybersecurity learning section, theoretical–practical learning modules, scores, and surveys are integrated [27]. For administration, the platform includes a panel that allows you to upload content for the competition and for learning, as well as manage teams, users, backups, reports, etc.
2.4. Method
For the development of the platform, the Scrum methodology is used, which is a framework that contains good practices, both for teamwork and for obtaining an optimal result. According to the Scrum methodology, the artifacts represent work or value. For this purpose, three main artifacts have been considered including Product Backlog, Sprint Backlog, and Increment.
2.4.1. Platform Development
For the development and implementation of the platform, several technologies recommended by developers for web platforms were selected as well as technologies found in the review of similar works [28,29]. Figure 1 shows the architecture designed for the application.
The programming language used for development, both in the front end and back end, is JavaScript. Its choice is based on its characteristics as a lightweight programming language that does not need a compiler since this is recognized and executed with web browsers directly [30]. For the development of the front end, several libraries are used, such as ReactJS and JavaScript, that allow dynamic interfaces of all kinds and in a very simple way. For the development and implementation of the back end, Node.js is used, which is an environment that works at runtime with the front end. The functionality of this environment is that it allows developers to create all kinds of server-side tools and applications based on JavaScript [31].
Figure 2 shows the flowchart of the module design and how it interacts with the user. In the first stage, the user contacts the home web page where relevant information about the platform is presented. In the second stage, the registration page is presented to create new users; on the access page, authentication data are requested, for which it is necessary to enter a username and password [32]. The system verifies authentication parameters using middleware that verifies the role of the user, which can be an administrator or student. According to the role of the user, the necessary modules are enabled for each user. Every time a user accesses a new page, the system will verify the user’s role again to guarantee the authorization of the platform, this process is represented in Figure 2.
Two types of users and roles have been created on the platform including the administrator and the student. Based on this, a menu and dashboard adapted to the needs of each type of user are shown. The administrator role is presented in the block diagram of Figure 3. This role has at its disposal a dashboard that allows it to manage the platform using various modules such as the configuration of challenges, users, teams, competition, learning, writeups, and global settings [33]. Another important action of this role is the ability to configure, support, and report on student learning. In addition, the administrator has access to all the sections and functionalities of the student to manage, monitor, and control the use and access to the platform [34].
The student role has access to two modules of the platform. The competition module has access to the challenges, scores, and team section [35]. The learning module integrates the learning resources, the user profile, and the writeups, as presented in Figure 4.
2.4.2. Main Sections of the CTF Platform
Among the main sections is the CTF competition section. Its main objective is to present the instructions for the competition and the different existing categories. Within the instructions the total number of challenges is presented, as well as all the challenges solved [36]. In Figure 5, the main interface is presented, in which the user views the challenges section, where each user has their name, score, description, clues, and attachments. In addition, a field is shown to enter the flag and solve the challenge. In this figure, the native language of the application is maintained, and when representing the interface of the application, it is considered important to maintain this characteristic.
Another section of the platform is that of scores. This section presents the scores of the users during a competition according to the progress in the resolution of the challenges. The results are presented using graphs that indicate the top 10 equipment and their scores [13,37]. Users can visualize in detail the challenges they have resolved when clicking on each of the bars presented in the Dashboard, an example is evidenced in Figure 6.
In the Writeups section, a repository with basic information and Writeups Links of challenges for the skills initiated by the students are obtained in order to obtain collaborative learning. The learning section has learning modules divided into categories and levels [38,39]. At the beginning of each competition, the low-level modules are activated, and as the student solves these modules, the intermediate and difficult levels are enabled. Each module is composed of theoretical resources and challenges as a practical part. Script Kiddie to Pro Hacker is contemplated on the cybersecurity platform. In the administration Dashboard section, the user manages all the platform functionalities that manage the CTF and learning competition. In addition, from this section, they manage the backups of information and results reports to validate students learning.
3. Results
The evaluation of the platform was carried out in two stages. In the first stage, 22 participants are considered, and a simulation event is carried out that is considered to determine the adjustments to be made to the platform. In addition, with this first simulation, the operation of the platform is measured. It is worth mentioning that the CTF for the participants is transparent, that is, for the participants of this normal contest. However, for the authors, this first CTF is part of the evaluation of the system. In the second stage, we work with 2 groups with a total of 65 participants. In this CTF, the necessary adjustments to the platform and the contest have been included, and the necessary criteria are established for the evaluation of the participants’ learning.
In the first stage, for the evaluation of the platform, it should be considered that its objective is to improve learning on cybersecurity issues. For this, an evaluation mechanism was applied to the students within the platform. The evaluation consists of a questionnaire of five questions for each level. The population considered for the evaluation is made up of 22 participants. Upon entering the learning section, they solve an initial-level questionnaire. Once the student solves the questionnaire, the platform enables the categories with five content modules. Each time a student completes a module, a questionnaire is presented with questions about the competition and must be resolved on a mandatory basis. As a result of the evaluation, it was found that, on average, the students improved by 24.78% in the easy-level questionnaire compared to the initial questionnaire. With this result, it can be verified that even though the students already have prior cybersecurity knowledge, it was possible to increase it to a higher level.
During the Capture The Flag competition, the resolution of a survey on the use of the platform was applied as part of a challenge. One of the questions refers to the accessibility and usability of the platform. Of the 22 participants, it was found that only 1 student considered the use and interaction with the platform difficult. In addition, 55% of the population responded that the use of the platform is simple; however, 40% of the students assume that the use of the platform is normal. Therefore, it is considered useful training, before the development of competence, to improve the acquisition of knowledge.
Another objective for the design of the CTF platform is that the participants become interested in the world of cybersecurity to encourage them to follow a specialty oriented to said area. For this, a question has been raised that seeks to measure the level of interest of each participant in cybersecurity after using the platform. As a result, it was found that 95% of the participants have an interest in the area because of the use of the web application and its content.
In the second stage, 65 participants are divided into 2 groups (A and B), group “A” is made up of 28 people, and group “B” is made up of 37. There is no specific assignment in the groups, and each group is simply determined using the registration fee. In the registration, the user determines whether to register a person or a team that can be made up of a maximum of four people. For the evaluation of the learning results, each participant has their scores, which depend on the questionnaires that are carried out. The number of questionnaires that each participant must complete is four; the first is completed before starting the CTF. The following questionnaires correspond to each stage and are executed after completing the corresponding level. Table 5 shows the results obtained in group “A”. In the first column, an ID per participant is recorded, from columns two to five, the scores obtained by the participants in each questionnaire are recorded, and column six shows the average scores per user and activity.
As a complement to the previous table, Figure 7 is presented, in which one can graphically observe the progress or stagnation in the progress of each participant. According to the figure, the results obtained using the initial questionnaire are the lowest in the entire process. However, in most cases, substantial progress is observed at each of the levels. In the final questionnaire that the participants complete at the end of the CTF, it can be seen that they present a substantial knowledge of the cybersecurity topics exposed during the contest, where the progress is shown with the yellow line within the figure.
Figure 8 shows the results obtained in group “B”. According to the progress that can be seen in the figure, the learning of the participants in the two groups presents the same trend. That is to say, the designed CTF platform complies with what is established within the objectives and hypotheses raised.
4. Discussion
A CTF is a competition in which participants must solve a series of computer security challenges to find a “flag” [40,41]. Flags can be a text string, a file, a URL, an IP address, etc., and they are usually hidden within computer systems, files, programs, and web pages, among others. CTF competitions are usually organized by computer security groups, universities, or companies, and can last anywhere from a few hours to several days. According to the reviewed works, few CTF platforms can include point tracking features, real-time rankings, user activity monitoring, and tools to create and manage challenges. Although there are relatively few scientific studies specifically addressing the use of CTFs by students, some studies suggest that CTFs may have significant educational benefits [42].
For example, a study published in 2021 [43,44] found that the use of CTF improved student motivation and performance in a computer security course. Students who participated in the CTF also reported higher satisfaction with the course compared to those who did not participate in the CTF. Another study published in 2022 [45,46] found that the use of CTFs can be an effective approach to teaching ethical hacking and computer security skills. The study authors also suggested that CTFs may be an effective way to engage students in information technology education [47].
According to the results obtained from the developed platform, it was identified that a CTF platform is an effective tool to improve the motivation, performance, and commitment of students’ education in information technology and computer security. In addition, during the developed contest, various benefits were identified, such as interactive learning, since the platform developed allows users to learn using challenges and practical problems, which encourages exploration and experimentation. This allows users to learn at their own pace and increase their understanding of computer security and programming concepts [48]. Another aspect is skill development, by offering a variety of challenges at different difficulty levels, allowing users to develop and improve their computer security and programming skills. Additionally, CTF platforms encourage friendly competition and teamwork, which can motivate users to improve their skills and knowledge. Users can participate in competitions and measure themselves against other users, which can help them gauge their progress [49].
With the development of this work, it can be mentioned that a CTF platform is an effective tool to improve skills and knowledge in computer security and programming. These systems offer an interactive learning experience, encourage friendly competition, and are accessible from anywhere in the world. The advantage of developing this work is that it is designed in Spanish considering the needs of the population [50]. This work, being developed at a university in Ecuador, has been fully developed in the native language of the country, and its use at the regional level (South America) has also been proposed to generate international competencies in cybersecurity. Most of the platforms are developed in English, including their resources, which can create a barrier for those who do not speak the language and limit their ability to participate in the CTF and compete effectively.
With the development of a CTF platform, several characteristics have been identified that must be met to generate knowledge in the participants. Therefore, it is important to understand that a CTF platform is a security game in which participants compete to find vulnerabilities in computer systems and exploit them to find flags that are hidden in the system [51]. There are several CTF platforms on the market, each with its strengths and weaknesses. Some of the more popular platforms include CTFd, picoCTF, and HackTheBox. These platforms are designed to be easy to use, scalable, and customizable to meet the specific needs of each CTF competition [7,52]. However, the design of its own CTF platform has considered several factors that make it suitable for our needs and generate several advantages about the mentioned CTF. In the first place, for the design, the purpose of the CTF platform has been determined in such a way that features such as teaching computer security skills to students, generation of internal competitions, and competitions open to the public are integrated [53]. These characteristics in the designed platform create a competitive advantage over other platforms available in the market.
5. Conclusions
There are several parameters to consider in the management of cybersecurity contest platforms. Among these, CTF organizers must consider the linguistic diversity of the participants and take measures to ensure that all participants have access to the resources and challenges necessary, regardless of language. This may include providing translations and materials in different languages, offering language support, and being aware of the language barriers some participants may face.
Several important points can be concluded from the use of CTF platforms, among which it stands out that CTF platforms are a valuable tool for teaching computer security skills. Using challenges and activities on the platform, participants can learn about common vulnerabilities, ethical hacking techniques, and general computer security. Additionally, CTFs are a fun and exciting way to test participants’ computer security skills. Participants can compete as a team or individually, which can encourage collaboration and teamwork. The designed CTF platform is a great way to identify and recruit information security talent. Many companies and organizations use CTF platforms to search for candidates with advanced information security skills.
CTF platforms must be designed to be secure and ensure that the privacy of participants is protected. CTF organizers need to be aware of security vulnerabilities and take steps to protect participants. CTF platforms are a valuable tool for teaching computer security skills, assessing participant skills, and fostering collaboration and teamwork. If designed and used properly, CTF platforms can be a great way to engage stakeholders in cybersecurity and foster their interest in the field.
Even though the results were obtained to guarantee the correct functioning of the CTF platform, as well as the fulfillment of the objectives set out in this investigation, certain limitations have been identified during the process and evaluation of this tool. Among these limitations, it can be mentioned that the CTF platform presented problems in scale to many participants due to technical or resource limitations. In addition, security is a major concern in this development, as a breach in the platform’s security could allow participants to gain unauthorized access or perform malicious actions. Another limitation is the variety of necessary challenges that can be posed to the participants, which can make the competition repetitive and boring. It is important to consider these limitations and those that are identified as specific to each organization when selecting or designing a CTF platform, to ensure that it meets the specific needs of the competition and the participants.
In future work, it is recommended to improve the scalability of the platform using scalable technologies and architectures, such as the use of microservices and horizontal scaling. One can also consider using cloud services to take advantage of the scalability offered by providers such as Amazon Web Services or Microsoft Azure. To improve security, it is recommended to implement additional security measures, such as two-factor authentication, data encryption, and user authentication using virtual private networks. In addition, platform penetration tests must be carried out to identify and fix vulnerabilities. To increase the variety of challenges, different categories and difficulty levels can be incorporated, such as web hacking, cryptography, reverse engineering, and steganography. Thematic challenges can also be included to keep the interest of the participants.
I.O.-G. and R.G. contributed to the following: the conception and design of this study, acquisition of data, analysis, and interpretation of data, drafting of this article, and approval of the submitted version. The authors D.G., S.S.-V. and R.G. contributed to this study by design, conception, interpretation of data, and critical revision. W.V.-C. and I.O.-G. made the following contributions to this study: analysis and interpretation of data and approval of the submitted version. All authors have read and agreed to the published version of the manuscript.
The authors declare no conflict of interest.
Footnotes
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Enlarge this image.Figure 6. Dashboard of the scores section. (In this figure, the native language of the application is maintained, and when representing the interface of the application, it is considered important to maintain this characteristic).
Enlarge this image.Figure 7. Dashboard of scores obtained during the CTF by participant and activity.
Enlarge this image.Figure 8. Dashboard of scores obtained using CTF by participant and activity in group “B”.
Functional criteria measurement table.
| Functionality | FBCTF | CTFd | HTB | PicoCTF | TryHackMe |
|---|---|---|---|---|---|
| Visualization and Competition | Challenge Map and Leaderboard | Challenges, categories, scores | Challenges, categories, scores | Challenges, categories, scores | Challenges, categories, scores |
| Documentation | Gitpage | Gitpage | Platform | Private | Private |
| Language | English | English | English | English | English |
| User classification | No | No | Yes | No | Yes |
| Record | Depends on the organizer | Depends on the organizer | Free and subscription | Open | Free and subscription |
| Learning content | No | No | Yes | No | Yes |
Evaluation analysis of the facilitator’s criteria.
| Facilitator | FBCTF | CTFd | HTB | PicoCTF | TryHackMe |
|---|---|---|---|---|---|
| Advertisements | Interactive ads | Popup window | Popup window | Popup window | No ads |
| Platform customization | Not allowed | Extra pages and changing themes | Not allowed | Not allowed | Not allowed |
| Statistics | Logs and score tables | Logs and tables of scores and graphs | Logs and score tables | Logs and score tables | Logs and tables of scores and graphs |
| Hidden or locked challenges | Do not exist | Sub-challenges, prerequisites, and hidden challenges | Do not exist | Do not exist | Do not exist |
Analysis of the evaluation for the criteria of the characteristics in the CTF platforms.
| Characteristics | FBCTF | CTFd | HTB | PicoCTF | TryHackMe |
|---|---|---|---|---|---|
| Flag Management | Penalty, sensitive case, and clues | Penalty, sensitive case, clues, and multiple flags | Sensitive case and clues | Sensitive case and clues | Penalty and sensitive case |
| Reward for Flags | Points to the team | Points to individual or team | Points to the team | Points to individual or team | Individual points |
| Categories | Yes | Yes | Yes | Yes | Yes |
| Points | Statistical and dynamic | Statistical and dynamic | Statistical and dynamic | Statistical | Statistical and dynamic |
Evaluation of existing platforms in the market according to the “Category” criterion.
| Category | FBCTF | CTFd | HTB | PicoCTF | TryHackMe |
|---|---|---|---|---|---|
| Steganography | x | x | x | ||
| Cryptography | x | x | x | x | x |
| OSINT | x | x | |||
| pwn | x | x | x | x | x |
| Web exploitation | x | x | x | x | x |
| Trivia | x | ||||
| Fundamentals | x | x | x | x | |
| Reverse engineering | x | x | x | x | x |
| Programming | x | x | x | ||
| Mobile Security | x | x | x | ||
| Miscellaneous | x | x | x | ||
| Forensic Analysis | x | x | x | x | x |
Learning results obtained using questionnaires for each level.
| Participant Id | Initial | Easy | Half | High | Average Score |
|---|---|---|---|---|---|
| 1 | 3 | 3 | 2 | 4 | 3 |
| 2 | 1 | 2 | 5 | 5 | 3 |
| 3 | 1 | 3 | 2 | 4 | 3 |
| 4 | 3 | 3 | 2 | 3 | 3 |
| 5 | 2 | 4 | 2 | 5 | 3 |
| 6 | 1 | 5 | 3 | 5 | 3 |
| 7 | 3 | 2 | 5 | 2 | 4 |
| 8 | 4 | 4 | 5 | 5 | 4 |
| 9 | 3 | 3 | 3 | 5 | 4 |
| 10 | 4 | 2 | 5 | 5 | 3 |
| 11 | 1 | 5 | 1 | 1 | 3 |
| 12 | 4 | 5 | 5 | 4 | 4 |
| 13 | 1 | 4 | 4 | 5 | 3 |
| 14 | 1 | 4 | 1 | 3 | 3 |
| 15 | 5 | 2 | 5 | 5 | 4 |
| 16 | 1 | 4 | 3 | 5 | 3 |
| 17 | 1 | 4 | 4 | 4 | 3 |
| 18 | 3 | 2 | 3 | 5 | 4 |
| 19 | 2 | 5 | 3 | 5 | 4 |
| 20 | 2 | 4 | 5 | 5 | 3 |
| 21 | 2 | 5 | 1 | 3 | 3 |
| 22 | 2 | 3 | 4 | 4 | 4 |
| 23 | 3 | 5 | 4 | 5 | 4 |
| 24 | 5 | 4 | 1 | 1 | 3 |
| 25 | 1 | 3 | 5 | 5 | 4 |
| 26 | 1 | 5 | 4 | 4 | 3 |
| 27 | 1 | 2 | 5 | 5 | 3 |
| 28 | 4 | 2 | 4 | 4 | 4 |
| Average score | 2.3 | 3.5 | 3.4 | 4.1 | 3.4 |
References
1. Voortman, L.M.; Franken, E.M.; van Vliet, L.J.; Rieger, B. Fast, spatially varying CTF correction in TEM. Ultramicroscopy; 2012; 118, pp. 26-34. [DOI: https://dx.doi.org/10.1016/j.ultramic.2012.05.002] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/22728402]
2. Wei, M.; Zhou, W.; Xu, F.; Wang, Y. Nanofluidic Behaviors of Water and Ions in Covalent Triazine Framework (CTF) Multilayers. Small; 2019; 16, e1903879. [DOI: https://dx.doi.org/10.1002/smll.201903879] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31599122]
3. Tahir, M.B.; Nabi, G.; Sagir, M.; Rafique, M.; Alrobei, H.; Nawaz, T.; Inayat, A.; Hussain, S.; Naz, G.; Iqbal, K. Role of CTF in Bi2WO6/ZnO photocatalysts for effective degradation and hydrogen energy evolution. Int. J. Hydrogen Energy; 2021; 46, pp. 30606-30614. [DOI: https://dx.doi.org/10.1016/j.ijhydene.2021.02.059]
4. Galaz-Montoya, J.G.; Hecksel, C.W.; Baldwin, P.R.; Wang, E.; Weaver, S.C.; Schmid, M.F.; Ludtke, S.J.; Chiu, W. Alignment algorithms and per-particle CTF correction for single particle cryo-electron tomography. J. Struct. Biol.; 2016; 194, pp. 383-394. [DOI: https://dx.doi.org/10.1016/j.jsb.2016.03.018]
5. Villegas-Ch, W.; Palacios-Pacheco, X. Proposal for a Secure Architecture for the Internet of Things on a Smart Campus. Proceedings of the Advances in Intelligent Systems and Computing; Hangzhou, China, 29–31 May 2021; Volume 1277.
6. Marabini, R.; Carragher, B.; Chen, S.; Chen, J.; Cheng, A.; Downing, K.H.; Frank, J.; Grassucci, R.A.; Heymann, J.B.; Jiang, W. et al. CTF Challenge: Result summary. J. Struct. Biol.; 2015; 190, pp. 348-359. [DOI: https://dx.doi.org/10.1016/j.jsb.2015.04.003]
7. Werther, J.; Zhivich, M.; Leek, T.; Zeldovich, N. Experiences in Cyber Security Education: The MIT Lincoln Laboratory Capture-the-Flag Exercise. Proceedings of the 4th Workshop on Cyber Security Experimentation and Test, CSET 2011; San Francisco, CA, USA, 8 August 2011.
8. Ibad, M.N. Pengembangan Aplikasi Computer Based Test dengan Protokol Two Central Facilities. JISKA (Jurnal Inform. Sunan Kalijaga); 2020; 4, pp. 12-17. [DOI: https://dx.doi.org/10.14421/jiska.2020.43-02]
9. Antonioli, D.; Ghaeini, H.R.; Adepu, S.; Ochoa, M.; Tippenhauer, N.O. Gamifying Education and Research on Ics Security: Design, Implementation and Results of S3. arXiv; 2017; arXiv: 1702.03067
10. Wang, T.; Kailasam, K.; Xiao, P.; Chen, G.; Chen, L.; Wang, L.; Li, J.; Zhu, J. Adsorption removal of organic dyes on covalent triazine framework (CTF). Microporous Mesoporous Mater.; 2013; 187, pp. 63-70. [DOI: https://dx.doi.org/10.1016/j.micromeso.2013.12.016]
11. Tamanna, T.A.; Belal, S.A.; Shibly, M.A.H.; Khan, A.N. Characterization of a new natural fiber extracted from Corypha taliera fruit. Sci. Rep.; 2021; 11, pp. 1-13. [DOI: https://dx.doi.org/10.1038/s41598-021-87128-8]
12. Zhang, K. Gctf: Real-time CTF determination and correction. J. Struct. Biol.; 2016; 193, pp. 1-12. [DOI: https://dx.doi.org/10.1016/j.jsb.2015.11.003]
13. Voortman, L.M.; Stallinga, S.; Schoenmakers, R.H.; van Vliet, L.J.; Rieger, B. A fast algorithm for computing and correcting the CTF for tilted, thick specimens in TEM. Ultramicroscopy; 2011; 111, pp. 1029-1036. [DOI: https://dx.doi.org/10.1016/j.ultramic.2011.03.001]
14. Trickel, E.; Disperati, F.; Gustafson, E.; Kalantari, F.; Mabey, M.; Tiwari, N.; Safaei, Y.; Doupé, A.; Vigna, G. Shell We Play A Game? CTF-as-a-Service for Security Education. Proceedings of the ASE 2017-2017 USENIX Workshop on Advances in Security Education, Co-Located with USENIX Security 2017; Vancouver, BC, Canada, 15 August 2017.
15. Fayyaz, F.; Yar, M.; Gulzar, A.; Ayub, K. First Principles Calculations of the Adsorption of Fluorouracil and Nitrosourea on CTF-0; Organic Frameworks as Drug Delivery Systems for Cancer Treatment. J. Mol. Liq.; 2022; 356, [DOI: https://dx.doi.org/10.1016/j.molliq.2022.118941]
16. Hanafi, A.H.A.; Rokman, H.; Ibrahim, A.D.; Ibrahim, Z.-A.; Zawawi, N.A.; Rahim, F.A. A CTF-Based Approach in Cyber Security Education for Secondary School Students. Electron. J. Comput. Sci. Inf. Technol.; 2021; 7, pp. 777-778. [DOI: https://dx.doi.org/10.52650/ejcsit.v7i1.107]
17. Mansurov, A. A CTF-Based Approach in Information Security Education: An Extracurricular Activity in Teaching Students at Altai State University, Russia. Mod. Appl. Sci.; 2016; 10, 159. [DOI: https://dx.doi.org/10.5539/mas.v10n11p159]
18. Matias, P.; Barbosa, P.; Cardoso, T.N.; Campos, D.M.; Aranha, D.F. NIZKCTF: A Noninteractive Zero-Knowledge Capture-the-Flag Platform. IEEE Secur. Priv.; 2018; 16, pp. 42-51. [DOI: https://dx.doi.org/10.1109/MSEC.2018.2875324]
19. Costa, G.; Lualdi, M.; Ribaudo, M.; Valenza, A. A NERD DOGMA: Introducing CTF to Non-Expert Audience. Proceedings of the SIGITE 2020-Proceedings of the 21st Annual Conference on Information Technology Education; Virtual, 7–8 October 2020.
20. Cviljušac, V.; Brkić, A.L.; Sviličić, B.; Čačić, M. Computer-generated hologram manipulation and fast production with a focus on security application. Appl. Opt.; 2021; 61, B43. [DOI: https://dx.doi.org/10.1364/AO.439004]
21. Yan, Q.; Lai, W.; Wang, Z. Online Experiments Based on the CTF Model for Information Security MOOC Courses. Proceedings of the ICCSE 2021-IEEE 16th International Conference on Computer Science and Education; Lancaster, UK, 17–21 August 2021.
22. Wahyono, I.D.; Saryono, D.; Asfani, K.; Ashar, M.; Sunarti, S. Smart Online Courses Using Computational Intelligence. Int. J. Interact. Mob. Technol. (iJIM); 2020; 14, 29. [DOI: https://dx.doi.org/10.3991/ijim.v14i12.15601]
23. Szedlak, D.; M’Manga, A. Eliciting Requirements for a Student-Focussed Capture the Flag. Proceedings of the Proceedings of 2020 7th IEEE International Conference on Behavioural and Social Computing, BESC 2020; Bournemouth, UK, 5–7 November 2020.
24. Tian, Y.; Chang, J.C.; Fan, E.Y.; Flajolet, M.; Greengard, P. Adaptor complex AP2/PICALM, through interaction with LC3, targets Alzheimer’s APP-CTF for terminal degradation via autophagy. Proc. Natl. Acad. Sci. USA; 2013; 110, pp. 17071-17076. [DOI: https://dx.doi.org/10.1073/pnas.1315110110]
25. Hargreaves, P.R.; Peets, S.; Chamen, W.C.T.; White, D.R.; Misiewicz, P.A.; Godwin, R.J. Improving grass silage production with controlled traffic farming (CTF): Agronomics, system design and economics. Precis. Agric.; 2019; 20, pp. 260-277. [DOI: https://dx.doi.org/10.1007/s11119-018-09633-7]
26. Dey, S.; Bügel, S.; Sorribas, S.; Nuhnen, A.; Bhunia, A.; Coronas, J.; Janiak, C. Synthesis and Characterization of Covalent Triazine Framework CTF-1@Polysulfone Mixed Matrix Membranes and Their Gas Separation Studies. Front. Chem.; 2019; 7, 693. [DOI: https://dx.doi.org/10.3389/fchem.2019.00693]
27. Hussein, M.A.; Antille, D.L.; Kodur, S.; Chen, G.; Tullberg, J.N. Controlled traffic farming effects on productivity of grain sorghum, rainfall and fertiliser nitrogen use efficiency. J. Agric. Food Res.; 2021; 3, 100111. [DOI: https://dx.doi.org/10.1016/j.jafr.2021.100111]
28. Rashmeei, M.; Shekarabi, S.P.H.; Mehrgan, M.S.; Paknejad, H. Assessment of dietary chaste tree (Vitex agnus-castus) fruit extract on growth performance, hemato-biochemical parameters, and mRNA levels of growth and appetite-related genes in goldfish (Carassius auratus). Aquac. Fish.; 2021; 7, pp. 296-303. [DOI: https://dx.doi.org/10.1016/j.aaf.2021.01.007]
29. Laulagnier, K.; Javalet, C.; Hemming, F.J.; Chivet, M.; Lachenal, G.; Blot, B.; Chatellard, C.; Sadoul, R. Amyloid precursor protein products concentrate in a subset of exosomes specifically endocytosed by neurons. Cell. Mol. Life Sci.; 2017; 75, pp. 757-773. [DOI: https://dx.doi.org/10.1007/s00018-017-2664-0] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28956068]
30. Kelly, D.J.; Kelly, A.E.; Aviles, B.N.; Godfrey, A.T.; Salko, R.K.; Collins, B.S. MC21/CTF and VERA multiphysics solutions to VERA core physics benchmark progression problems 6 and 7. Nucl. Eng. Technol.; 2017; 49, pp. 1326-1338. [DOI: https://dx.doi.org/10.1016/j.net.2017.07.016]
31. Bombiński, S.; Kossakowska, J.; Jemielniak, K. Detection of accelerated tool wear in turning. Mech. Syst. Signal Process.; 2021; 162, 108021. [DOI: https://dx.doi.org/10.1016/j.ymssp.2021.108021]
32. Yu, J.; Lee, H.; Kim, H.; Zhang, P.; Lee, D. Simulations of BEAVRS benchmark cycle 2 depletion with MCS/CTF coupling system. Nucl. Eng. Technol.; 2019; 52, pp. 661-673. [DOI: https://dx.doi.org/10.1016/j.net.2019.09.007]
33. Rohou, A.; Grigorieff, N. CTFFIND4: Fast and accurate defocus estimation from electron micrographs. J. Struct. Biol.; 2015; 192, pp. 216-221. [DOI: https://dx.doi.org/10.1016/j.jsb.2015.08.008]
34. Davies, S.; Rohde, U.; Litskevich, D.; Merk, B.; Bryce, P.; Levers, A.; Detkina, A.; Atkinson, S.; Ravindra, V. CTF and FLOCAL Thermal Hydraulics Validations and Verifications within a Multiscale and Multiphysics Software Development. Energies; 2021; 14, 1220. [DOI: https://dx.doi.org/10.3390/en14051220]
35. Zanetti, G.; Riches, J.D.; Fuller, S.D.; Briggs, J.A. Contrast transfer function correction applied to cryo-electron tomography and sub-tomogram averaging. J. Struct. Biol.; 2009; 168, pp. 305-312. [DOI: https://dx.doi.org/10.1016/j.jsb.2009.08.002]
36. Kong, D.; Han, X.; Xie, J.; Ruan, Q.; Windle, C.D.; Gadipelli, S.; Shen, K.; Bai, Z.; Guo, Z.; Tang, J. Tunable Covalent Triazine-Based Frameworks (CTF-0) for Visible-Light-Driven Hydrogen and Oxygen Generation from Water Splitting. ACS Catal.; 2019; 9, pp. 7697-7707. [DOI: https://dx.doi.org/10.1021/acscatal.9b02195]
37. Fernández, J.; Li, S.; Crowther, R. CTF determination and correction in electron cryotomography. Ultramicroscopy; 2006; 106, pp. 587-596. [DOI: https://dx.doi.org/10.1016/j.ultramic.2006.02.004]
38. Kielur, D.S.; Powden, C.J. Changes of Ankle Dorsiflexion Using Compression Tissue Flossing: A Systematic Review and Meta-Analysis. J. Sport Rehabil.; 2021; 30, pp. 306-314. [DOI: https://dx.doi.org/10.1123/jsr.2020-0129]
39. Nikitin, K.; Clifford, I.; Dokhane, A.; Ferroukhi, H. Methodology for CPR estimations of BWR cycle specific transient reload analyses using CTF sub-channel code. Nucl. Eng. Des.; 2022; 389, 111649. [DOI: https://dx.doi.org/10.1016/j.nucengdes.2022.111649]
40. Naheem, M.A. Analysis of Bahrain’s anti-money laundering (AML) and combatting of terrorist financing (CTF) practices. J. Money Laund. Control.; 2020; 24, pp. 834-847. [DOI: https://dx.doi.org/10.1108/JMLC-04-2018-0033]
41. Kunz, M.; Frangakis, A.S. Three-dimensional CTF correction improves the resolution of electron tomograms. J. Struct. Biol.; 2017; 197, pp. 114-122. [DOI: https://dx.doi.org/10.1016/j.jsb.2016.06.016]
42. Karagiannis, S.; Magkos, E. Adapting CTF challenges into virtual cybersecurity learning environments. Inf. Comput. Secur.; 2020; 29, pp. 105-132. [DOI: https://dx.doi.org/10.1108/ICS-04-2019-0050]
43. Fernandez, J.-J.; Li, S. TomoAlign: A novel approach to correcting sample motion and 3D CTF in CryoET. J. Struct. Biol.; 2021; 213, 107778. [DOI: https://dx.doi.org/10.1016/j.jsb.2021.107778]
44. Kucek, S.; Leitner, M. An Empirical Survey of Functions and Configurations of Open-Source Capture the Flag (CTF) Environments. J. Netw. Comput. Appl.; 2020; 151, 102470. [DOI: https://dx.doi.org/10.1016/j.jnca.2019.102470]
45. Lindgren, R.; Morphew, J.W.; Kang, J.; Planey, J.; Mestre, J.P. Learning and transfer effects of embodied simulations targeting crosscutting concepts in science. J. Educ. Psychol.; 2021; 114, 462. [DOI: https://dx.doi.org/10.1037/edu0000697]
46. Araújo, J.L.; Morais, C.; Paiva, J.C. Student participation in a coastal water quality citizen science project and its contribution to the conceptual and procedural learning of chemistry. Chem. Educ. Res. Pr.; 2021; 23, pp. 100-112. [DOI: https://dx.doi.org/10.1039/D1RP00190F]
47. Gunathilaka, T.M.A.U.; Fernando, M.S.D.; Pasqual, H. Identification of the Learning Behavior of the Students for Education Personalization. Proceedings of the International Conference on IoT in Social, Mobile, Analytics and Cloud, I-SMAC 2017; Palladam, India, 10–11 February 2017.
48. Stek, P.E. Identifying spatial technology clusters from patenting concentrations using heat map kernel density estimation. Scientometrics; 2020; 126, pp. 911-930. [DOI: https://dx.doi.org/10.1007/s11192-020-03751-8]
49. Lin, H.-M.; Wu, J.-Y.; Liang, J.-C.; Lee, Y.-H.; Huang, P.-C.; Kwok, O.-M.; Tsai, C.-C. A review of using multilevel modeling in e-learning research. Comput. Educ.; 2023; 198, 104762. [DOI: https://dx.doi.org/10.1016/j.compedu.2023.104762]
50. Turoňová, B.; Schur, F.K.; Wan, W.; Briggs, J.A. Efficient 3D-CTF correction for cryo-electron tomography using NovaCTF improves subtomogram averaging resolution to 3.4 Å. J. Struct. Biol.; 2017; 199, pp. 187-195. [DOI: https://dx.doi.org/10.1016/j.jsb.2017.07.007] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28743638]
51. Bin Ibrahim, A.D.; Ashrofie Hanafi, A.H.; Rokman, H.; Ahmad Zawawi, M.N.; Ibrahim, Z.A.; Rahim, F.A. Comparative Analysis on Student’s Interest in Cyber Security among Secondary School Students Using CTF Platform. Proceedings of the 2020 8th International Conference on Information Technology and Multimedia, ICIMU 2020; Selangor, Malaysia, 24–26 August 2020.
52. Kolegov, K.; Chemushenko, Y. About The Ctf-Computer Security Competitions. Prikl. Disk. Mat.; 2008; 1, pp. 81-83. [DOI: https://dx.doi.org/10.17223/20710410/2/17]
53. Tang, Y.; Huang, H.; Xue, W.; Chang, Y.; Li, Y.; Guo, X.; Zhong, C. Rigidifying induced fluorescence enhancement in 2D porous covalent triazine framework nanosheets for the simultaneously luminous detection and adsorption removal of antibiotics. Chem. Eng. J.; 2020; 384, 123382. [DOI: https://dx.doi.org/10.1016/j.cej.2019.123382]
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Guerra, David 1 ; Sanchez-Viteri, Santiago 2 ; Villegas-Ch, William 1
1 Escuela de Ingeniería en Tecnologías de la Información, Federal Insurance Contributions Act (FICA), Universidad de Las Américas, Quito 170125, Ecuador;
2 Departamento de Sistemas, Universidad Internacional del Ecuador, Quito 170411, Ecuador;