Content area
This paper describes the development of a training module to improve students individual online behaviors. We developed this module to integrate cyber hygiene concepts into a hands-on learning activity where students develop and secure a mobile web application using the Salesforce Developer tool. This new module aims to prepare the next generation of workers by improving cyber hygiene behaviors through an engaging hands-on activity. We hired two students to help create the dialogue and structure of the module in the summer of 2022. Instructors then implemented the module in introductory information systems courses during the 2022-2023 academic year. During the module, each student a) took a survey to establish a baseline of current knowledge and behaviors (pre-survey), b) performed the training module, and с) completed a survey so we could assess knowledge improvement (post-survey). Post-survey results showed that students were satisfied with the assignment, and that the module taught them essential knowledge and tools for improving cyber hygiene behaviors. Three months later, we sent each student a follow-up survey so we could determine behavioral changes. This follow-up survey showed that students improved self-reported behavioral changes, specifically about using multi-factor authentication, identifying phishing messages, assessing social media settings, identifying antivirus and firewall software, backing up data, and updating software. This study demonstrates that students may benefit from this module to improve online behaviors while preparing them to enter the workforce and help organizations, regardless of their work focus.
ABSTRACT
This paper describes the development of a training module to improve students individual online behaviors. We developed this module to integrate cyber hygiene concepts into a hands-on learning activity where students develop and secure a mobile web application using the Salesforce Developer tool. This new module aims to prepare the next generation of workers by improving cyber hygiene behaviors through an engaging hands-on activity. We hired two students to help create the dialogue and structure of the module in the summer of 2022. Instructors then implemented the module in introductory information systems courses during the 2022-2023 academic year. During the module, each student a) took a survey to establish a baseline of current knowledge and behaviors (pre-survey), b) performed the training module, and с) completed a survey so we could assess knowledge improvement (post-survey). Post-survey results showed that students were satisfied with the assignment, and that the module taught them essential knowledge and tools for improving cyber hygiene behaviors. Three months later, we sent each student a follow-up survey so we could determine behavioral changes. This follow-up survey showed that students improved self-reported behavioral changes, specifically about using multi-factor authentication, identifying phishing messages, assessing social media settings, identifying antivirus and firewall software, backing up data, and updating software. This study demonstrates that students may benefit from this module to improve online behaviors while preparing them to enter the workforce and help organizations, regardless of their work focus.
Keywords: Cybersecurity, Introductory course, Cyber hygiene, Security education, Computer literacy, Teaching tip
1. INTRODUCTION
Companies are routinely under attack through viruses, password hacks, and phishing attempts from outside and inside threats (Cain et al., 2018). Unfortunately, most organizational training to strengthen employees" knowledge and safety behaviors has been ineffective. Helpful behaviors, known as cyber hygiene, are conceptually defined as "the cyber security practices that online consumers should engage in to protect the safety and integrity of their personal information on their Internet-enabled devices from being compromised in a cyberattack" (Vishwanath et al., 2020, p. 2). Providing current students With a security education, training, and awareness (SETA) module will better prepare them to enter the workforce - and help organizations, regardless of their major.
Researchers have advocated for security training to be theory-based (Puhakainen & Siponen, 2010). Using a game-like approach to SETA improves the effectiveness of the training and the participants are much happier with the process compared to a more traditional pedagogical approach (Baxter et al., 2016). While our approach is not a game, we aimed to make the module fun and engaging (i.e., game-like).
The present research and education program aimed to answer the question: How can we prepare the next generation of students and employees to understand the importance of effective and appropriate cyber hygiene?
To answer this question, we developed a training module based on cyber hygiene research to improve the cyber hygiene of the next generation of workers and leaders. Hill and Nance (2016) have developed six labs to integrate Information Systems (IS) concepts into a series of hands-on activities. Using these labs, students gained valuable skills and reinforced course concepts through an innovative activity (Sclarow et al., 2024). However, one important activity, cybersecurity, is missing from the labs.
We created this module focused on cybersecurity using the Salesforce Developer tool. We chose this tool for three reasons. First, the platform is free to use. Second, the students learn about Salesforce and other Customer Relationship Management products in the course, giving them hands-on experience they can add to their resumes. Third, because Hill and Nance's labs were also developed in Salesforce.
2. MODULE DESCRIPTION
2.1 Development of the Salesforce Module
We gathered the necessary requirements for the module from previous research on cyber hygiene. Then we hired two students who were familiar with Salesforce Developer and cyber hygiene concepts to help us write a step-by-step module that integrated cyber hygiene concepts while guiding students through the activity. Consistent with Hill and Nance's Salesforce labs, the module was "written" by a fictional college student in an informal blog post.
We also created surveys to measure the module's effectiveness. At the beginning of the module, students completed a pre-survey to establish a baseline for their current cyber hygiene knowledge and behaviors. After completing the module, they filled out the post-survey, furnishing data about their satisfaction with it and measuring their knowledge improvement. Three months later, we sent a follow-up survey to measure whether their knowledge and behaviors improved or regressed over time. This follow-up survey aimed to measure actual behavior changes rather than behavioral intention.
2.2 Module Components
The module begins with an overview of the Salesforce Developer platform and how to create an account. The platform has a drag-and-drop interface and does not require coding, so students do not need technical skills to complete it. The students then begin the journey of developing a Customer Relationship Management (CRM) system for a fictional company while learning important cyber hygiene concepts. The learning objectives include the following: understanding what cyber hygiene means, the importance of password strength and length, the importance of using a password manager, identifying and protecting against phishing scams, backing up data, how to manage antivirus and firewall settings, managing and improving personal computer security, and managing social media privacy settings. We integrated these key components as they stem from previous cyber hygiene research (see Cain et al., 2018; Kalhoro et al., 2021; Neigel et al., 2020; Parsons et al., 2017; Such et al., 2019; Vishwanath et al., 2020).
In the introductory portion of the module, students learn about malware, types of viruses, phishing, and ransomware and how to protect themselves from these attacks. The remainder of the module explores these concepts in-depth. First, students assume the role of a systems administrator and then create password policies for new user accounts. With these policies, students learn how to enable multi-factor authentication for accounts they create in the Salesforce platform. This helps the students understand the importance of using a password manager.
Next, they set up a new user in the system who will receive these password policies. The user they create will be a fictional victim whom the student will later attack. They then create a phishing message within the Salesforce administrator interface and send the phishing email to their new victim user. By logging into the email account of their victim user, they see the phishing message, Which helps them understand how to identify phishing messages and how easy it is for attackers to perform massive phishing attacks.
The students then learn about ransomware and how they can protect themselves from ransomware attacks (e.g., backing up files to external drives or the cloud). They also learn to identify their built-in antivirus software (e.g., X-protect on Mac; Defender on Windows) and firewalls. The module encourages students to use third-party software for antivirus and firewalls to add an extra layer of protection. The module also walks students through how to find out whether automatic updates are enabled for their applications and operating systems.
Next, students learn about protecting their web browsing behavior. The module walks students through turning on a popup blocker, clearing cookies, identifying an SSL (Secure Sockets Layer) connection on the browser, and enabling incognito or private browsing. Students also learn that using a stronger web browser, such as Brave or Vivaldi, includes all these secure settings without needing to be manually configured. Once students learn that ads no longer appear at the beginning of YouTube videos, they are often eager to adopt these behaviors.
Next, the module discusses how encryption helps protect the user. It encourages students to use a Virtual Private Network (VPN) when using public Wi-Fi and suggests vendors such as Surfshark or Proton. Finally, it encourages students to improve their privacy settings on social media, such as turning on multifactor authentication (MFA) and removing from social media connections people they do not know and trust personally. See Appendix B for the complete module.
2.3 Module Deployment
The authors deployed the module at the University of Colorado at Colorado Springs in undergraduate classes of varying levels in the College of Business. The first level was the equivalent of an Introduction to Information Systems course, which mainly consisted of third-year students. The second level to adopt the module was an introductory course for first year and transfer students, focusing on business applications such as Excel, PowerPoint, Access, and Outlook. Instructors could assign the module for points in the course at their discretion, either as a regular assignment or as extra credit. Instructors deployed the module in six sections of these courses in the 2022-2023 academic year.
Students took another brief survey that captured their satisfaction with the module and indicated whether their knowledge improved because of the module. After the survey, students received a code to submit to attain credit for the assignment. The module also reminded them they would receive a follow-up survey three months later. If they completed that survey, we entered them into a drawing for a $25 Amazon gift card. All measures from the surveys are provided in Appendix A.
3. RESULTS
This section presents the results of the three surveys.
3.1 Pre-survey
Initially, 160 students completed the module, although sixteen students did not complete the pre-survey completely, giving us 144 valid responses. The pre-survey allowed us to establish a baseline for individual cyber hygiene knowledge and behaviors and to capture demographics. The mean age of participants was 21.89 years (SD = 4.26; range = 18 to 45 years), with a mode of 19 years (20.8%). 89 students identified as a man (61.8%), 50 as a woman (34.7%), three preferred not to say (2.1%), and two as non-binary or third gender (1.4%). The race and ethnicity distribution included 93 white or Caucasian (64.6%), 19 Hispanic or Latinx (13.2%), 12 Asian or Pacific Islander (8.3%), eight multiracial or biracial (5.6%), five who preferred not to say (3.5%), three black or African American (2.1%), two Native American or Alaska Native (1.4%), and two identified as a race/ethnicity not listed (1.4%).
We also asked students which operating system they use, and results showed that 79 used Windows (54.9%), 59 used MacOS (41.0%), four chose another/more than one operating system (2.8%), and 2 did not know (1.4%). The highest level of education achieved included 93 with some college but no degree (64.6%), 36 with an associate degree (25.0%), 11 with a high school diploma, GED, or less (7.6%), three preferred not to say (2.1%), and one other (0.7%).
Lastly, we asked students about their current cyber hygiene knowledge and behaviors. Knowledge responses ranged from strongly disagree to strongly agree, while the behavior responses ranged on a scale from never to always. For both question types, we also included an option of "do not know/understand." Unfortunately, most cyber hygiene scales focus on general or workforce samples (Cain et al., 2018; Parsons et al., 2017; Vishwanath et al., 2020). Therefore, we adopted multiple cyber hygiene measures to fit the context of a student sample. We assess these results together with the follow-up survey results later in this section.
3.2 Post-Survey
The post-survey asked students to evaluate their learning, their satisfaction with the hands-on nature of the module versus traditional learning methods, and how well their understanding of cyber hygiene improved because of the module. Using a fivepoint Likert-type scale, we adapted the training effectiveness scales from Tan et al. (2003), with each question ranging from strongly disagree to strongly agree. The questions fall into categories of a general evaluation (six questions, Cronbach's alpha reliability = 0.89), the hands-on knowledge and tools for improving their cybersecurity behaviors (three questions, alpha = 0.94), the level of understanding gained (three questions, alpha = 0.76), and the level of improvement in cyber hygiene knowledge (four questions, alpha = 0.84). The questions, categories, Cronbach's alpha (reliability), means, and standard deviations are shown in Table Al in Appendix A.
3.3 Follow-Up Survey (3 Months After Post-Survey)
The follow-up survey was used to measure the long-term impact of the cyber hygiene module. It contained the same questions as the pre-survey except for demographics. To incentivize the completion of the optional follow-up survey, we entered students into a drawing to receive a $25 Amazon gift card-ten students received a gift card for their participation. Although only 39 of 144 students (27%) completed the followup survey, we could still capture results using a paired samples t-test measuring improvements between the pre-survey and follow-up survey.
We observed the following statistically significant selfreported behavioral improvements after three months and present them in Table 1 below:
We did not identify any significant differences in the opposite direction, although password knowledge 2 (р = .058) was close, and a few other items did have opposite effects.
The complete results are shown in Table 2, which contains the expected direction of the survey questions (whether the improvement is expected to decrease or increase between the pre-survey and follow-up survey), means, standard deviations, significance, and whether the result was in the expected direction or the opposite direction. Note that the comparisons between the pre-survey and follow-up survey only assessed differences for those who participated in the follow-up survey, so the t-test only included a sample size of 39. In the follow-up columns, the mean and standard deviation are included for both the pre-survey (top of each cell) and the follow-up survey (bottom of each cell). We will discuss these results in-depth in the next section.
4. DISCUSSION
The results of our newly deployed Salesforce module showed an overall improvement in cyber hygiene, indicating the effectiveness of the module for this group of students. The post-survey showed that students were highly satisfied with the module, and they would recommend other students participate in the module. The students felt they received knowledge and tools to help them in the future and that they learned how to protect themselves from online threats. They also appreciated this method of learning and sensed they became knowledgeable about the landscape of cybersecurity because of the module. Thus, the game-like module was successful in engaging students in this hands-on learning activity.
The follow-up survey revealed that, after three months, students were surprisingly more likely to think it is acceptable to share passwords with colleagues, classmates, and friends, indicating their knowledge of password sharing decreased. While this finding was disappointing, we were encouraged that their behavior improved regarding sharing their passwords, albeit at a non-significant level. One reason why this could have happened is that the lab focused on other password issues, such as password strength and MFA, but insufficiently on password behavior. This misalignment is interesting and could warrant further research.
Students demonstrated improved knowledge concerning using stronger and longer passwords although this behavior improvement was non-significant. One significant password behavior showed that students improved at enabling MFA for logins, meaning students may be more willing to accept MFA requirements in school and when they are in the workforce.
We did not see any significant changes regarding email use knowledge. However, one behavior improved regarding clicking links from unknown email senders. This indicates that the module helped students become more aware of the prevalence and impact of phishing tactics. We did not observe any significant changes in internet use knowledge or behavior, nor with mobile device behavior, which was disappointing but not surprising. This may indicate their current knowledge and behavior were already strong.
One major improvement that we aimed for was in social media usage, and we were pleased to learn that students improved in knowledge of the importance of periodically reviewing privacy settings on social media accounts. Unfortunately, their behavior did not match this improvement. Two social media behaviors did improve; as they now consider possible negative consequences of posting on social media, and they further assess the authenticity of social media friends and information requests. This implies that students are more cognizant of how they may be perceived on social media, particularly by employers or other authority figures. They also will be less likely to be connected to malicious actors or bots posing as legitimate connections.
Next, we saw improvements in device protection and backup behaviors. Regarding their devices, they improved by knowing they have a firewall running on their computer, they block web browser ads more often, and they now either block or regularly clear cookies on their web browser. By taking these actions, their devices will be less vulnerable to attacks and can improve their online anonymity and privacy (Cain et al., 2018). Moreover, their understanding of the importance of maintaining cloud backups improved. This will help students to keep their files long-term, even when they switch to new devices, while also protecting themselves from ransomware attacks (Vishwanath et al., 2020). Moreover, other researchers have published studies since the implementation of our training module, so future research may refer to them for measures related to cyber hygiene knowledge, awareness, and behaviors (Barakovi & Barakovic Husic, 2023).
Last, one of the easiest ways for attackers to gain access to systems is when their victims do not update their software. We saw significant improvement with students updating their devices to ensure they have the latest operating system updates, software updates, patches, and antivirus software. This will decrease the chances of students getting viruses and their devices being a target for denial-of-service attackers trying to build a zombie farm (Cain et al., 2018; Vishwanath et al., 2020).
While impactful, we recognize several limitations of the study: the module was only implemented for undergraduate college of business students, the survey method in general, and the small sample size for the follow-up survey. For example, the module was written in a way that engaged undergraduateaged students, but it is unclear whether and to what extent this training may be appropriate for workplace education, training, and awareness programs.
Also, the College of Business has courses that are focused on technology or innovation, but students from other colleges, especially those from colleges with less focus on technology, may benefit more from the training. In addition, the module does not account for other courses or training students are receiving. Thus, in the three-month period between the module and the follow-up survey, students may have been exposed to other improvement factors unrelated to this module. Future research will investigate the effects of modes of delivery, such as performing the module for points (i.e., an assignment), where students may be more engaged versus extra credit, where students may not pay attention. These differences may show that the mode of delivery has an impact on module effectiveness.
Last, we saw limited improvement in items in the followup survey, such as password sharing. This could indicate the module may have limitations in effectively addressing all aspects of password security. While more depth could enhance password understanding, it could also lengthen the module. As such, we encourage instructors to go beyond the module for password depth. Because the surveys are self-report only, it may be useful to measure behaviors using a tracking or inventory tool on participant computers. For example, Esparza et al. (2020) introduced a knowledge-attitude-behavior selfassessment framework, which could be useful for accounting for human factors when designing cyber hygiene questionnaires. Some of these limitations could be resolved by future research with larger and more diverse samples and longer-term follow-up data collection.
5. CONCLUSIONS AND FUTURE DIRECTIONS
The purpose of this project was to improve current cyber hygiene knowledge, awareness, and behaviors through an engaging, game-like, hands-on learning activity developed and evaluated for student learners. The module we developed appears to be an effective way to engage students while improving their behaviors, as seen in the high self-reported satisfaction and improvement questions in the post-survey (see Table Al in Appendix A). While not all student knowledge and behaviors improved, we observed improvements in several important behaviors such as using MFA, recognizing and protecting against phishing messages, assessing their social media settings, identifying protective software (e.g., antivirus and firewall) on their devices, backing up their data, and updating their software. This will allow students to be safer while using online systems and be better prepared to enter the workforce as employees who understand and value the importance of protecting an organization's systems.
This module targets undergraduate students, and as such, the language and context are currently too informal to target graduate students or professional employees. We are in the process of adapting the language for a more professional audience and hope to provide certified micro-credentials for those who complete it.
Finally, if anyone is interested in using the cyber hygiene module in a course, the entire module is included in Appendix B. Survey components have been removed from the module, so only the cyber hygiene components remain, which take approximately one hour to complete. As one hour may not be significant enough to improve knowledge, attitudes, and behaviors, we also included possible discussion questions and how to deploy this in your learning management system in Appendix C. The module works well prior to cybersecurity and privacy units in information systems courses but can be applicable to introductory cybersecurity courses as well.
6. ACKNOWLEDGMENTS AND FUNDING
We would like to thank Audrey Bloomquist and Arianna Russell, two UCCS College of Business students we hired to help develop the dialogue and content for this training module.
This project was funded by the UCCS Cybersecurity Programs Office and the Office of Research through a Cybersecurity Seed Grant. The grant covered the costs of the student workers, a stipend for one author, travel funds for another author to present this work at a conference, and gift cards for participants.
Next, this module was developed on the foundation of Hill and Nance's (2016) Salesforce Max Labs project, out of San Jose State University.
AUTHOR BIOGRAPHIES
David Kocsis holds a Ph.D. in information technology with a concentration in information systems from the University of Nebraska at Omaha and is an assistant professor of information systems (IS) at the University of Colorado at Colorado Springs. He teaches courses in Networking, Introductory Information Systems, and the capstone IS projects course. His research interests include collaboration science, social issues in IS, cybersecurity threats, and security education, training, and awareness. He had more than 15 years of industry experience in information technology, networking, and cybersecurity before pursuing academia in 2012.
Morgan Shepherd holds a Ph.D. in information systems and is a full professor at the University of Colorado at Colorado Springs. He teaches courses in Networking, Information Systems Literacy, the capstone IS projects class at the undergraduate level, and Information Systems at the graduate level, both on-campus and online. He has over ten years of industry experience, most of which came at IBM.
Daniel L. Segal is the Kraemer Family Professor of aging studies and professor of psychology at the University of Colorado at Colorado Springs. His program of research focuses on the assessment of psychopathology among older adults, the expression and measurement of anxiety in later-life, suicide risk and resilience and aging, - and the impact of personality disorders across the lifespan. He is also interested in cyberpsychology. He is a Fellow of the Gerontological Society of America and of the American Psychological Association (Division 12 and Division 20). He has published over 200 peerreviewed journal articles and book chapters and 6 professional books.
7. REFERENCES
Barakovic, S., & Barakovi Husic, J. (2023). Cyber Hygiene Knowledge, Awareness, and Behavioral Practices of University Students. Information Security Journal: A Global Perspective, 32(5), 347-370. https://doi.org/10.1080/19393555.2022.2088428
Baxter, R. J., Holderness Jr, D. K., & Wood, D. A. (2016). Applying Basic Gamification Techniques to IT Compliance Training: Evidence From the Lab and Field. Journal of Information Systems, 3003), 119-133. https://doi.org/10.2308/isys-51341
Cain, A. A., Edwards, M. E., € Still, J. D. (2018). An Exploratory Study of Cyber Hygiene Behaviors and Knowledge. Journal of Information Security and Applications, 42, 36-45. https://doi.org/10.1016/j.jisa.2018.08.002
Esparza, J., Caporusso, N., & Walters, A. (2020). Addressing Human Factors in the Design of Cyber Hygiene SelfAssessment Tools. In I. Corradini, E. Nardelli, & T. Ahram (Eds.), Advances in Human Factors in Cybersecurity (pp. 88-94). Springer International Publishing. https://doi.org/https://doi.org/10.1007/978-3-030-525811.12
Hill, T., & Nance, W. (2016). Innovating Business Systems Labs for Engaging Igeneration Students. AMCIS 2016 Proceedings, 29<https://aisel.aisnet.org/amcis201 6/ISEdu/Presentations/29
Kalhoro, S., Rehman, M., & Shaikh, F. (2021). Extracting Key Factors of Cyber Hygiene Behaviour Among Software Engineers: A Systematic Literature Review. IEEE Access, 9, 99339-99363. https://doi.org/10.1109/ACCESS.2021.3097144
Neigel, A. R., Claypoole, V. L., Waldfogle, G. E., Acharya, S., & Hancock, G. M. (2020). Holistic Cyber Hygiene Education: Accounting for the Human Factors. Computers & Security, 92, 101731. https://doi.org/10.1016/j.cose.2020.101731
Parsons, K., Calic, D., Pattinson, M., Butavicius, M., McCormac, A., & Zwaans, T. (2017). The Human Aspects of Information Security Questionnaire (Hais-Q): Two Further Validation Studies. Computers & Security, 66, 4051. https://doi.org/10.1016/j.cose.2017.01.004
Puhakainen, P., & Siponen, M. (2010). Improving Employees' Compliance Through Information Systems Security Training: An Action Research Study. MIS Quarterly, 34(4), 757-778. https://doi.org/10.2307/25750704
Sclarow, S., Raven, A., & Doyle, M. (2024). Teaching Tip: Leveraging Learning Strategies at Scale-Big and Small Changes in a Big IS Course. Journal of Information Systems Education, 35(1), 1-13. https://doi.org/10.62273/FLSR7630
Such, J. M., Ciholas, P., Rashid, A., Vidler, J., & Seabrook, T. (2019). Basic Cyber Hygiene: Does It Work? Computer, 52(4), 21-31. https://doi.org/10.1109/MC.2018.2888766
Tan, J. A., Hall, К. J., & Boyce, С. (2003). The Role of Employee Reactions in Predicting Training Effectiveness. Human Resource Development Quarterly, 14(4), 397-411. https://doi.org/10.1002/hrdq.1076
Vishwanath, A., Neo, L. S., Goh, P., Lee, S., Khader, M., Ong, G., & Chin, J. (2020). Cyber Hygiene: The Concept, Its Measure, and Its Initial Tests. Decision Support Systems, 128, 113160. https://doi.org/10.1016/j.dss.2019.113160
Copyright EDSIG 2025