Content area
Universities and educators need to stay abreast of the rapidly changing job market to prepare students to be market ready. There is a growing disconnect between the education higher institutions offer and the skills employers seek. In this paper, through a content analysis of job advertisements using data mining techniques, we identify patterns underlying technical competencies for business jobs by discipline and location. We provide both a visualization as well as a descriptive analysis of our findings. Our findings highlight sought after technical skills such as programming languages and data analytics tools, as well as the regional variation in job availability. The findings underscore the importance of updating university curricula to align with market demands, providing actionable insights for students, educators, employees, and policymakers.
ABSTRACT
Universities and educators need to stay abreast of the rapidly changing job market to prepare students to be market ready. There is a growing disconnect between the education higher institutions offer and the skills employers seek. In this paper, through a content analysis of job advertisements using data mining techniques, we identify patterns underlying technical competencies for business jobs by discipline and location. We provide both a visualization as well as a descriptive analysis of our findings. Our findings highlight sought after technical skills such as programming languages and data analytics tools, as well as the regional variation in job availability. The findings underscore the importance of updating university curricula to align with market demands, providing actionable insights for students, educators, employees, and policymakers.
Keywords: Text mining, Business intelligence, Technical skills, Content analysis, Curriculum revision
1. INTRODUCTION
As industries and technologies evolve, so do the nature of jobs and the way they are performed; thus, the skills required for those professions change. This necessitates re-evaluating the base-level skill needs on a regular basis (Wierschem & Méndez Mediavilla, 2018). Universities and educators need to stay abreast of the rapidly changing job market to prepare students to be market-ready (Aoun, 2017). There is a growing disconnect between the education higher institutions are offering and the skills employers are seeking (AACU, 2018; Behn et al., 2012; Borner et al., 2018; Donovan et al., 2022; Nguyen et al., 2020; Radovilsky & Hedge, 2022; Smith & Ali, 2014). Universities must introduce rapidly growing and highly in demand technologies to satisfy organizational demands and educate their students for a world in which they efficiently integrate new skills such as Artificial Intelligence (AI) with human intelligence (Topi, 2019). This is often alluded to as the "expectations" and "skills" gap referring to what are considered important skills by industry and what is taught in the curriculum. While this gap covers a diverse set of soft and hard skills, however, nowhere is this gap more evident than as it relates to technical competencies that are rapidly evolving because of the digital revolution. For example, in addition to identifying the critical skills required by the analytics job market, various studies address the gaps between job market requirements and capabilities taught in analytics programs and courses (Nguyen et al., 2020; Radovilsky & Hegde, 2022). Rapid advances in automation and AI, for example, are redefining job responsibilities and the very nature of work. In response to the growing technological advancement and applications of AI, numerous business schools have incorporated AI topics into their curricula, primarily considering Al as a natural progression of data analytics and decision-making tools within their business analytics programs (Davenport, 2018). For instance, some business schools have integrated machine learning, neural networks, and automation into several business courses such as accounting analytics, marketing analytics, and financial technology (Chen, 2022).
In this paper, through a content analysis of job advertisements using data mining techniques, we identify patterns underlying technical competencies for business jobs by discipline and location. While we concede that soft skills (e.g., communication, teamwork, professionalism) are as important or even more important than hard skills in some contexts, in this study we restrict our study to hard technical skills to provide for a more focused examination and targeted recommendations. Addressing gaps in soft skills development in curricula generally requires program-level interventions, while technical skill requirements can often be addressed at the course level. Thus, a focused study on technical competencies, we believe, can lead to more direct actionable recommendations that can be addressed more quickly and effectively.
While educators and university administrators use many strategies such as alumni outreach, external relations, and advisory boards to monitor changing job trends, analyzing job advertisements represents a relatively simple yet effective method to monitor the job skills and knowledge required in a rapidly changing environment. Direct surveys, which use available online or printed content, are also used by universities to gather data for text mining analysis. The primary benefit of this approach is that it allows researchers to concentrate on a particular study area, systematically gather data, and apply standardized methods for data quantification (Chen, 2022). Job posts not only identify the skills that companies deem essential for their current job position but may also include skills that they believe will become relevant soon. As such they provide a lens for emerging trends. Moreover, data mining techniques allow us to identify and extract data clutter from the massive amounts of data available as job searches and postings move predominantly online. Users have an easier time navigating, summarizing, and organizing the data to find "interesting" relationships in the data (Alshameri & Green, 2020). Data mining tools offer several computational strategies for swiftly and successfully extracting new information from massive, frequently semi-formatted databases. Clustering algorithms divide the data into groups that display the qualities of the data and the relationships between the groups. The analysis and extraction procedures are consequently improved and provide new insights by identifying the most crucial elements of the textual datasets. One crucial method for gathering related data items is data clustering. People can examine the data from a "bird's eye" perspective by grouping huge amounts of data into informative clusters. We leverage these data mining techniques in our study to uncover important patterns related to technical competencies in job advertisements clustered by discipline and location. We seek to identify the type of jobs available in organizations and the technical skills needed for each job position.
For the purpose of the study, we collected 10,000 job posts. Utilizing this dataset, we conducted a deep analysis using text mining techniques to uncover job trends by business discipline and geographical location. We provide both a visualization as well as a descriptive analysis of our findings.
Our findings should be of interest to several important stakeholders. First, students benefit by being aware of the technical competencies demanded by employers for jobs in different business disciplines. As a result, they can select courses that emphasize these competencies or acquire desired technical skills through certifications/badges offered by universities and professional organizations. Second, for individuals already in the workforce seeking to transition to a new job or career path, the results of our study should be useful in providing them with important information on the additional technical competencies they may currently lack in making them competitive in the marketplace. Additionally, our findings also provide locational biases for the availability of jobs by business discipline. Third, educators and universities can use the findings of our study to modify and update their curricula to align with the changing technical skills demanded by the marketplace. This may involve incorporating new technical skills in the current courses or making them available to students through extracurricular offerings. Fourth, our study provides important information for companies to calibrate their job postings more carefully as they compete for qualified candidates in a highly competitive job market. Job postings with detailed skill requirements can help companies attract and recruit the right type of personnel (Cao et al., 2023). Lastly, our study also has public policy implications as governments increasingly intervene in labor markets and invest in training/re-skilling programs as we transition to a knowledge economy.
We organize the rest of the paper as follows. In the next section, we provide a brief literature review. In Section 3, we discuss the data source for the study and a description of the research methodology including the data mining tools we used. In Section 4, we present and discuss our results. Finally, in Section 5, we conclude, discuss the limitations of our study, and recommend future research extensions.
2. LITERATURE REVIEW
There has been a wide range of studies on the identification of important skill requirements incorporated in job postings. Many of these studies have focused on jobs in one discipline (e.g., information systems, accounting, or marketing) or a specific job type such as business analytics. Smith and Ali (2014) use data mining techniques to analyze job advertisements in the programming field to identify job trends in information technology. Wierschem and Méndez Mediavilla (2018) undertake a comprehensive examination of how employers evaluate different methods for acquiring technology skills among job candidates by identifying and analyzing three key sources of advanced skill categories: academic degrees (qualifications awarded by accredited institutions, typically linked to both two-year and four-year public and private colleges and universities), professional certifications (are granted by non-accredited organizations that recognize an individua""s attainment of a defined level of proficiency in a particular skill or task, as established by the relevant certifying authority), and relevant work experience (experience encompasses various methods of acquiring a specific skill through active participation and observation of related activities).
Gardiner et al. (2018) use content analysis of job requirements to identify skills and competencies required for positions in big data and business analytics. Examining a sample of job qualifications required for analytics positions, Verma et al. (2019), identify skills most in demand for various analytics positions such as Business Analyst, Business Intelligence Analyst, Data Analyst, and Data Scientist. Bowers et al. (2018) develop 16 topics related to analytics job postings and subsequently analyze these topics against the skills produced by graduate analytics programs. Their findings reveal that the leading four skills identified from the job postings were communication and interpersonal skills (61%), managerial skills (33%), database skills (33%), and business domain knowledge (29%). From a study of text mining analysis of large pools of business data analytics and data science job markets, different skills are extracted and classified into four domains, technical, analytical, business, and communication (Radovilsky & Hegde, 2022). Similarly, Dong and Triche (2020) use a textmining approach to examine job advertisements for entry-level data positions and identify technical competencies sought in practice. Topi (2019) advocates systematic collaboration with other computing disciplines in information systems curriculum development to keep up with the newest technological and business trends. AI curricula at business schools must strike the right mix between business and technological skills (Chen, 2022).
Considering the recent surge in public awareness regarding cybersecurity threats, particularly due to significant ransomware attacks, the field of cybersecurity 1s rapidly emerging as one of the fastest-growing career sectors in the United States. According to the U.S. Bureau of Labor Statistics, the anticipated job growth rate for information security analysts is 31% from 2019 to 2029. Research on cybersecurity job descriptions indicates that these roles typically require a range of qualifications, including formal education, technical expertise, relevant work experience, and industry-recognized certifications (Ramezan, 2023).
While most text analyses of skill requirements studies have been in information-related fields, such studies have also been performed in other business disciplines. Leveraging content and cluster analysis of job ads, Dunbar et al. (2016) and Uwizeyemungu et al. (2020) identify technical competency requirements for accounting positions. In the field of marketing, Schlee and Harich (2010) use content analysis to examine 500 marketing job requirements ranging from entry-level to seniorlevel positions. Poba-Nzaou et al. (2020) use content analysis to study skills and competencies required for positions for human resource managers.
In contrast to the above studies that focus on examining skill requirements for a specific job position or business discipline, we take a more holistic and broader approach and focus on examining technical skills demanded for business jobs. The results of our study provide useful information for broad curriculum revision (e.g., undergraduate business curriculum or MBA curriculum revision) as well as for targeted curriculum revision for a major or specialization or a specialized program.
Lastly, like some of the above studies, we leverage data mining techniques that enable us to examine many job postings across the USA allowing us not only to draw robust inferences on technical skill requirements demanded for jobs in different business sub-disciplines but also identify location biases for these jobs.
3. RESEARCH METHODOLOGY
3.1 Data Source and Description
The raw dataset for this study is purchased from a job advertisement monitoring website, https://www.jobspikr.com/. The dataset contains 10,000 job advertisements that are posted from 01/01/2020 to 05/31/2020 in the United States. Of the 10,000 job advertisements, 899 are accounting jobs, 1,239 are cybersecurity jobs, 616 are data science jobs, 900 are finance jobs, 900 are healthcare jobs, 900 are human resources jobs, 2,146 are IT jobs, 1,200 are management jobs, and 1,200 are marketing jobs. As such, we have a broad sample that includes jobs in various business sub-disciplines. Each job advertisement in this raw dataset contains text information on the following features: job title, category, company name, city, state, country, job type, salary, and job description. Some metadata columns such as job title, job type (full time, part time), and salary contain a significantly large percentage of missing data. Because we are concerned more about the technical skills in the job postings, a data preprocessing (1.е., data reduction), is done before we used the dataset to handle the missing data and to remove the irrelevant attributes like company name, job title, country (all jobs in US), and salary. The remaining attributes like state, city, category, and job description are used 1n the analysis.
This study acknowledges several limitations. Firstly, it is based on a relatively small sample of 10,000 job advertisements that were posted from January 1, 2020, to May 31, 2020. The dataset is available for the public to access only by the end of the year 2020. As the demand for technical skill competencies can change swiftly, our research may not encompass the latest technical skills that are currently in demand in certain fields, especially considering the unique context of the COVID-19 period.
3.2 Training Data
We are interested in isolating technical skills in demand in all the nine business sub-disciplines that we capture in this study. We created a training dataset from the 616 Data Science job advertisements. We used a subscription-based online annotation tool, https://ubiai.tools, to create a training dataset of 150 samples. To annotate, we used the Inside-OutsideBeginning (IOB) tagging format (Chauhan et al., 2020; Nasiboglu & Gencer, 2021). This helps to train the model to decide the boundaries of target phrases more accurately.
Annotation focuses on mostly specific technical skills such as "Python" and "К" and occasionally marks high-level technical skills such as "Machine Learning" and "Relational Databases." However, we do not annotate generic technical skills such as "full life cycle of hypothesis" or "model validation." Before the manual annotation, we also retrieved a dictionary from https://github.com/mikeasilva/data-scientistskills.git. This dictionary contains some common technical skills, and we used this dictionary as a lookup list that UBIAI can use for automatic pre-annotation. Figure 1 provides an annotation example of Job Post No. 106.
We used 125 training samples from the annotated dataset and 25 samples for validation. We used the bidirectional encoder representations from transformers (BERT) for the spacy3 Transformer-NER library (Jwa et al, 2019). А transformer model is a novel, attention-based deep learning architecture that is regarded as the replacement of recurrent neural networks (RNN) such as long-short-term-memory (LSTM) for sequential data processing in natural language processing (NLP). The advantage of this architecture 1s that 1t allows pre-trained systems such as the spacy3 TransformerNER to be fine-tuned for more specific tasks such as the training process we used in this project to make the model recognize technical terms based upon our own definition. Figure 2 summarizes the research methodology framework.
4. DATA ANALYSIS AND RESULTS
4.1 Evaluation of Technical Skills Categories
Table 1 shows the technical skills needed in the 10,000 job postings. These technical skills are classified into nine different categories. For each category, we listed the most popular software skills mentioned in the job postings that distinguish each category. Most job postings require technical skills from more than one category. Table 2 shows the distribution of technical skills according to business sub-discipline. While there are some general technical skills such as Microsoft Office, data visualization such as Tableau, and business intelligence that are common across several business sub-disciplines, other technical skills are specific to a particular discipline area. For example, CRM, Salesforce, SEO, Marketo, SEM, Mailchimp, and Google Analytics are all software extensively used in digital marketing. Similarly, systems such as SPHR, HRIS, PHR, and Ceridian are specific to human resources practices and are highlighted in HR job postings. The results of Tables 1 and 2 show that employers demand both general technical and discipline-specific skills. The technical skills listed were presented based on the frequency of appearance in job advertisements.
Table 3 shows the technical skills categories percentages according to sub-discipline area. For example, in the accounting postings, the highest categories are general computer skills and enterprise systems. 33.47% of technical skills are general computer skills (e.g., MicrosoftSuite, and SharePoint), and 28.66% are enterprise system skills (ERP, SAP, CRM are the highest skills). 67.13% of the accounting postings require accounting software skills like Hyperion, QuickBooks, and General Ledger. On the other hand, in the cybersecurity job postings, 65.78% of the technical skills require programming languages (e.g., Python, C++, C, Java, and PHP), while 30.05% of the cybersecurity job postings require database skills (e.g., MySQL, Oracle, Hadoop, and XML). Our results are consistent with Ramezan (2023), who highly recommends that students interested in cybersecurity careers have some programming experience with Python, Java, or MySQL, particularly if they are interested in architecture, penetration testing, information security, or data analysis jobs. Table 3 shows that 60.04% of the job postings for an IT position mentioned the need for skills in enterprise systems. On the other hand, 22.63% of the advertisements for a marketing position required skills in programming languages, and we believe this is because of the increasing move to digital marketing in the discipline. Figure 3 provides the same information in Table 3 in graphical form, the figure shows the distribution of technical skills categories per subject.
Table 4 reports the distribution of job postings by US States. The table shows that California has the highest number of job postings (1,446), followed by Texas (741), and Virginia (686) job postings respectively. On the other hand, Vermont (6) and Wyoming (5) have the lowest numbers of job postings. Table 4 also indicates that not all job postings mentioned technical skills, only 5,569 of the 10,000 job postings included technical skills. For example, job postings in California, out of the 1,446 job postings, 854 (59%) postings specify the technical skills required for the job listed.
Considering that the top three states in Table 4 are also the ones with the highest populations, it is not very surprising that they have the highest job listings. To control for the population effect, we standardized the job metric by dividing the total number of job postings by the number of businesses within that state (Dong & Triche, 2020). Using the North American Industry Classification System (NAICS) Association website (NAICS Association, 2022), we collected the number of businesses in each state. The top states with the standardized job postings, in order, were the District of Columbia, Virginia, Massachusetts, Maryland, Delaware, and California. Figure 4 illustrates a heatmap for the number of standardized posting jobs.
5. CONCLUSION
Using a dataset of 10,000 job posts, we conducted a deep analysis using text mining techniques to uncover job trends by business discipline and geographical location. We provide both a visualization as well as a descriptive analysis of our findings. We identify the most important technical skills demanded by employers in different business sub-disciplines as reflected in job postings and isolate the states in which these jobs are clustered.
This information should be helpful to multiple stakeholders. As job-seeking stakeholders (students, parents, alumni) focus increasingly on student outcomes and job placement, this paper helps identify the gap between the education that universities provide, and the skills employers are seeking. This helps universities address the gap through curriculum redesign and relevant extracurricular activities, ensuring that students are market-ready when they graduate. On a more strategic level, it can help universities better formulate their mission statements in line with the changing needs of the marketplace. Because unlike past studies, we adopt a broad lens and examine business jobs that encompass several subdisciplines, our study provides useful information for broad curriculum revision as well as targeted curriculum revision for different sub-disciplines captured in our study. Students can leverage the findings of our study by adjusting their course map and learning to ensure they acquire the technical skill sets demanded by the market in the business sub-discipline that they are targeting. Employers and companies can use the information to calibrate their job postings to attract the most skilled talent in an ever-increasing competitive job market. Lastly, policymakers can use the information to make important resource allocation decisions regarding retraining and reskilling programs as they prepare the labor force for a knowledge economy.
Our research suffers from some limitations. First, we draw on a limited sample of 10,000 job advertisements that are posted from 1/1/2020 to 5/31/2020. To the extent that technical skill competencies demanded by the market may evolve rapidly, our study may not capture some of the most current technical skills required in some areas. Moreover, our study captures job announcements at the start of the COVID-19 period and data may have shifted following the pandemic. The study could be replicated periodically to update the evolving nature of job competencies. Second, as discussed earlier, we focus on technical skills and do not consider soft skills which may be as or even more important than technical skills in some contexts. We acknowledge the importance of soft skills in today's job market and recommend this for future research. Lastly, our data mining techniques may introduce some bias, however, this is a limitation common to all data mining studies. On the other hand, our methodology can be used in other fields beyond business and information systems to identify skills gaps in the respective fields and help the development of relevant curricula.
6. REFERENCES
Alshameri, F., & Green, N. (2020). Analyzing the Strength Between Mission and Vision Statements and Industry via Machine Learning. The Journal of Applied Business Research, 36(3), 121-128. https://doi.org/10.19030/jabr.v3613.10348
Aoun, J. E. (2017). Robot-Proof: Higher Education in the Age of Artificial Intelligence. Cambridge, MA, USA: MIT Press. https://doi.org/10.7551/mitpress/11456.001.0001
Association of American Colleges and Universities (AACU) (2018, July). Fulfilling the American Dream: Liberal Education and the Future of Work. Hart Research Associates. https://www.aacu.org/research/fulfilling-theamerican-dream-liberal-education-and-the-future-of-work
Behn, В. K., Ezzell, W. F., Murphy, L. A., Rayburn, J. D., Stith, M. T., & Strawser, J. К. (2012). The Pathways Commission on Accounting Higher Education: Charting a National Strategy for the Next Generation of Accountants. Issues in Accounting Education, 2703), 595-600. https://doi.org/10.2308/1ace-10300
Borner, K., Scrivner, O., Gallant, M., Ma, S., Liu, X., Chewning, K., Wu, L., & Evans, J. A. (2018). Skill Discrepancies Between Research, Education, and Jobs Reveal the Critical Need to Supply Soft Skills for The Data Economy. Proceedings of the National Academy of Sciences, 115(50), 12630-12637. https://doi.org/10.1073/pnas.1804247115
Bowers, M. R., Camm, J. D., & Chakraborty, G. (2018). The Evolution of Analytics and Implications for Industry and Academic Programs. Interfaces, 48(6), 487-499. https://doi.org/10.1287/inte.2018.0955
Cao, Y., Cheng, S., Tucker, J, & Wan, С. (2023). Technological Peer Pressure and Skill Specificity of Job Postings. Contemporary Accounting Research, 40(3), 2106-2139. https://doi.org/10.1111/1911-3846.12870
Chauhan, A., Malhotra, A., Singh, A., Arora, J., & Shukla, S. (2020). Encoding Context in Task-Oriented Dialogue Systems Using Intent, Dialogue Acts, and Slots. Innovations in Computer Science and Engineering: Proceedings of 7" ICICSE (pp. 287-295). Singapore: Springer. https://doi.org/10.1007/978-981-15-2043-3 34
Chen, L. (2022). Current and Future Artificial Intelligence (AI) Curriculum in Business School: A Text Mining Analysis. Journal of Information Systems Education, 33(4), 416-426.
Davenport, T. H. (2018). From Analytics to Artificial Intelligence. Journal of Business Analytics, 1(2), 73-80. https://doi.org/10.1080/2573234X.2018.1543535
Dong, T., & Triche, J. (2020). A Longitudinal Analysis of Job Skills for Entry-Level Data Analysts. Journal of Information Systems Education, 31(4), 312-326.
Donovan, S., Stoll, A., Bradley, D., & Collins, В. (2022). Skill Gaps: A Review of Underlying Concepts and Evidence. CRS Report R47059, Version 3. Congressional Research Service. https://www.congress.gov/crs-product/R47059
Dunbar, K., Laing, G., & Wynder, M. (2016). A Content Analysis of Accounting Job Advertisements: Skill Requirements for Graduates. E-Journal of Business Education andScholarship of Teaching, 10(1), 58-72.
Gardiner, A., Aasheim, C., Rutner, P., & Williams, S. (2018). Skill Requirements in Big Data: A Content Analysis of Job Advertisements. Journal of Computer Information Systems, 58(4), 374-384. https://doi.org/10.1080/08874417.2017.1289354
Jwa, H., Oh, D., Park, K., Kang, J., & Lim, H. (2019). Exbake: Automatic Fake News Detection Model Based on Bidirectional Encoder Representations From Transformers (BERT). Applied Sciences, 09(19), 4062-4070. https://doi.org/10.3390/app9194062
NAICS Association. (2022). https://www.naics.com/businesslists/counts-by-state/
Nasiboglu, R., & Gencer, M. (2021). Comparison of Spacy and Stanford Libraries' Pre-Trained Deep Learning Models for Named Fntity Recognition. Journal of Modern Technology and Engineering, 6(2), 104-111.
Nguyen, A., Gardner, L., & Sheridan, D. (2020). Data Analytics in Higher Education: An Integrated View. Journal of Information Systems Education, 31(1), 61-71.
Poba-Nzaou, P., Uwizeyemungu, S., & Clarke, С. (2020). Patterns Underlying Required HR and IT Competencies: À Content and Cluster Analysis of Advertisements of HR Manager Positions. The International Journal of Human Resource Management, 31(16), 2065-2088. https://doi.org/10.1080/09585192.2018.1424019
Radovilsky, Z., & Hegde, V. (2022). Contents and Skills of Data Mining Courses in Analytics Programs. Journal of Information Systems Education, 332), 182-194.
Ramezan, C. A. (2023). Examining the Cyber Skills Gap: An Analysis of Cybersecurity Positions by Sub-Field. Journal of Information Systems Education, 34(1), 94-105.
Schlee, R., 8 Harich, К. (2010). Knowledge and Skill Requirements for Marketing Jobs in the 21st Century. Journal of Marketing Education, 32(3), 341-352. https://doi.org/10.1177/0273475310380881
Smith, D., & Ali, A. (2014). Analyzing Computer Programming Job Trend Using Web Data Mining. Issues in Informing Science and Information Technology, 11(1), 203-214. https://doi.org/10.28945/1989
Topi, H. (2019). Reflections on the Current State and Future of Information Systems Education. Journal of Information Systems Education, 30(1), 1-9.
Uwizeyemungu, S., Bertrand, J., & Poba-Nzaou, P. (2020). Patterns Underlying Required Competencies for CPA Professionals: A Content and Cluster Analysis of Job Ads. Accounting Education, 2972), 109-136. https://doi.org/10.1080/09639284.2020.1737157
Verma, A., Yurov, K., Lane, P., € Yurova, Y. (2019). Ап Investigation of Skill Requirements for Business and Data Analytics Positions: A Content Analysis of Job Advertisements. Journal of Fducation for Business, 94(4), 243-250. https://doi.org/10.1080/08832323.2018.1520685
Wierschem, D., & Méndez Mediavilla, F. A. (2018). Entry Level Technology Positions: No Degree Required. Journal of Information Systems Education, 29(4), 253-268.
Copyright EDSIG 2025