Analyzing the Dimensionality of O*NET Cognitive Ability Ratings to Inform Assessment Design

Abstract

The O*NET database is an online repository of detailed information on the knowledge and skill requirements of thousands of jobs across the United States. Thus, it is a valuable resource for test developers who want to target cognitive and other abilities relevant to the contemporary workforce. In this study, we used multidimensional scaling (MDS) to analyze the mean importance ratings of the cognitive abilities and selected skills included in the O*NET database to identify the dimensionality of the data regarding importance and their consistency across job zones. Using the criteria of fit and interpretability, a two-dimensional MDS solution was selected as the best representation of the data. These dimensions reflected Social Interaction/Reasoning and Verbal/Non-Verbal skills and abilities. Interestingly, the dimensionality was not consistent across job zones. Job zones relative to lower education and training requirements were sufficiently represented by the Social Interaction/Reasoning dimension, and the Verbal/Non-Verbal dimension was most relevant to job zones requiring more education and experience. The implications of the results for developing assessments for adult learners are discussed, as is the utility of using MDS for understanding the dimensionality of O*NET data.

1. Introduction

1.1. Analyzing the Dimensionality of O*NET Cognitive Ability Ratings to Inform Assessment Design

The Occupational Information Network (O*NET) is a program sponsored by the U.S. Department of Labor that provides information about occupations in the United States (See https://www.onetcenter.org/overview.html accessed on 20 September 2024). A key component of the program is the O*NET database, which contains standardized descriptions of approximately 1000 occupations across the U.S. For each occupation, the database includes descriptions of worker characteristics, job requirements, and information about the knowledge, skills, and abilities required for success on the job (Donsbach et al. [1]). As Haigler [2] described, O*NET represents “a free online resource for employers and educators to benchmark job requirements against data derived from job analysis” (p. 272).

O*NET provides a “content model”, which is a framework that integrates the most important types of information about work into a theoretically and empirically sound system that can be used to understand the knowledge and characteristics associated with a wide variety of jobs (Tippins and Hilton [3]). The model contains 277 descriptors, both worker-oriented (i.e., worker characteristics, worker requirements, and experience requirements) and job-oriented (i.e., occupational requirements, workforce characteristics, and occupation-specific information). These descriptors characterize the knowledge, skills, experience, and other assets needed for success across numerous occupations.

In the present study, we analyze 45 of these descriptors—21 cognitive abilities and 24 basic skills—to understand how these abilities and skills can best be measured along one or more continua related to job success. Our interest in measuring these abilities and skills stems from our involvement in the Adult Skills Assessment Program (ASAP) (see https://blogs.umass.edu/asap/ 29 October 2024) which focuses on identifying the critical aspects of literacy and numeracy that bridge the adult education and workforce communities. Like many educators working with adults in academic and workplace settings, we are interested in assessing the critical abilities and skills adults need to meet their academic and occupational goals. Our analysis of the dimensionality of these O*NET cognitive abilities and basic skills seeks to provide valuable information for understanding the constructs to be measured that are relevant to a variety of contemporary occupations.

To investigate the dimensionality of these O*NET data, we investigated two primary research questions, as follows:

• (a). What dimensions can be used to characterize the O*NET cognitive ability and basic skill descriptors? Can a single dimension be used to summarize these descriptors or are they multidimensional? Specifically, will we uncover separate dimensions for the cognitive abilities and basic skills?

• (b). Do these ability/skill dimensions vary across O*NET job zones? That is, is the dimensionality of the data invariant across jobs that differ with respect to skills and experience or are separate dimensions needed for each job zone?

Our investigation of the first research question aims to refine our understanding of the underlying structure of the data to understand whether the cognitive and basic skills deemed important for various jobs represent multidimensional or unidimensional constructs. By conducting dimensionality analyses, we lay the groundwork for robust conceptualization and construct definition, which are necessary to accurately and authentically measure and assess workforce skills and competencies.

Our second research question focuses on identifying potential variations across O*NET job zones. This analysis is important in determining whether the dimensionality of the data remained consistent across diverse jobs characterized by varying skill sets and levels of experience. Understanding whether separate dimensions were required for each job zone or if a consistent dimensionality could be applied across all zones is important for understanding how assessments may need to be tailored to specific occupational contexts.

1.2. Use of O*NET Data in Previous Research

To address these questions, we built on previous research conducted using O*NET data to better understand characteristics of occupations in the U.S. Although a full review of the O*NET literature is beyond the scope of the current article, we note O*NET data have been used to relate job characteristics to health outcomes (Cifuentes et al. [4]); improve the predictive validity of personality assessments used in work settings [5]; cluster occupations based on their knowledge characteristics and identify regional trends in occupational knowledge [6,7]; develop machine learning models to predict changes in employment and skill demands over time [8]; and investigate sex differences in labor attrition [9]. As Lewandowski et al. [10] put it when they analyzed international work trends, “no other sources offer as rich and detailed data on occupations as O*NET” (p. 2).

Haigler [2], for example, described how O*NET was used to align knowledge, skills, and abilities (KSAs) in adult education with competencies needed in the workplace. Thompson and Koys [11] demonstrated how programs could leverage O*NET data to empirically define learning outcomes for learners similar to how they are defined for business management curricula. Homan and Sandall [12] similarly performed an undergraduate curriculum needs assessment that leveraged O*NET data. In the context of informing English language adult education, Oliveri and McCulla [13] used O*NET data to identify the English communication tasks common within different levels of preparation required for employment.

With respect to understanding the structure of O*NET data, Burrus et al. [14] used principal components analysis (PCA) to analyze the dimensionality of the 136 variables included in the database that described worker characteristics (abilities, work styles, skills, and knowledge). They found 15 factors relevant for describing the dimensionality of the data, and concluded that the following five factors were most important for all occupations: problem-solving (e.g., complex problem-solving), fluid intelligence (e.g., category flexibility), teamwork (e.g., cooperation), achievement/innovation (e.g., persistence), and communication skills (e.g., oral expression). Their findings underscore the importance of these core competencies in driving success and effectiveness across diverse job roles and industries. Few studies of this kind have been conducted. In the current study, we address this paucity in the literature by examining the structure of a subset of O*NET to provide valuable insights into the key dimensions that underpin workforce contexts, in order to inform more authentic and relevant assessment practices.

Our study also sought to determine if our analysis would uncover dimensions akin to those identified in prior research (e.g., problem-solving, fluid intelligence, achievement, teamwork, and communication; see Burrus et al. [14]), using multidimensional scaling (MDS), which is known to capture data structure in fewer dimensions than traditional factor analysis [15]. If our findings replicate those of previous studies, it suggests a robustness in the fundamental dimensions crucial for workplace success and provides empirical support for integrating these dimensions into task design aimed at assessing similar KSAs vital for effective cross-industry workplace performance.

2. Method

In this section, we describe some characteristics of the O*NET database and how we extracted occupational data for analysis. We then describe the analyses conducted.

2.1. Description of O*NET Data

The data analyzed here come from the O*NET 26.1 Database [16]. The O*NET database specifies various kinds of KSAs and their required competency levels and relative importance in over 800 occupations. The O*NET Data Collection Program starts with occupational definitions from the Standard Occupational Classification system, a United States federal framework for classifying workers and occupations [17]. To gather information about each occupation, the program polls samples of workers in those occupations and subject matter experts (e.g., members of trade associations and analysts), asking them to rate how important various KSAs are to the occupation, and at what level the occupation requires. The level rating indicates the degree, or point along a continuum, to which a particular descriptor is required or needed to perform the occupation. The importance rating indicates the degree of importance a particular descriptor is to the occupation. For our focus on importance, the possible ratings range from “Not Important” (1) to “Extremely Important” (5). Additional information is collected about the work context, activities, and worker respondent’s background [18].

In the most recent O*NET data file release [16], the experts’ ratings of 120 KSAs across 873 occupational roles are summarized. Of those KSAs, there are 21 cognitive abilities and 24 basic skills, which are the focus of this study. The cognitive skills are presented in Table 1, and the basic skills are presented in Table 2, along with a brief description of each. We selected these abilities and skills given our interest in developing numeracy and literacy assessments that bridge adult education and workplace settings. These selected cognitive abilities and basic skills are paramount in adult education curricula, job training programs, and workforce development. Thus, assessments of these abilities and skills are necessary for upskilling adult learners. Moreover, the dimensionality of these targeted abilities and skills must be known for assessment design and result reporting, given that O*NET has identified these cognitive abilities and basic skills as important for and relevant to different classes of jobs in the U.S. A better understanding of how these abilities relate to one another across substantively different occupations should inform the development of assessment frameworks for measuring important numeracy and literacy competencies relevant to both adult educators and employers.

O*NET also classifies the 873 occupations according to five “job zones”. The job zones range from 1 to 5, with occupations needing no or little preparation coded as 1, and occupations needing extensive preparation coded as 5. The job zone descriptions are presented in Table 3. These job zones are very helpful for broadly classifying the huge number of occupations across the U.S. into categories that are similar with respect to requisite KSAs, experience, and other requirements.

2.2. Data Extraction

For each cognitive ability and basic skill (Table 1 and Table 2), the O*NET database provides the mean importance and level ratings across experts. We used the mean importance rating for each cognitive ability to compute a dissimilarity matrix across these 45 skills and abilities. Euclidean distances were computed across each pair of ability/skill by summing the squared differences in the mean importance ratings for each pair across all occupations, as follows:

(1)${\delta }_{j{j}^{\prime }}=\sqrt{{\sum }_{i=1}^N{(u_{ij}-{u_{ij}}^{\prime })}^2}$

where j and j′ are two different abilities/skills, u_ij is the mean importance rating for ability or skill j on occupation i, and N represents the total number of occupations for which the abilities/skills were rated (N = 873). This process resulted in a 45 × 45 square matrix of Euclidean distances across abilities and skills as defined by the degree of similarity in mean importance ratings across occupations.

As indicated in the second research question, we were interested in the consistency of the dimensionality of the ability/skill space across job zones. Thus, in addition to computing an overall dissimilarity matrix across all job zones, we computed a separate matrix for each of the five job zones. There were 32 jobs in job zone 1, 278 in job zone 2, 208 in job zone 3, 200 in job zone 4, and 155 in job zone 5. Given the number of jobs in each zone differed across zones, we divided the summed differences across job zones to put the distances on the same scale, as follows:

(2)${\delta }_{j{jk}^{\prime }}=\sqrt{{\displaystyle \frac{{\sum }_{i=1}^{N_k}{(u_{ij}-{u_{ij}}^{\prime })}^2}{N_k}}}$

where ${\delta }_{j{jk}^{\prime }}=$ the dissimilarity between skills/abilities j and j′ for job zone k, and N_k = the number of jobs in job zone k. It was these five job zone-specific matrices that were the focus of our analyses.

2.3. Weighted Multidimensional Scaling Analyses

We analyzed the ability/skill dissimilarity matrices using multidimensional scaling (MDS). MDS is a non-linear, exploratory procedure that can be used for single- or multi-group analyses. When applied to a single group, the matrix of observed item dissimilarities is modeled in 1, 2, …, or R-dimensional space as follows:

(3)$d_{{jj}^{\prime }}=\sqrt{{\sum }_{r=1}^R{\left(x_{jr}-x_{j^{\prime }r}\right)}^2}$

where $d_{{jj}^{\prime }}$ is the distance between ability/skill j and ability/skill j′, R indicates the maximum dimensionality of the model, and x_jr is the coordinate for ability/skill j on dimension r.

To analyze the data simultaneously across all five job zone matrices, we used the INDSCAL model [19], which incorporates a weight on each dimension for each matrix, as follows:

(4)$d_{{jj}^{\prime }}=\sqrt{{\sum }_{r=1}^R{w}_{kr}{\left(x_{jr}-x_{j^{\prime }r}\right)}^2}$

where $w_{kr}$ corresponds to the weight associated with dimension r for group k. Thus, INDSCAL is described as a “weighted” MDS model.

Weighted MDS solutions provide a multidimensional configuration that best fits the data for all groups when considered simultaneously, and a matrix of group weights (with elements $w_{kr}$) that represent how this configuration should be adjusted to best fit the data for a particular group (k). Thus, the weights ($w_{kr}$) contain the information regarding differences in dimensionality (if any) across job zones. If a similar pattern of weights is observed across job zones, the same dimensions can be used to account for the dimensionality of the cognitive abilities. That is, similar weights across groups suggest invariance in the dimensionality of the data across groups, while differences between group weights indicate differences in dimensionality across groups. Using simulated data, Sireci, Bastari, and Allalouf [20] found that, when structural differences existed across groups, one or more groups had weights near zero on one or more dimensions relevant to at least one other group.

Two- through five-dimensional weighted MDS solutions were applied to the data across the five job zones (using SPSS version 28.0). Weights are not relevant to a one-dimensional model; therefore, a replicated MDS analysis [21] was used to fit a one-dimensional solution across the job zones.

To assess the fit of the MDS solutions we used the following fit measures: Stress, which is a normalized measure of misfit ranging from zero (perfect fit) to one, and R², which represents the proportion of variance accounted for in the (transformed) inter-cognitive ability distances by the MDS solution. Smaller values of Stress and larger values of R² indicate a better fit. A key criterion in evaluating fit is a relatively large decrease in Stress across adjacent solutions [22].

3. Results

The mean ratings for each of the 45 cognitive abilities and basic skills for each job zone are presented in Table 4, in descending order according to mean ratings for job zone 1. These ratings come from the samples of workers and subject matter experts involved in rating the importance of these knowledge and skills in the creation of the O*NET database [16].

Fit and Selection of Dimensionality

Table 5 presents the Stress and R² fit measures for the overall solution and for each job zone matrix. The largest improvement in fit occurred between the unidimensional solution and the two-dimensional solution. Only job zone 3 appears to adequately fit in the 1D solution. The two-dimensional solution fit the data well overall, accounting for 92% of the variation in the transformed dissimilarity data. All job zones had Stress values under 2.0 and R² at or above 0.89. We inspected the higher-dimensional solutions and noted some differentiation in the weights associated with job zones 4 and 5 in the four-dimensional solution, but we ultimately concluded the two-dimensional solution was best in terms of parsimoniously capturing the substantive dimensionality across job zones. Therefore, the two-dimensional solution was selected as the best representation of the data. We now turn to interpreting the stimulus (cognitive ability and skill) space, as well as the weight (job zone) space.

Figure 1 displays the skill space for the two-dimensional weighted MDS solution. The horizontal dimension separates skills and abilities that deal with some kind of social interaction (e.g., oral communication, oral expression) from skills and abilities that are more associated with more independent reasoning (e.g., science, spatial orientation, management). It also pulls cognitive abilities and skills rated very low for job zones 1 and 2 (e.g., science, management of financial resources) away from those rated more important for these job zones. The vertical dimension orders the skills and abilities along a verbal (e.g., written comprehension, reading comprehension) versus non-verbal (e.g., spatial orientation, visualization) continuum. Interestingly, separate dimensions for the cognitive abilities and skills did not emerge. These results suggest the general areas of social interaction, reasoning, verbal, and non-verbal information would be important constructs to measure (and to teach!) for adults seeking employment.

The job zone weight space for the two-dimensional solutions is presented in Figure 2. The weight vectors for job zones 1 and 2 indicate dimension 1 (Social Interaction vs. Reasoning), which essentially accounts for all of the variation in the (transformed) similarities among the mean skill importance ratings. For job zones 4 and 5, it is dimension 2 (Verbal vs. Non-Verbal) that accounts for almost all of the variation in the data. Job zone 3 appears to be the only one that requires both dimensions to account for the variation in mean skill importance ratings.

To facilitate the interpretation of the dimensions, we correlated the MDS coordinates with the mean importance ratings of the cognitive abilities and skills presented in Table 4. These correlations are presented in Table 6. The mean ratings for job zones 3, 4, and 5 exhibited extremely large negative correlations with the dimension 1 coordinates, ranging from −0.92 (job zone 3) to −0.99. This reflects the fact that the cognitive abilities and skills most related to “social interaction” (the negative or left side of the figure) were rated more highly in these job zones. The coordinates of dimension 2 exhibited extremely large negative correlations with job zones 1 and 2 (r = −0.99), as well as job zone 3 (r = −0.92). This finding suggests that the cognitive abilities and skills more related to verbalization and language-based skills had relatively high mean ratings.

4. Discussion

In this study, we analyzed the mean importance ratings for 45 cognitive abilities and basic skills across 873 jobs listed in the O*NET database. We were interested in the dimensionality of these data; specifically, what dimensions could be used to summarize these skills and how these dimensions might differ across job zones.

The results suggested that two dimensions—Social Interaction/Reasoning and Verbal/Non-Verbal—sufficiently characterized the structure of these data across all five job zones. Interestingly, the importance of these dimensions in capturing the structure was not consistent across job zones. The Social Interaction/Reasoning dimension accounted for the structure of the mean importance ratings for job zones 1 and 2, which makes sense because those job zones require less education and experience (see Table 3), which likely require more reasoning skills. The structure of the mean importance ratings for job zones 4 and 5 was essentially accounted for by the Verbal/Non-Verbal dimension, meaning the differences between ratings were best described by the extent to which skills were language-based. This observation corroborates the intuition that these job zones require the acquisition and application of skills such as scientific knowledge and other specialized information-processing skills, which are typically acquired through post-secondary and post-college academic training. Job zone 3, which requires training and experience that is more “on-the-job” than academic, required both dimensions to adequately fit the mean importance ratings.

These results suggest assessments targeted at KSAs associated with job zones 1 and 2 should focus on basic social interactive and reasoning skills, while assessments focused on job zones 4 and 5 would need to assess verbal and non-verbal skills that are likely to fall into advanced math, science, and specialized areas. Assessments for job zone 3 would likely require measurement of communicating, reasoning, and more specialized processing skills.

Burrus et al. [14] used principal components analysis to analyze the dimensionality of 136 O*NET worker characteristic variables and concluded that the following five factors were most important for all occupations: problem-solving, fluid intelligence (e.g., category flexibility), teamwork, achievement/innovation, and communication skills. Our analyses involved only a subset of O*NET variables related to cognitive abilities and basic skills, but the dimensions we uncovered similarly distinguished skills related to problem-solving, fluid intelligence, achievement, and communication. These dimensions reflect skills essential for workplace readiness and preparation (e.g., problem-solving, fluid intelligence, achievement, and communication), which are broader than basic math and literacy skills. Although automation may transform the world of work for routine tasks, it is less helpful for non-routine abstract tasks, such as creative problem-solving. Thus, these findings underscore the importance of skills aligned with completing non-routine tasks.

One interesting finding is that our analysis of both cognitive abilities and selected skills did not yield a dimensional solution that differentiated these abilities and skills from each other. This outcome likely reflects the fact that the basic skills we selected from O*NET draw upon these cognitive abilities.

MDS illustrates the potential for advancing our understanding of the nuanced relationships between various dimensions of workplace competencies and devising more effective assessment strategies tailored to the evolving demands of the modern workforce. To illustrate, for numeracy assessments, recognizing multiple dimensions beyond basic arithmetic could enable the inclusion of several related yet unique dimensions associated with an expanded construct model, which might require assessing additional skills (e.g., data interpretation, problem-solving, and decision-making skills). Assessments solely focused on arithmetic operations would overlook these critical competencies essential for successful workplace functioning. By incorporating multiple dimensions, numeracy assessments become more comprehensive, providing a holistic evaluation of workers’ abilities to analyze and apply numerical information in diverse contexts.

Similarly, in literacy assessments, acknowledging multiple dimensions beyond reading and writing could help ensure assessments reflect the multifaceted literacy skills needed in the workplace. Tasks that solely measure reading comprehension may overlook workers’ abilities to interpret technical documents, follow instructions, communicate effectively, and collaborate with colleagues. By assessing multiple dimensions, literacy assessments encompass a broader range of skills essential for navigating complex workplace environments.

Overall, by identifying and incorporating multiple dimensions into literacy and numeracy assessments, we ensure assessments accurately reflect the diverse skill sets needed for effective workplace performance, leading to better alignment between assessed skills and job requirements.

5. Limitations and Suggestions for Future Research

This study analyzed O*NET data for the purpose of understanding the dimensionality of the worker characteristic space underlying various jobs. Our study had several limitations, perhaps the most significant of which was restricting our selection to only 45 skills, which is a small subset of the 136 knowledge, skills, abilities, aspirations, and other variables included in the O*NET database. However, our exclusion of physical skills and abilities helped focus the dimensional scaling for the purpose of cognitive assessment. That is, uncovering where physical, psychomotor, and sensory abilities are in relation to cognitive abilities would not be of interest for inclusion where assessment is not intended or equipped to assess physical ability. Nevertheless, future research may benefit from incorporating all O*NET descriptor variables into the analysis, especially for those interested in comprehensively understanding the overall structure of these descriptors rather than solely focusing on those amenable to educational intervention. Additionally, exploring alternative methodologies, such as simulations, virtual reality environments, and job task simulations, could provide richer insights into the multidimensional nature of workplace skills. Moreover, longitudinal studies tracking the development and evolution of skills over time, as well as cross-cultural comparisons, could offer further depth to our understanding of workforce competencies. By embracing diverse approaches and methodologies, future research can advance our understanding of multidimensional skills and inform the design of more effective interventions and assessment strategies tailored to the complex demands of the contemporary workplace.

Author Contributions

The conceptualization of this paper was largely from the first author. All authors contributed equally to the methodology, software, validation, analysis, and writing. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

The research conducted here did not involve the use of human subjects.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data analyzed in this study can be freely downloaded from National Center for O*NET Development. O*NET OnLine. at https://www.onetonline.org/ 29 October 2024.

Conflicts of Interest

The authors declare no conflicts of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1. Two-dimensional MDS solution for cognitive ability and skill space. Notes: ActLrn = active learning, ActLstn = active listening, CatFlex = category flexibility, Coor = coordination, CPS = complex problem-solving, CrTh = critical thinking, DedR = deductive reasoning, FlexCl = flexibility of closure, IdeaFlu = fluency of ideas, IndR = inductive reasoning, InfOr = information ordering, Instr = instructing, JDM = judgment and decision-making, LrnStr = learning strategies, MathR = mathematical reasoning, Maths = mathematics, Mem = memorization, MFR = management of financial resources, MMR = management of material resources, Mntr = monitoring, MPR = management of personnel resources, Neg = negotiation, NumFac = number facility, OralCo = oral comprehension, OralEx = oral expression, Orig = originality, PercSp = perceptual speed, Pers = persuasion, ProbSen = problem sensitivity, RC = reading comprehension, Sci = science, SelAtt = selective attention, SoPer = social perceptiveness, SpClosr = speed of closure, Speak = speaking, SpO = spatial orientation, SrvOr = service orientation, SysA = systems analysis, SysE = systems evaluation, TimeM = time management, TmShr = time sharing, WrCo = written comprehension, WrEx = written expression, Write = writing, Viz = visualization.

Figure 2. Job zone weight space from two-dimensional MDS solution.

Table 1

O*NET cognitive abilities.

Cognitive Ability	Description:
Category Flexibility	Generate or use different sets of rules for combining or grouping things in different ways.
Deductive Reasoning	Apply general rules to specific problems to produce answers that make sense.
Flexibility of Closure	Identify or detect a known pattern (a figure, object, word, or sound) that is hidden in other distracting material.
Fluency of Ideas	Come up with a number of ideas about a topic (the number of ideas is important, not their quality, correctness, or creativity).
Inductive Reasoning	Combine pieces of information to form general rules or conclusions (includes finding a relationship among seemingly unrelated events).
Information Ordering	Arrange things or actions in a certain order or pattern according to a specific rule or set of rules (e.g., patterns of numbers, letters, words, pictures, mathematical operations).
Mathematical Reasoning	Choose the right mathematical methods or formulas to solve a problem.
Memorization	Remember information such as words, numbers, pictures, or procedures.
Number Facility	Add, subtract, multiply, or divide quickly and correctly.
Oral Comprehension	Listen to and understand information and ideas presented through spoken words and sentences.
Oral Expression	Communicate information and ideas in speaking so others will understand.
Originality	Come up with unusual or clever ideas about a given topic or situation, or to develop creative ways to solve a problem.
Perceptual Speed	Quickly and accurately compare similarities and differences among sets of letters, numbers, objects, pictures, or patterns. Also includes comparing a presented object with a remembered object.
Problem Sensitivity	Tell when something is wrong or is likely to go wrong. It does not involve solving the problem, only recognizing that there is a problem.
Selective Attention	Concentrate on a task over a period of time without being distracted.
Spatial Orientation	Know your location in relation to the environment or to know where other objects are in relation to you.
Speed of Closure	Quickly make sense of, combine, and organize information into meaningful patterns.
Time Sharing	Shift back and forth between two or more activities or sources of information (such as speech, sounds, touch, or other sources).
Visualization	Imagine how something will look after it is moved around or when its parts are moved or rearranged.
Written Comprehension	Read and understand information and ideas presented in writing.
Written Expression	Communicate information and ideas in writing so others will understand.

Source: https://www.onetonline.org/find/descriptor/browse/Abilities/1.A.1/ (accessed on 22 September 2024).

Table 2

Selected O*NET skills.

Skill	Description
Active Learning	Understanding the implications of new information for both current and future problem-solving and decision-making.
Active Listening	Giving full attention to what other people are saying, taking time to understand the points being made, asking questions as appropriate, and not interrupting at inappropriate times.
Complex Problem-Solving	Identifying complex problems and reviewing related information to develop and evaluate options and implement solutions.
Coordination	Adjusting actions in relation to others’ actions.
Critical Thinking	Using logic and reasoning to identify the strengths and weaknesses of alternative solutions, conclusions, or approaches to problems.
Instructing	Teaching others how to do something.
Judgment and Decision-Making	Considering the relative costs and benefits of potential actions to choose the most appropriate one.
Learning Strategies	Selecting and using training/instructional methods and procedures appropriate for the situation when learning or teaching new things.
Management of Financial Resources	Determining how money will be spent to carry out the work, and accounting for these expenditures.
Management of Material Resources	Obtaining and seeing to the appropriate use of equipment, facilities, and materials needed to carry out certain work.
Management of Personnel Resources	Motivating, developing, and directing people as they work, identifying the best people for the job.
Mathematics	Using mathematics to solve problems.
Monitoring	Monitoring/assessing the performances of yourself, other individuals, or organizations to make improvements or take corrective action.
Negotiation	Bringing others together and trying to reconcile differences.
Persuasion	Persuading others to change their minds or behavior.
Reading Comprehension	Understanding written sentences and paragraphs in work-related documents.
Science	Using scientific rules and methods to solve problems.
Service Orientation	Actively looking for ways to help people.
Social Perceptiveness	Being aware of others’ reactions and understanding why they react as they do.
Speaking	Talking to others to convey information effectively.
Systems Analysis	Determining how a system should work and how changes in conditions, operations, and the environment will affect outcomes.
Systems Evaluation	Identifying measures or indicators of system performance and the actions needed to improve or correct performance, relative to goals of the system.
Time Management	Managing one’s own time and the time of others.
Writing	Communicating effectively in writing using appropriate language for the audience’s needs.

Sources: https://www.onetonline.org/find/descriptor/browse/2.A (accessed on 29 October 2024); https://www.onetonline.org/find/descriptor/browse/2.B (accessed on 22 September 2024).

Table 3

O*NET job zones.

Job Zone	Education Level	Experience	Training	Examples
1	May require high school (HS) diploma or General Educaitonal Development test (GED)	Little to no previous experience	A few days or months	Food preparation workers, dishwashers, landscaping workers, logging equipment operators
2	HS diploma or GED	Some previous work-related skills, knowledge, or experience are usually needed	A few months to a year	Orderlies, counter and rental clerks, customer service representatives, security guards, upholsterers, tellers
3	Vocational schools, on-the-job experience, or associate’s degree	Previous work-related skills, knowledge, or experience are required	Two years of training or an apprenticeship	Hydroelectric production managers, travel guides, electricians, agricultural technicians, barbers, court reporters, medical assistants
4	Bachelor’s Degree (most)	Considerable work-related skills, knowledge, or experience	Several years	Real estate brokers, sales managers, database administrators, graphic designers, chemists, art directors, cost estimators
5	Postgraduate degree (M.A., M.D., J.D., Ph.D.)	Extensive	Assumed in education	Pharmacists, lawyers, astronomers, biologists, clergy, neurologists, veterinarians

Source: Adapted from https://www.onetonline.org/help/online/zones#zone1 (accessed on 29 October 2024).

Table 4

Mean (standard deviation) importance ratings by job zone.

Ability/Skill	Job Zone
	1(n. of Jobs = 32)	2(n. of Jobs = 278)	3(n. of Jobs = 208)	4(n. of Jobs = 200)	5(n. of Jobs = 155)
Science	1.21 (0.26)	1.39 (0.41)	1.93 (0.64)	2.2 (0.88)	3.01 (0.9)
Mgmt. of Financial Resources	1.58 (0.28)	1.67 (0.33)	1.87 (0.42)	2.19 (0.56)	1.96 (0.48)
Mgmt. of Material Resources	1.72 (0.27)	1.81 (0.34)	2 (0.4)	2.18 (0.5)	2.02 (0.42)
Spatial Orientation	1.79 (0.62)	1.86 (0.61)	1.65 (0.57)	1.3 (0.45)	1.15 (0.28)
Mathematical Reasoning	1.96 (0.28)	2.25 (0.40)	2.58 (0.46)	3.02 (0.56)	2.95 (0.66)
Systems Analysis	1.96 (0.26)	2.2 (0.35)	2.66 (0.39)	3.18 (0.41)	3.17 (0.32)
Systems Evaluation	1.99 (0.25)	2.15 (0.34)	2.58 (0.39)	3.09 (0.44)	3.12 (0.32)
Mathematics	1.99 (0.3)	2.25 (0.39)	2.5 (0.44)	2.91 (0.58)	2.86 (0.67)
Number Facility	2.03 (0.29)	2.28 (0.39)	2.53 (0.42)	2.89 (0.51)	2.79 (0.58)
Memorization	2.04 (0.17)	2.14 (0.29)	2.41 (0.31)	2.56 (0.28)	2.73 (0.25)
Originality	2.12 (0.28)	2.32 (0.36)	2.81 (0.38)	3.23 (0.37)	3.23 (0.31)
Speed of Closure	2.13 (0.26)	2.28 (0.32)	2.53 (0.35)	2.63 (0.29)	2.64 (0.35)
Negotiation	2.13 (0.28)	2.31 (0.39)	2.59 (0.44)	2.99 (0.42)	2.88 (0.39)
Learning Strategies	2.15 (0.28)	2.33 (0.38)	2.74 (0.38)	3.03 (0.4)	3.45 (0.43)
Fluency of Ideas	2.16 (0.30)	2.40 (0.36)	2.89 (0.32)	3.31 (0.35)	3.34 (0.31)
Mgmt. of Personnel Resources	2.18 (0.32)	2.32 (0.41)	2.6 (0.49)	2.89 (0.46)	2.89 (0.32)
Writing	2.22 (0.29)	2.62 (0.41)	3.1 (0.38)	3.63 (0.32)	3.89 (0.25)
Instructing	2.22 (0.32)	2.44 (0.4)	2.77 (0.46)	3.07 (0.44)	3.46 (0.47)
Persuasion	2.25 (0.4)	2.42 (0.42)	2.73 (0.41)	3.09 (0.38)	3.07 (0.32)
Written Expression	2.27 (0.34)	2.75 (0.43)	3.26 (0.39)	3.78 (0.27)	3.98 (0.20)
Time Sharing	2.32 (0.37)	2.58 (0.34)	2.71 (0.31)	2.63 (0.31)	2.62 (0.31)
Active Learning	2.38 (0.24)	2.67 (0.34)	3.06 (0.28)	3.41 (0.27)	3.76 (0.26)
Complex Problem-Solving	2.5 (0.33)	2.8 (0.31)	3.15 (0.31)	3.55 (0.32)	3.73 (0.28)
Written Comprehension	2.51 (0.32)	3.01 (0.40)	3.5 (0.39)	3.97 (0.16)	4.11 (0.19)
Service Orientation	2.52 (0.48)	2.69 (0.51)	2.97 (0.47)	3.08 (0.43)	3.24 (0.46)
Reading Comprehension	2.53 (0.3)	2.98 (0.35)	3.41 (0.36)	3.89 (0.2)	4.06 (0.18)
Perceptual Speed	2.54 (0.43)	2.84 (0.41)	2.92 (0.38)	2.79 (0.32)	2.71 (0.39)
Flexibility of Closure	2.56 (0.43)	2.75 (0.37)	2.98 (0.34)	3.03 (0.33)	3.03 (0.38)
Visualization	2.61 (0.46)	2.75 (0.50)	2.92 (0.49)	2.80 (0.49)	2.65 (0.48)
Judgment and Decision Making	2.68 (0.26)	2.9 (0.26)	3.19 (0.28)	3.56 (0.3)	3.73 (0.3)
Inductive Reasoning	2.72 (0.27)	3.00 (0.24)	3.36 (0.36)	3.74 (0.28)	3.98 (0.23)
Category Flexibility	2.72 (0.28)	2.91 (0.23)	3.09 (0.20)	3.31 (0.29)	3.40 (0.32)
Time Management	2.72 (0.23)	2.9 (0.27)	3.1 (0.27)	3.3 (0.31)	3.26 (0.29)
Social Perceptiveness	2.72 (0.32)	2.9 (0.33)	3.15 (0.38)	3.35 (0.41)	3.54 (0.46)
Coordination	2.78 (0.26)	2.94 (0.3)	3.11 (0.37)	3.34 (0.38)	3.27 (0.32)
Deductive Reasoning	2.83 (0.28)	3.09 (0.27)	3.47 (0.33)	3.81 (0.23)	3.96 (0.20)
Selective Attention	2.84 (0.24)	3.03 (0.24)	3.1 (0.23)	3.04 (0.21)	3.07 (0.23)
Critical Thinking	2.85 (0.21)	3.11 (0.28)	3.5 (0.32)	3.85 (0.19)	4 (0.2)
Speaking	2.88 (0.34)	3.17 (0.36)	3.49 (0.37)	3.83 (0.27)	4.04 (0.26)
Monitoring	2.89 (0.23)	3.1 (0.31)	3.28 (0.34)	3.45 (0.35)	3.59 (0.35)
Information Ordering	2.94 (0.19)	3.13 (0.23)	3.38 (0.28)	3.59 (0.30)	3.58 (0.32)
Active Listening	2.96 (0.26)	3.22 (0.35)	3.6 (0.36)	3.88 (0.2)	4.05 (0.21)
Problem Sensitivity	3.04 (0.32)	3.35 (0.35)	3.66 (0.33)	3.79 (0.28)	3.86 (0.38)
Oral Expression	3.07 (0.31)	3.33 (0.40)	3.68 (0.38)	3.97 (0.21)	4.16 (0.25)
Oral Comprehension	3.15 (0.33)	3.39 (0.38)	3.74 (0.34)	3.97 (0.14)	4.10 (0.19)

Note: Cognitive abilities are bolded.

Table 5

Summary of fit statistics for one- to five-dimensional MDS solutions.

Job Zone	Fit Index	Dimensional Solution
		1D	2D	3D	4D	5D
All (average)	Stress	0.27	0.18	0.13	0.10	0.09
	R 2	0.80	0.92	0.95	0.97	0.97
5	Stress	0.34	0.17	0.13	0.09	0.07
	R 2	0.70	0.95	0.97	0.99	0.99
4	Stress	0.27	0.18	0.12	0.08	0.07
	R 2	0.81	0.90	0.95	0.98	0.98
3	Stress	0.14	0.17	0.14	0.11	0.11
	R 2	0.95	0.89	0.92	0.94	0.94
2	Stress	0.24	0.16	0.12	0.97	0.09
	R 2	0.85	0.92	0.95	0.10	0.97
1	Stress	0.34	0.17	0.13	0.11	0.09
	R 2	0.70	0.93	0.95	0.97	0.98

Notes: Stress is a badness of fit measure, with lower values indicating better fit. R² indicates proportion of variance accounted for by the model and so higher values indicate better fit. Shaded cells indicate improvements in fit of interest, as described in the text.

Table 6

Correlations between 2D MDS coordinates and job zone importance ratings.

Variable	Dim. 1	Dim. 2	JZ1	JZ2	JZ3	JZ4	JZ5
Dim. 1	1	0.7 **	−0.7 **	−0.8 **	−0.92 **	−0.98 **	−0.99 **
Dim. 2	0.7 **	1	−0.99 **	−0.99 **	−0.92 **	−0.75 **	−0.66 **
Job Zone 1	−0.7 **	−0.99 **	1	0.98 **	0.91 **	0.75 **	0.66 **
Job Zone 2	−0.8 **	−0.99 **	0.98 **	1	0.96 **	0.83 **	0.75 **
Job Zone 3	−0.92 **	−0.92 **	0.91 **	0.96 **	1	0.94 **	0.9 **
Job Zone 4	−0.98 **	−0.75 **	0.75 **	0.83 **	0.94 **	1	0.96 **
Job Zone 5	−0.99 **	−0.66 **	0.66 **	0.75 **	0.9 **	0.96 **	1

Notes: The sign of the correlation simply notes the direction of the rated abilities or skills in the MDS space. ** p < 0.001.

References

1. Donsbach, J.; Tsacoumis, S.; Sager, C.; Updegraff, J. O*NET Analyst Occupational Abilities Ratings: Procedures; FR-03-22 HumRRO: Alexandria, VA, USA, 2003; Available online: https://www.onetcenter.org/dl_files/AnalystProc.pdf (accessed on 23 May 2024).

2. Haigler, K. The role of occupational research in complex assessments: Aligning educational practices with workplace requirements. J. Writ. Anal.; 2021; 5, pp. 259-291. [DOI: https://dx.doi.org/10.37514/JWA-J.2021.5.1.08]

3. National Research Council (U.S.). Database for a Changing Economy: Review of the Occupational Information Network (O*NET); Tippins, N.; Hilton, M. The National Academies Press: Washington, DC, USA, 2010.

4. Cifuentes, M.; Boyer, J.; Lombardi, D.A.; Punnett, L. Use of O* NET as a job exposure matrix: A literature review. Am. J. Ind. Med.; 2010; 53, pp. 898-914. [DOI: https://dx.doi.org/10.1002/ajim.20846] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/20698022]

5. Burrus, J.; Way, J.D. Using O* NET to develop a framework of job characteristics to potentially improve the predictive validity of personality measures. Pers. Assess. Decis.; 2017; 3, 3. [DOI: https://dx.doi.org/10.25035/pad.2017.003]

6. Feser, E.J. What regions do rather than make: A proposed set of knowledge-based occupation clusters. Urban Stud.; 2003; 40, pp. 1937-1958. [DOI: https://dx.doi.org/10.1080/0042098032000116059]

7. Koo, J. How to analyze the regional economy with occupation data. Econ. Dev. Q.; 2005; 19, pp. 356-372. [DOI: https://dx.doi.org/10.1177/0891242405279910]

8. Bakhshi, H.; Downing, J.M.; Osborne, M.A.; Schneider, P. The Future of Skills: Employment in 2030; Pearson: Mountain View, CA, USA, 2017.

9. Torre, M. Attrition from male-dominated occupations: Variation among occupations and women. Sociol. Perspect.; 2017; 60, pp. 665-684. [DOI: https://dx.doi.org/10.1177/0731121416683160]

10. Lewandowski, P.; Park, A.; Hardy, W.; Du, Y. Technology, skills, and globalization: Explaining international differences in routine and nonroutine work using survey data. World Bank Econ. Rev.; 2022; 36, pp. 687-708. [DOI: https://dx.doi.org/10.1093/wber/lhac005]

11. Thompson, K.R.; Koys, D.J. The management curriculum and assessment journey: Use of Baldrige criteria and the occupational network database. J. Leadersh. Organ. Stud.; 2010; 17, pp. 156-166. [DOI: https://dx.doi.org/10.1177/1548051810369341]

12. Homan, S.R.; Sandall, D. An analysis of the results of an occupational information network (O*NET) curriculum needs assessment performed by the faculty of the Organizational Leadership and Supervision Department at Purdue University. Proceedings of the Association of Leadership Educators Conference; Anchorage, AK, USA, 16–19 July 2003.

13. Oliveri, M.E.; McCulla, L. Using the occupational network database to assess and improve English language communication for the workplace. ETS Res. Rep. Ser.; 2019; 2019, pp. 1-17. [DOI: https://dx.doi.org/10.1002/ets2.12265]

14. Burrus, J.; Jackson, X.; Steinberg, S. Identifying the most important 21st century workforce competencies: An analysis of the Occupational Information Network (O*NET). ETS Research Report Series; Educational Testing Service: Princeton, NJ, USA, 2013.

15. Davison, M.L.; Sireci, S.G. Multidimensional scaling. Handbook of Multivariate Statistics and Mathematical Modeling; Tinsley, H.E.A.; Brown, S. American Psychological Association: Washington, DC, USA, 2000; pp. 325-349.

16. National Center for O*NET Development. O*NET Online. Available online: https://www.onetonline.org/ (accessed on 1 June 2021).

17. Bureau of Labor Statistics, U.S. Department of Labor. Occupational Outlook Handbook. Available online: https://www.bls.gov/soc/ (accessed on 1 June 2024).

18. Rivkin, D.; Craven, D.E. Procedures for O*NET Job Zone Assignment: Updated to Include Procedures for Developing Preliminary Job Zones for New O*NET-SOC Occupations; U.S. Department of Labor: Washington, DC, USA, 2021; Available online: https://www.onetcenter.org/dl_files/JobZoneProcedureUpdate.pdf (accessed on 1 June 2024).

19. Carroll, J.D.; Chang, J.J. An analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition. Psychometrika; 1970; 35, pp. 238-319. [DOI: https://dx.doi.org/10.1007/BF02310791]

20. Sireci, S.G.; Bastari, B.; Allalouf, A. Evaluating construct equivalence across adapted tests. Proceedings of the Annual Meeting of the American Psychological Association (Division 5); San Francisco, CA, USA, 17 August 1998.

21. Young, F.W.; Harris, D.F. Multidimensional scaling. SPSS for Windows: Professional Statistics; Computer Manual, Version 6.0 Noursis, M.J. SPSS, Inc.: Chicago, IL, USA, 1993; pp. 155-222.

22. Kruskal, J.B.; Wish, M. Multidimensional Scaling; Sage: Newbury Park, CA, USA, 1978.