Investigating Expertise Transfer and Experts’ Cognitive Decision-Making Processes in Socio-Technical Systems

Abstract

Socio-technical systems are becoming increasingly complex and require more expertise to solve the problems associated with these systems. Even though technological advancements of data collection and processing are enhancing decision-making processes, there is always an expert judgment process to make sense of complexity. As experts judgments are crucial for improving socio-technical systems, it is necessary to understand how experts perceive complex information, contextualize it, and incorporate it into their judgments, all of which are cognitive processes. Therefore, in this dissertation, our objective is to explore the cognition of expert judgments. Specifically, we aim to address the research questions of what factors influence expert judgment processes, how experts perceive risk, process information, and incorporate this information into their judgments. In order to address these questions, we draw on a cognitive decision-making theory Fuzzy Trace Theory in order to explore the cognition of expertise development and transfer. We also examine experts’ judgment performances by focusing on the Structured Expert Judgment protocol. By relying on these research frameworks, we worked to explore the most effective information transfer strategies in the context of expert judgments. We conducted several survey-based research methodologies in which we collected data from various samples, such as NASA scientists and engineers, laypeople, and medical experts. Into those surveys, we embedded experimental designs in order to test the hypotheses that were theoretically motivated by Fuzzy Trace Theory. In addition to these social science methodologies, we analyzed expert performances in the context of Structured Expert Judgment (SEJ) protocol. We performed simulations in order to investigate the best practices to aggregate expert judgments. Our research findings showed that experts primarily rely on gist representations, which is qualitative and the least precise form of information. Expertise enables the ability to contextualize information and make meaningful connections within complex data. This gist-based thinking, which increases as a result of expertise development, drives decisions, and ultimately leads to good judgments. Also, gist transfer is effective in making novices’ risk perceptions similar to those of experts. Verbatim information transfer, which is quantitative and the most precise form of information, is not as effective on its own, and therefore should be combined with a gist transfer. We also showed that SEJ, which is based on performance-weight opinion pooling, outperforms the equal-weight approach as we showed that the differences among expert performances are not due to random stressors. Therefore, these differences need to be incorporated into the aggregation process in order to maximize the quality of the expert judgment data. Finally, psychometric paradigms such as experts’ numerical ability have been shown to influence experts’ judgment performances. We showed that experts, regardless of their professional background, are prone to similar cognitive processes of judgment making, which is predicted by Fuzzy Trace Theory. Experts rely primarily on gist-based thinking, which is an advanced mode of cognition. Therefore, gist-based information transfer should be utilized when transferring information to and from experts, particularly in complex socio-technical systems. Since our findings showed that expert judgment performances are not solely a function of experts’ knowledge and are influenced by experts’ numerical ability, SEJ protocols should be designed by taking into account experts’ individual differences. Our dissertation has important implications for those who design, implement, and use socio-technical systems, as experts play a crucial role in improving such systems. Future studies should further explore the implications of cognitive processes of expertise transfer and development. Future work could also entail designing effective platforms for novices to develop substantial expertise.