Content area
Data visualization is a multidisciplinary field that focuses on the visual representation and communication of data and information. As a relatively young field, data visualization has many unresolved, debated, and untested theories, methods, practices, guidelines, and assumptions that require further validation and strengthening. The most common mechanisms to achieve this type of validation are evaluation and replication studies. Replication studies examine prior research to confirm the validity and dependability of the findings. Although visualization researchers acknowledge the epistemological significance of replication studies, the field's current publication culture does not readily accommodate such studies. More conventional design studies and evaluations are better incentivized for publication but most of them, especially design studies, can be very tedious and time-consuming, taking months to even years to complete. Meta-analysis, post hoc analysis, and surveys of design studies can be used to validate and develop theories and frameworks specific to the domain being addressed or even for more general applications. However, conducting enough design studies to implement such rigorous analysis to validate or test theories may take years. The lengthy time commitments also make certain design studies less incentivized, particularly the ones that do not guarantee novel outcomes, such as collaborations with nonprofit organizations. Researchers are driven to work on novel problems and are reluctant to invest time to conduct replications and certain design studies that are difficult to publish, despite the epistemological significance or societal impact this type of research may carry.
The overarching aim of this dissertation is to address the challenge of quickly and effectively validating visualization theories and methods and providing incentives and solutions to the visualization community to conduct this type of hard-to-publish but extremely important research. The field of data visualization is rapidly gaining popularity due to its multidisciplinary nature and significance in today's data-driven world, prompting more institutions to offer visualization courses. This increasing number of learners presents us with an opportunity to conduct such research in the classroom, which will not only enhance visualization pedagogy and benefit the students, but also benefit the visualization community. These learners are potential visualization researchers and practitioners, and the skills and values we instill in them will help them build a stronger visualization community in the future. Therefore, my dissertation aims to answer the following research questions:
1. How can we leverage visualization learners to conduct design studies more rapidly to facilitate theory validation?
2. How can we leverage visualization learners to conduct and systematically report more replication studies?
This dissertation aims to address these questions through two approaches. Firstly, this work contributes methods and frameworks to leverage visualization learners to conduct design and replication studies within the classroom as part of their course curricula. Secondly, this work uses the results of these student-led studies to develop, validate, and extend frameworks and theories. Therefore, the contribution of my dissertation is to provide incentives and methodologies to facilitate more replication studies and design studies in the visualization community by leveraging visualization learners to conduct these studies within their classrooms. In the current “publish or perish” environment, this will provide alternative incentives for researchers to conduct replications, which are imperative to advance the field and design studies that can be used to validate and develop theories, while also having the potential to do social good.
This dissertation is divided into two parts. The first part focuses on rapidly conducting more design studies in the classroom to validate and develop theories and methodologies, benefit students, and to also facilitate visualization for social good. The second part focuses on conducting more replication studies both in the classroom and in research to validate, strengthen, and advance the field of data visualization.