Art of Code Review: Examining the Usefulness of Code Review Comments in Commercial, Open-Source, and Scientific Software

Code Review is crucial for advancing computer science. It ensures high-quality software, contributes to the reliability of artificial intelligence (AI) systems, and addresses security vulnerabilities. Effective peer code review requires helpful comments and supportive automated tools in collaborative development. It is important to ensure these Code Review comments serve their purposes. This dissertation reflects the evolution of research on the usefulness of Code Review comments. First, I examine papers that define the usefulness of Code Review comments, mine and annotate datasets, study developers’ perceptions, analyze factors from different aspects, and use classifiers to predict the usefulness of Code Review comments automatically. Second, I investigate the usefulness of Code Review comments through novel textual feature-based and untapped featureless approaches. Third, I present the generalization performance of the benchmark models and datasets, subsequently providing their explanations. Fourth, I dive into the utility of Code Review comments by scrutinizing the non-verbal cues that often carry emotive or instructive weight for developers. Finally, I investigate the utility of Code Review comments in the unexplored scientific software domain. My investigation into scientific software extends findings from Code Review comment usefulness research on general-purpose software, benefiting developers, scientists, and software engineering researchers in both the general and scientific software communities.