Power and False Negatives in Qualitative Comparative Analysis: Foundations, Simulation and Estimation for Empirical Studies

Articles

Contributing Editor: Jonathan N. Katz.

1

Introduction

The consistency value fulfills an important role in Qualitative Comparative Analysis (QCA). Empirical QCA researchers use it as one, if not exclusive criterion to distinguish associations between a term and an outcome supporting the inference that a set relation is present from associations that fail to support it (Ragin 2006).¹Braumoeller (2015) recently pointed out that the consistency score is unsuitable for this purpose because we need to know the probability of obtaining such a value under the null hypothesis that there is, in fact, no set relation. Without determining the probability, we might commit a false positive by incorrectly inferring that a set relation is in place when it is not. Braumoeller shows for fuzzy-set QCA (fsQCA) that permutation tests allow one to derive the distribution of consistency values for calculating the [formula omitted, refer to PDF] value and statistical significance for the observed consistency score.²

Permutation tests and [formula omitted, refer to PDF]-value calculation are valuable additions to the QCA toolbox for avoiding false positives. However, it is equally valuable to know the probability of rejecting the null hypothesis when it is false. This is a matter of the power of a truth table analysis that is inversely related to the probability of committing a false negative. So far, there has been no consideration of the role of power for truth table analyses in the QCA literature, although it is essential to avoiding producing and interpreting false QCA solutions.³

In this paper, I introduce power analysis to further the development of truth table analyses and equip empirical researchers with a new tool that will contribute to the validity of their results. I first discuss what a null hypothesis and an alternative hypothesis are in QCA because they need to be specified for power estimation (Section 2). Throughout the paper, I discuss power analysis for set-relational analyses on sufficiency and note here that all arguments generalize to studies of necessity because they invoke consistency values for making similar decisions. Drawing on the distinction between a null and alternative hypothesis, I elaborate on the benefits of high power for a truth table analysis and the consequences of low-powered studies (Section 3). I argue...