Content area
Full text
DAVID CLARK-CARTER on why you just can't carry on reporting statistical significance alone.
JUST when you've got to grips with how to decide whether the result of your research is statistically significant, you find that some people are suggesting that you need to do things differently. The latest edition of the American Psychological Association's Publication Manual identifies 'failure to report effect sizes' as one of the 'defects in the design and reporting of research' (APA, 2001, p.5). In addition, the British Psychological Society now has a statement in the 'Notes for contributors' for all its journals that in normal circumstances, effect size should be incorporated. This article explains why statistical significance, on its own, is inadequate as a way of deciding the worth of a piece of research and suggests how we can use a combination of types of evidence to come to our decisions.
By the 1930s some psychologists had already been using a form of statistical analysis similar to today's. However, it wasn't until they had read Fisher (1935) that the process became formalised and psychologists started to use the tests and some of the conventions that we now employ. According to that approach, a null hypothesis is stated - the equivalent of saying that there is no effect, when looking at differences between groups, or no relationship in the case of correlation. Thus, if our study was into the effectiveness of an intervention to reduce truancy compared with a control group who received no intervention, then the null hypothesis is likely to be The intervention and control groups show the same amount of truancy. If the study was about the relationship between two variables, for example locus of control (LoC) and willingness to initiate a conversation in a job interview, the null hypothesis might be There is no relationship between LoC and willingness to initiate a conversation in a job interview.
The data from the study are collected and analysed and a probability is produced. That probability is answering the question: How likely would be the result of my study if the null hypothesis were true? We then decide whether that probability is sufficiently unlikely that we can reject the null hypothesis; conventionally we decide to reject the null hypothesis if...