9. Hypothesis testing

This chapter explores hypothesis testing, the second foundational concept in inferential statistics, after estimation. At its core, hypothesis testing involves determining whether the data support a specific theory about the world. Despite its conceptual simplicity, many find hypothesis testing challenging due to its detailed procedures and the inherent psychological biases that accompany interpretation. The chapter begins with an overview of the logic and mechanics of hypothesis testing, explaining its foundational elements through practical examples. Emphasis is placed on understanding the relationship between research hypotheses, which make substantive claims about the world, and statistical hypotheses, which translate these claims into mathematically precise statements about population parameters.

The chapter delves into key distinctions, such as those between null and alternative hypotheses, and explains the principles of decision-making under uncertainty. Concepts such as Type I and Type II errors, significance levels, power, and effect size are discussed to highlight the trade-offs inherent in hypothesis testing. By using examples like testing for extrasensory perception (ESP), the chapter illustrates the importance of critical regions, sampling distributions, and test statistics in the testing process. Issues surrounding the interpretation of p-values, reporting standards, and the philosophical underpinnings of hypothesis testing frameworks (e.g., Neyman-Pearson versus Fisherian perspectives) are critically examined. The chapter concludes by addressing common pitfalls and the need for thoughtful analysis, preparing readers to apply hypothesis testing responsibly in scientific research.