P-Value Math Example 4
Follow the full solution, then compare it with the other examples linked below.
Example 4
hardStudy A: , , effect size (tiny). Study B: , , effect size (large). Discuss which study's result is more practically meaningful and why we shouldn't rely solely on p-values.
Solution
- 1 Study A: significant at ฮฑ=0.05 but effect size d=0.15 is trivially small โ large n detected an unimportant difference
- 2 Study B: not significant but d=0.8 is a large effect โ small n had insufficient power to detect a real, large effect
- 3 Study B is more practically meaningful despite not being 'statistically significant'
- 4 Lesson: p-value depends on both effect size AND sample size; always report both; significance โ importance
Answer
Study B (large effect, underpowered) is more practically meaningful than Study A (trivial effect, large n).
This illustrates the critical importance of effect size alongside p-values. P-values conflate effect magnitude with sample size. With large enough n, even meaningless effects become 'significant.' Practical significance (effect size) must be evaluated separately from statistical significance.
About P-Value
The probability of observing a test statistic at least as extreme as the one computed from the sample data, assuming the null hypothesis is true.
Learn more about P-Value โMore P-Value Examples
Example 1 medium
A hypothesis test produces [formula] for a two-tailed test. Calculate the p-value and interpret it a
Example 2 hardCorrect the following misconceptions about p-values: (a) 'p=0.03 means there's a 3% chance Hโ is tru
Example 3 easyA one-tailed test has [formula]. Find the p-value and determine if we reject [formula] at [formula].