Statistical Significance Statistics Example 4

Follow the full solution, then compare it with the other examples linked below.

Example 4

hard
A treatment improves average test scores by 12 points, but the p-value is 0.08. At ฮฑ=0.05\alpha = 0.05 is the result statistically significant, and could the effect still be practically important?

Solution

  1. 1
    Step 1: Since 0.08>0.050.08 > 0.05, the result is not statistically significant at the 0.05 level.
  2. 2
    Step 2: A 12-point improvement could still matter in practice, so practical importance and statistical significance are not the same thing.

Answer

It is not statistically significant at ฮฑ=0.05\alpha = 0.05, but it could still be practically important.
Statistical significance depends on the strength of the evidence relative to sampling variability, while practical significance depends on whether the size of the effect matters in the real world.

About Statistical Significance

A result is statistically significant when the p-value falls below a predetermined threshold (alpha, typically 0.05), indicating that the observed effect is unlikely to have occurred by random chance alone. Statistical significance is a binary decision criterion used in hypothesis testing โ€” it does not measure the size or practical importance of the effect.

Learn more about Statistical Significance โ†’

More Statistical Significance Examples

Example 1 hard

A drug trial finds a statistically significant reduction in blood pressure ([formula]). The mean red

Example 2 hard

Explain the difference between a Type I error and a Type II error in hypothesis testing.

Example 3 hard

A study with 10,000 participants finds a statistically significant difference in test scores between

โ† All Statistical Significance Examples Statistical Significance Concept