Practice Statistical Significance in Statistics

Use these practice problems to test your method after reviewing the concept explanation and worked examples.

Worksheet PDF Free Answer-key PDF Family

Quick Recap

A result is statistically significant when the p-value falls below a predetermined threshold (alpha, typically 0.05), indicating that the observed effect is unlikely to have occurred by random chance alone. Statistical significance is a binary decision criterion used in hypothesis testing — it does not measure the size or practical importance of the effect.

Statistical significance is a decision rule: before looking at data, you set a threshold (usually 5%). If your p-value is below this threshold, you declare the result 'significant' - meaning unlikely to be just random noise. It's not about importance; it's about confidence that something real is happening.

Showing a random 20 of 50 problems.

Example 1

hard
An experiment runs at α=0.05\alpha=0.05 but is repeated 4 independent times against the same H0H_0. What is the family-wise probability of at least one false rejection when H0H_0 is true?

Example 2

easy
Is statistical significance the same as proving causation?

Example 3

easy
A scientist sets α=0.05\alpha=0.05 but reports significance after seeing the data is close. Why is this problematic?

Example 4

hard
A team plans an A/B test with target α=0.05\alpha=0.05 and 80% power to detect a 1% lift. Suppose during the test they peek and stop early when significance is hit. What is the danger?

Example 5

easy
If α=0.05\alpha=0.05 and the null is true, what fraction of experiments will falsely be called significant?

Example 6

easy
Statistical significance is a ____ decision (yes/no), not a continuous measure.

Example 7

easy
A result is statistically significant when the p-value falls below what?

Example 8

medium
Using α=0.01\alpha=0.01, a test gives p =0.03=0.03. Is the result significant at this level?

Example 9

medium
A 95% confidence interval for an effect is [0.5, 2.0] and excludes 0. Is the effect statistically significant at α=0.05\alpha=0.05?

Example 10

medium
Two independent studies report p=0.05p=0.05 each. Is the combined evidence stronger?

Example 11

medium
Two results: A has p =0.04=0.04 with a large effect, B has p =0.04=0.04 with a tiny effect. Are they equally important?

Example 12

challenge
A trial sets α=0.05\alpha=0.05. A result gives z=1.9z=1.9 (two-sided p ≈0.057\approx 0.057). The team lowers the bar to α=0.10\alpha=0.10 after seeing this to claim significance. Identify the methodological error.

Example 13

medium
Why is using α=0.05\alpha=0.05 for a high-stakes medical decision potentially inappropriate?

Example 14

easy
With p =0.03=0.03 and α=0.05\alpha=0.05, is the result statistically significant?

Example 15

medium
A drug trial fails to show statistical significance (p=0.20p=0.20, n=20n=20). Why might it still be wrong to conclude 'the drug doesn't work'?

Example 16

medium
A drug lowers blood pressure by 0.2 mmHg with p =0.001=0.001 in a huge trial. Is it statistically significant? Is it practically important?

Example 17

hard
A treatment improves average test scores by 12 points, but the p-value is 0.08. At α=0.05\alpha = 0.05 is the result statistically significant, and could the effect still be practically important?

Example 18

medium
Fill in: at α=0.10\alpha=0.10, it becomes ____ to declare significance than at α=0.05\alpha=0.05.

Example 19

easy
At α=0.05\alpha=0.05, is a result with p=0.049p=0.049 statistically significant?

Example 20

medium
A result has p =0.001=0.001 at α=0.05\alpha=0.05. Is it statistically significant?
← Back to Statistical Significance Worked Examples