Statistical Significance Examples in Statistics

Start with the recap, study the fully worked examples, then use the practice problems to check your understanding of Statistical Significance.

This page combines explanation, solved examples, and follow-up practice so you can move from recognition to confident problem-solving in Statistics.

Concept Recap

A result is statistically significant when the p-value falls below a predetermined threshold (alpha, typically 0.05), indicating that the observed effect is unlikely to have occurred by random chance alone. Statistical significance is a binary decision criterion used in hypothesis testing — it does not measure the size or practical importance of the effect.

Statistical significance is a decision rule: before looking at data, you set a threshold (usually 5%). If your p-value is below this threshold, you declare the result 'significant' - meaning unlikely to be just random noise. It's not about importance; it's about confidence that something real is happening.

Read the full concept explanation →

How to Use These Examples

  • Read the first worked example with the solution open so the structure is clear.
  • Try the practice problems before revealing each solution.
  • Use the related concepts and background knowledge badges if you feel stuck.

What to Focus On

Core idea: Statistical Significance uses a sample result and a variation model to make a careful population statement.

Common stuck point: Students often know a procedure related to statistical significance but skip the recognition step: Am I using sample-to-sample variation to make a population claim with uncertainty stated clearly? That leads to a calculation or graph that looks reasonable but answers a different question.

Sense of Study hint: Ask: Am I using sample-to-sample variation to make a population claim with uncertainty stated clearly?

Worked Examples

Example 1

medium
A test gives p=0.07p=0.07 at α=0.10\alpha=0.10. Is the result statistically significant? Would it be at α=0.05\alpha=0.05?

Answer

Yes at α=0.10; No at α=0.05\text{Yes at } \alpha=0.10; \text{ No at } \alpha=0.05

First step

1
At α=0.10\alpha=0.10: 0.07≤0.100.07 \le 0.10, reject.

See the full worked solution + why-it-works coaching

SetupKey insightWhy it worksCommon pitfallConnection

Unlock answer keys One Family plan — every worked solution, all subjects

Example 2

hard
An experiment reports p=0.045p=0.045 for a treatment effect of 0.20.2 standard deviations. Is the result statistically significant and important?

Example 3

hard
An experiment runs at α=0.05\alpha=0.05 but is repeated 4 independent times against the same H0H_0. What is the family-wise probability of at least one false rejection when H0H_0 is true?

Example 4

challenge
A two-sided test of H0:μ=0H_0:\mu=0 gives a 95% CI of [0.4, 0.8][0.4,\ 0.8] for μ\mu. Is the result statistically significant at α=0.05\alpha=0.05? What about at α=0.01\alpha=0.01 if we know the corresponding 99% CI is [0.3, 0.9][0.3,\ 0.9]?

Example 5

hard
A drug trial finds a statistically significant reduction in blood pressure (p=0.02p = 0.02). The mean reduction was 2 mmHg. Is this result practically significant?

Example 6

hard
Explain the difference between a Type I error and a Type II error in hypothesis testing.

Practice Problems

Try these problems on your own first, then open the solution to compare your method.

Example 1

easy
A result is statistically significant when the p-value falls below what?

Example 2

easy
What is the most common value of the significance level α\alpha?

Example 3

easy
Is statistical significance a measure of how large an effect is?

Example 4

easy
With p =0.03=0.03 and α=0.05\alpha=0.05, is the result statistically significant?

Example 5

easy
With p =0.08=0.08 and α=0.05\alpha=0.05, is the result statistically significant?

Example 6

easy
Statistical significance is a ____ decision (yes/no), not a continuous measure.

Example 7

easy
When should the significance level α\alpha be chosen?

Example 8

easy
If α=0.05\alpha=0.05 and the null is true, what fraction of experiments will falsely be called significant?

Example 9

medium
A drug lowers blood pressure by 0.2 mmHg with p =0.001=0.001 in a huge trial. Is it statistically significant? Is it practically important?

Example 10

medium
Using a stricter α=0.01\alpha=0.01 instead of 0.05, does it become easier or harder to declare significance?

Example 11

medium
Why is using α=0.05\alpha=0.05 for a high-stakes medical decision potentially inappropriate?

Example 12

medium
A 95% confidence interval for an effect is [0.5, 2.0] and excludes 0. Is the effect statistically significant at α=0.05\alpha=0.05?

Example 13

medium
Researchers run 40 subgroup analyses and highlight the one with p =0.04=0.04 as 'significant.' Why is this misleading?

Example 14

medium
A study reports 'not significant (p =0.30=0.30).' Does this prove there is no effect?

Example 15

medium
Two results: A has p =0.04=0.04 with a large effect, B has p =0.04=0.04 with a tiny effect. Are they equally important?

Example 16

medium
A result has p =0.001=0.001 at α=0.05\alpha=0.05. Is it statistically significant?

Example 17

medium
Using α=0.01\alpha=0.01, a test gives p =0.03=0.03. Is the result significant at this level?

Example 18

challenge
A trial sets α=0.05\alpha=0.05. A result gives z=1.9z=1.9 (two-sided p ≈0.057\approx 0.057). The team lowers the bar to α=0.10\alpha=0.10 after seeing this to claim significance. Identify the methodological error.

Example 19

challenge
Out of 100 independent tests with all nulls true and α=0.05\alpha=0.05, about how many significant results occur, and what does this teach about a single 'significant' finding among many?

Example 20

challenge
Explain why a statistically significant result with a 95% CI of [0.1%, 0.3%] improvement might still lead a company to NOT adopt a change.

Example 21

easy
At α=0.05\alpha=0.05, is a result with p=0.049p=0.049 statistically significant?

Example 22

easy
At α=0.01\alpha=0.01, is a result with p=0.03p=0.03 statistically significant?

Example 23

easy
True or false: at α=0.05\alpha=0.05, if H0H_0 is actually true, about 5% of experiments will be 'statistically significant.'

Example 24

easy
A scientist sets α=0.05\alpha=0.05 but reports significance after seeing the data is close. Why is this problematic?

Example 25

medium
A study reports a 95% CI for an effect of [0.01, 0.04][0.01,\ 0.04]. Is the effect statistically significant at α=0.05\alpha=0.05?

Example 26

medium
Why might a huge clinical trial detect a 'statistically significant' but irrelevant 0.05 mmHg drop in blood pressure?

Example 27

medium
A team reports many subgroup analyses at α=0.05\alpha=0.05 each and highlights the one with p=0.01p=0.01. Why is this misleading?

Example 28

medium
A 99% CI for a difference is [−2, 5][-2,\ 5]. Is the difference statistically significant at α=0.01\alpha=0.01?

Example 29

medium
A drug trial fails to show statistical significance (p=0.20p=0.20, n=20n=20). Why might it still be wrong to conclude 'the drug doesn't work'?

Example 30

medium
Two independent studies report p=0.05p=0.05 each. Is the combined evidence stronger?

Example 31

hard
A team prefers α=0.005\alpha=0.005 for a new drug-approval pipeline. Why is this stricter threshold sensible?

Example 32

hard
A journal publishes only statistically significant results. What bias does this create in the literature?

Example 33

hard
A start-up reports 'the new feature led to a statistically significant lift of 0.1%.' Should the engineering team ship it?

Example 34

medium
True or false: 'statistically significant' is a continuous measure of how strong the evidence is.

Example 35

medium
A study reports p=0.04p=0.04 and a 95% CI of [0.01, 0.40][0.01,\ 0.40] for an effect. What does the CI add over the p-value?

Example 36

hard
A team plans an A/B test with target α=0.05\alpha=0.05 and 80% power to detect a 1% lift. Suppose during the test they peek and stop early when significance is hit. What is the danger?

Example 37

hard
A study with 10,000 participants finds a statistically significant difference in test scores between two teaching methods (p=0.001p = 0.001), but the difference is only 0.5 points out of 100. Discuss.

Example 38

hard
A treatment improves average test scores by 12 points, but the p-value is 0.08. At α=0.05\alpha = 0.05 is the result statistically significant, and could the effect still be practically important?
← Back to Statistical Significance Practice Mode

Background Knowledge

These ideas may be useful before you work through the harder examples.

p valuehypothesis testing