On this concept: Formula

Not sure which lesson to do first? Find your level and get a plan →

Math · Statistics & Probability · Grade 9-12

Power of a Test

Power is the probability a hypothesis test correctly rejects a false null hypothesis — its ability to detect a real effect — equal to 1−β.

Power=1−β=P(reject H0∣H0 is false)

Quick Answer

Power is the probability a hypothesis test correctly rejects a false null hypothesis — its ability to detect a real effect — equal to 1−β. You use it when planning a study (how many subjects are needed?) and when judging a result that came back 'not significant'. Power rises with a larger sample, a larger true effect, less noisy data and a larger α. Researchers usually aim for 0.80 or more; a study with power 0.30 misses a real effect seven times out of ten.

Why This Matters

Power tells you whether a study could ever have found what it was looking for. With it you can choose a sample size before collecting any data, and you can read a 'not significant' result correctly: from a high-power study it suggests the effect is small or absent, but from a low-power study it says almost nothing. Ignoring power produces small studies that miss treatments that really work, followed by false claims of 'no effect'.

Intuitive Explanation

Picture a smoke alarm in a kitchen where a pan really is burning. Power is the chance the alarm goes off. A sensitive alarm right above the stove almost always sounds; a cheap one down the hallway often stays quiet. The fire is real either way; what differs is how well the detector picks it up. In a hypothesis test the 'fire' is a real effect (so H0 is false), and 'the alarm sounds' means the test rejects H0.

Three things make a test more sensitive. A bigger effect is easier to catch: a drug that lowers blood pressure by 15 mmHg is detected more often than one that lowers it by 2. More data helps: the standard error is σ/n, so four times as many subjects halves the noise and the same effect stands out. A looser threshold helps too: raising α from 0.01 to 0.05 makes rejection easier, at the price of more false alarms.

Do not confuse power with α. Both are a 'probability of rejecting H0', but in opposite worlds. α is the rejection rate when H0 is true, an alarm with no fire. Power is the rejection rate when H0 is false, the alarm doing its job. Read which world the problem describes: 'the drug actually works' points to power; 'the drug is useless but the test says it works' points to α.

Think of it like this. Power works like a fishing net's chance of landing a fish that really is in the lake. A bigger fish (a larger effect) or a finer mesh (a larger sample) raises the catch rate. Unlike a net, a test can also 'catch' a fish that is not there; that false catch is α, a separate number.

Core idea. Power is the probability a test correctly rejects a false null: 1−β.


When to Use

Reach for power when the problem says the effect is real or the null is false and asks how likely the test is to detect it, or when it asks how large a sample a study needs. It also belongs in any discussion of a 'not significant' result. If the problem describes a true null and a wrong rejection, that is α; if it asks how often a real effect is missed, that is β; if it asks how often an interval captures a parameter, that is a confidence level.

Tip. Ask: Am I asking for the probability of correctly rejecting the null GIVEN it is false (the detection rate)?

How to Recognize It

Before working with power, check whether the problem assumes the null hypothesis is true or false and whether the study is finished or still being planned.

  1. Does the problem state or assume that a real effect exists, such as a coin that truly lands heads 60% of the time?

    If yes, the chance of rejecting is the power. If it assumes the null is true, the chance of wrongly rejecting is the significance level α, a different number set before the test.

  2. Is it asking how often the test would miss the real effect?

    If yes, that is β, the Type II error rate, and power is 1−β: a 0.25 chance of missing means power 0.75. Name which of the two the question wants before writing a number.

  3. Is the study still being planned, with a sample size to choose?

    If yes, power drives the choice: find the smallest n that reaches a target, often 0.80. Doubling the sample does not double the power; it shrinks the standard error by a factor of 2.

  4. Did a study come back not significant, and is someone now claiming there is no effect?

    If yes, ask how much power the study had. A small, low-power study often misses real effects, so its result means the effect was not found, not that it is absent. A confidence interval shows how large an effect is still possible.

Power of a Test compared with Type II error β, Significance level α, and Confidence level

All four rows are long-run rates tied to a test or an interval, so they are easy to swap. Ask what the rate assumes: a false null, caught (power) or missed (β); a true null wrongly rejected (α); or an interval capturing a parameter (confidence level).

Power of a Test

Meaning
Use this when the null is actually false and you want the probability the test correctly detects the effect.
Key test
Am I asking for the probability of correctly rejecting the null GIVEN it is false (the detection rate)?
Formula
Power=1−β=P(reject H0∣H0 is false)
Example
A drug truly lowers blood pressure, and a trial is designed so that β=0.20. What is the power, and what does it mean?

Type II error β

Meaning
The probability of MISSING a real effect; power is its complement.
Key test
Use when describing the failure-to-detect rate.
Formula
β=P(fail to reject∣H0 false)
Example
20% chance of missing a true effect when power is 0.80

Significance level α

Meaning
The false-positive rate when the null is TRUE, set before the test.
Key test
Use when describing how often a true null is wrongly rejected.
Formula
α=P(reject∣H0 true)
Example
α=0.05

Confidence level

Meaning
An interval's long-run capture rate, unrelated to detecting a specific false null.
Key test
Use when estimating a parameter with an interval.
Formula
1−α
Example
95% confidence interval

Formula & Notation

Power=1−β=P(reject H0∣H0 is false)
Power=1−β=P(reject H0∣Ha true) where β=P(Type II error)

How to read it: Power =1−β. β=P(Type II error).

Worked Examples

Example 1 — Reading power

Easy

A drug truly lowers blood pressure, and a trial is designed so that β=0.20. What is the power, and what does it mean?
Show the steps and the answer
  1. Note what is given: the drug really works, so H0 (no effect) is false, and β=0.20.

    β is the chance this trial misses the real effect, a Type II error.

  2. Take the complement: power =1−β=1−0.20=0.80.

    When H0 is false the test either rejects or fails to reject, so those two probabilities add to 1.

  3. Interpret: if this trial were run many times on this effective drug, about 80 runs in 100 would give a significant result and about 20 would not.

    Power is a long-run detection rate for an effect of this size, not the probability that the drug works.

Answer: Power =0.80

Takeaway: When the effect is real, the test either catches it (probability 1−β) or misses it (probability β).

Example 2 — False-alarm rate

Standard

Instead you're told the test wrongly flags an ineffective drug as working 5% of the time. Is that the power?
Show the steps and the answer
  1. Ask which world the 5% lives in: the drug is ineffective, so H0 is true.

    Power is defined only when H0 is false; this number conditions on the opposite case.

  2. Name it: rejecting a true H0 is a Type I error, so this is α=0.05.

    α is the false-alarm rate, chosen before any data are collected.

  3. Note what power would need: β, or else the true effect size, the sample size n and the spread of the data.

    α alone does not fix power; a test at α=0.05 can have power 0.3 with a small sample and 0.95 with a large one.

Answer: No — that's α=0.05, not power

Takeaway: α is how often the test raises an alarm when nothing is there; power is how often it raises one when something is.

Example 3 — Find the mistake: 'not significant' means 'no effect'

Application

A pilot study of a study app uses 12 students with α=0.05 and has power 0.30 to detect a 5-point gain in test scores. The result is not significant (p=0.21). Lee writes: "The test did not reject H0, so the app has no effect." Find the mistake and correct it.
Show the steps and the answer
  1. Find the miss rate: β=1−0.30=0.70.

    Even if the app really adds 5 points, this study fails to reject H0 70% of the time.

  2. Compare with Lee's claim: failing to reject is the most likely outcome here whether or not the app works.

    A result the test would usually produce anyway cannot count as evidence that the effect is zero.

  3. Rewrite the conclusion: there is not enough evidence of an effect, and the study was too small to tell.

    A non-significant result means the data are consistent with no effect, not that no effect exists.

  4. Suggest the fix: a larger study. With 4 times as many students the standard error σ/n is halved, so a 5-point gain stands out twice as clearly.

    Sample size is the lever researchers control most directly when they want more power.

Answer: The study is inconclusive, not proof of no effect: with power 0.30 it misses a real 5-point gain 70% of the time.

Takeaway: Before reading 'not significant' as 'no effect', check the power; a low-power test usually fails to reject even when the effect is real.

Where You'll See It

  • Sizing a blood-pressure drug trial

    Researchers pick the smallest effect worth finding, say 5 mmHg, and choose n so that power is at least 0.80. If 30 patients give power of about 0.65, roughly 1 trial in 3 would miss a drug that works; about 100 patients push power above 0.95.

  • A/B testing a website button

    A company hopes a new button raises clicks from 10% to 11%. To detect a 1-point difference with power 0.80 at α=0.05, it needs about 15,000 visitors per version. Stopping after 500 visitors and declaring 'no difference' is a low-power mistake.

Common Mistakes

  1. The slip: Confusing power with α

    Instead: power conditions on H0 being FALSE; α conditions on H0 being TRUE.

  2. The slip: Thinking power and β add to nothing useful

    Instead: power =1−β, so a Type II error rate of 0.2 means power 0.8.

  3. The slip: Believing a non-significant result proves no effect

    Instead: a low-power test often misses real effects, so 'not significant' isn't 'no effect.'


Mini Practice

Try these on your own. Tap Reveal when you want to check.

  1. A study is designed so that β=0.10. What is its power, and what does that number mean?

    Hint: Power and β add up to 1.

  2. A test has power 0.65. In 200 studies where the effect is real, about how many will detect it, and how many will miss it?

    Hint: Multiply 200 by the power, then by 1−power.

  3. Which change raises the power of a test: (a) increasing n from 50 to 200, or (b) lowering α from 0.05 to 0.01? Assume nothing else changes.

    Hint: Think about how each change affects how easy it is to reject H0.

  4. A test of H0:μ=100 rejects when xˉ>104.9. The standard error is 3, and the true mean is 106. Find the power, using P(Z>−0.37)≈0.64.

    Hint: Find the z-score of the cutoff 104.9 when the true mean is 106.

  5. A researcher says: "If our drug truly does nothing, our test will wrongly call it effective 1% of the time." Is 0.01 the power or the significance level?

    Hint: Is H0 true or false in this sentence?

  6. A study with power 0.30 finds no significant effect. Rae writes: "This proves the program has no effect." Correct Rae.

    Hint: How often would this study miss a real effect?

Want the full set?

50 practice questions for this concept — free to try, every one with a complete worked solution showing the why, not just the answer.

Frequently Asked Questions

Why not make the power 0.99 for every study?

Because power costs something. To raise it you can enlarge the sample, which costs money and time, or raise α\alpha, which means more false alarms. For a two-sided test at α=0.05\alpha = 0.05, going from power 0.80 to 0.99 takes about 2.3 times as many subjects. Researchers usually settle on 0.80 as a balance: a real effect of the planned size is found four times out of five without an impossible budget.

Does a smaller α make a test better?

It makes false alarms rarer, but it also lowers the power. A smaller α\alpha moves the rejection cutoff further out, so a real effect has to look more extreme before the test notices it. For a two-sided zz-test with power 0.80 at α=0.05\alpha = 0.05, switching to α=0.01\alpha = 0.01 with the same sample drops the power to about 0.59. The only way to lower both error rates at once is more data.

Can you work out a study's power after it is finished?

You can, but power computed from the effect you happened to observe adds little: it is tied to the p-value, so a non-significant result always comes with low observed power. What helps is the power for an effect size that would matter, chosen in advance. If a study had only a 20% chance of detecting a meaningful effect, its non-significant result says little about whether the effect exists.

What does the size of the true effect have to do with power?

The further the truth is from the null, the easier it is to see through random noise. A coin that lands heads 90% of the time will almost surely be caught in 20 flips, while one that lands heads 52% of the time needs thousands of flips to be detected reliably. So a test has no single power: it depends on which alternative value is true, and a power curve shows it for each one.

Learning Path

Power of a Test

You are here

Next →

Nothing after this yet
Build on Hypothesis Testing (null, alternative, rejecting), Type I and Type II Errors (what α and β mean) and Sampling Distribution (why the standard error shrinks as n grows). Power ties these together into one design question. With it you can plan sample sizes for experiments, read 'not significant' results honestly, and see why a paired design often detects an effect with fewer subjects.

See Also