Hypothesis testing is a procedure that decides whether sample data gives enough evidence to reject a default claim (the null hypothesis H0) about a population.
z=σ/nxˉ−μ0
Quick Answer
Hypothesis testing is a procedure that decides whether sample data gives enough evidence to reject a default claim (the null hypothesis H0) about a population. Use it when someone states a specific value, such as "bottles hold 500 mL" or "this coin is fair," and you have sample data to check it against. You assume H0 is true, measure how far the sample falls from it in standard errors, and reject H0 only if a result that extreme would be rare, judged against a significance level α fixed in advance.
Why This Matters
Any sample differs a little from any claim, so "the average was 498 mL, not 500" proves nothing by itself. A test separates gaps that random sampling often produces from gaps it rarely produces. That is how drug trials, factory checks and website experiments decide whether to act; without it, people either chase every random wobble or dismiss real effects as noise.
Intuitive Explanation
A machine is supposed to fill bottles with 500 mL. You measure 36 bottles and get an average of 498 mL. Is the machine off, or did you just draw a slightly low sample? A hypothesis test plays devil's advocate: assume the machine is fine (H0:μ=500), then ask how unusual a sample mean of 498 would be under that assumption.
The sampling distribution supplies the yardstick. If single fills vary with σ=6 mL, means of 36 bottles vary with standard error 366=1 mL, so 498 sits z=1498−500=−2 standard errors from the claim. Under H0, a sample mean at least that far out in either direction turns up only about 4.6% of the time. With α=0.05 chosen beforehand, 4.6% counts as rare enough, and you reject H0.
The logic resembles a trial in which H0 plays "innocent" and the data must be strong evidence to convict. Failing to reject is not proving innocence. A sample mean of 499.5 would give z=−0.5, an entirely ordinary result, so you would keep H0 while still not knowing that the true mean is exactly 500.
A confidence interval uses the same numbers for a different job. "What is the true fill amount?" asks for an estimate, 498±1.96 mL. "Is it 500?" asks for a verdict on one stated value. Test a value someone has claimed; estimate a value nobody has named.
Think of it like this.A hypothesis test works like a smoke detector. It assumes "no fire" until the smoke is too thick to blame on toast. The threshold is fixed before anything burns: a more sensitive setting (larger α) means more false alarms over toast, a less sensitive one means it may sleep through a real fire.
Core idea.Hypothesis testing checks whether sample data is surprising enough to reject a default claim about a population.
When to Use
Use a hypothesis test when a problem states a claim about a population value (a mean, a proportion, "no difference") and asks whether the sample evidence is strong enough to reject it, usually at a significance level such as α=0.05. Write H0 as the claimed value and Ha as what you suspect. If the question asks for a plausible range of values instead, build a confidence interval. If the data are counts in categories, use a chi-square test.
Tip.Ask: Am I deciding whether sample data is surprising enough to reject a specific stated claim about a population?
How to Recognize It
Find the claimed value the problem gives and what the sample actually showed, then check whether the question wants a decision, a range, a probability or a judgment about possible mistakes.
1.
Does the problem state a specific population value to check, such as μ=500 mL or p=0.5?
Yes means that value becomes H0 and you test it. With no stated value, the question is usually how big the parameter is, which a confidence interval answers.
2.
Is the suspected direction of the difference given, such as bottles holding less than 500 mL?
Yes makes Ha one-sided, μ<500, and only that tail counts. If a difference either way matters, write Ha:μ=500 and count both tails, which doubles the tail area for the same test statistic.
3.
Does the question ask only for the probability of a result this extreme if H0 holds?
Yes is the p-value, one step inside the test. The full test also states H0 and Ha, fixes α in advance, and ends with reject or fail to reject, written in context.
4.
Is the question about what would happen if the decision turned out to be wrong?
Yes is the territory of Type I and Type II errors: rejecting a true H0, or keeping a false one, and what each mistake would cost in the situation described.
Hypothesis Testing compared with Confidence interval, P-value, and Type I / Type II errors
All four rows belong to one decision process. Ask which part is wanted: the procedure from H0 to verdict (hypothesis test), the evidence number inside it (p-value), the ways the verdict could be wrong (Type I and II errors), or an estimate with no claim (confidence interval).
Hypothesis Testing compared with Confidence interval, P-value, and Type I / Type II errors
Type
Meaning
Key test
Formula
Example
Hypothesis Testing
Use this when you have a specific claim about a population to test against sample evidence, not a parameter to estimate.
Am I deciding whether sample data is surprising enough to reject a specific stated claim about a population?
z=σ/nxˉ−μ0
A machine should fill bottles to μ0=500 mL. A sample of n=36 gives xˉ=498 mL with σ=6. Test H0:μ=500 at α=0.05.
Confidence interval
Estimates a RANGE for the parameter rather than judging one claimed value.
Use when you want to estimate the parameter, not test a specific value.
xˉ±z∗ns
Estimate the true mean height
P-value
The probability of data this extreme IF H0 is true; the evidence measure inside the test.
Use when quantifying the strength of evidence, not stating the whole decision procedure.
P(data∣H0)
p = 0.03
Type I / Type II errors
The two ways a test's decision can be wrong.
Use when discussing the risk of a wrong conclusion, not running the test itself.
—
Rejecting a true H0
Hypothesis Testing
Meaning
Use this when you have a specific claim about a population to test against sample evidence, not a parameter to estimate.
Key test
Am I deciding whether sample data is surprising enough to reject a specific stated claim about a population?
Formula
z=σ/nxˉ−μ0
Example
A machine should fill bottles to μ0=500 mL. A sample of n=36 gives xˉ=498 mL with σ=6. Test H0:μ=500 at α=0.05.
Confidence interval
Meaning
Estimates a RANGE for the parameter rather than judging one claimed value.
Key test
Use when you want to estimate the parameter, not test a specific value.
Formula
xˉ±z∗ns
Example
Estimate the true mean height
P-value
Meaning
The probability of data this extreme IF H0 is true; the evidence measure inside the test.
Key test
Use when quantifying the strength of evidence, not stating the whole decision procedure.
Formula
P(data∣H0)
Example
p = 0.03
Type I / Type II errors
Meaning
The two ways a test's decision can be wrong.
Key test
Use when discussing the risk of a wrong conclusion, not running the test itself.
Example
Rejecting a true H0
Formula & Notation
z=σ/nxˉ−μ0
z=σ/nxˉ−μ0; reject H0 if ∣z∣>zα/2 (two-tailed) or equivalently if p-value <α
How to read it:H0: null hypothesis (the default claim). Ha: alternative hypothesis (what we suspect). α: significance level (typically 0.05).
Worked Examples
Example 1 — Test a claimed mean
Easy
A machine should fill bottles to μ0=500 mL. A sample of n=36 gives xˉ=498 mL with σ=6. Test H0:μ=500 at α=0.05.
Show the steps and the answer
State the hypotheses: H0:μ=500 and Ha:μ=500.
The claim is the null; a fill that is off in either direction matters, so the alternative is two-sided.
Find the standard error: nσ=366=1 mL.
The test is about the mean of 36 bottles, so the yardstick is the spread of sample means, not of single bottles.
Standardize the sample mean: z=1498−500=−2.
The sample mean sits 2 standard errors below the claimed 500 mL.
Compare with the critical values ±1.96: −2<−1.96, so reject H0.
At α=0.05, only 5% of samples from a correctly set machine land beyond ±1.96; the two-sided p-value here is about 0.046.
Answer:Reject H0 — the machine is likely off
Takeaway:Reject H0 when the sample sits farther from the claim than sampling luck usually allows; here 2 standard errors just beats the 1.96 cutoff.
Example 2 — Estimating, not testing
Standard
Using the same sample, estimate a 95% range for the true fill amount.
Show the steps and the answer
Notice that no claimed value is being judged; the question asks what the true mean is.
That calls for a confidence interval, not a test statistic measured from μ0.
Center on the sample mean and reuse the standard error: 498±1.96×1.
The interval and the test share one sampling distribution, but the interval is centered on the data rather than on the claim.
Compute the endpoints: 498−1.96=496.04 and 498+1.96=499.96 mL.
Every value in this range is consistent with the data at 95% confidence.
Notice that 500 lies just outside the interval.
That matches the test's narrow rejection: a value outside the 95% interval is rejected by a two-sided test at α=0.05.
Answer:498±1.96(1)=(496.04,499.96)
Takeaway:An interval answers 'what is it?' and a test answers 'is it this value?'; both rest on the same standard error.
Example 3 — Find the mistake: failing to reject is not proof
Application
A cereal company claims its boxes average 400 g. A sample of 25 boxes has xˉ=398 g, and σ=10 g. Ravi computes z=10/25398−400=−1 and writes: "∣z∣<1.96, so this proves the boxes average exactly 400 g." Find the mistake and correct it.
Show the steps and the answer
Check the arithmetic: standard error 2510=2 g, so z=2−2=−1.
Ravi's statistic is right; the error is in what he concludes.
Find how ordinary the result is: under H0, P(∣Z∣≥1)≈0.32.
About a third of samples from a true 400 g process land this far out, so the data are not surprising.
Show that other means fit just as well: if the true mean were 397 g, then 398 would give z=2398−397=0.5, also ordinary.
If 397 g is as consistent with the data as 400 g, the test cannot have proved 400 g.
Restate the decision: fail to reject H0.
The sample gives no strong evidence against 400 g, which is weaker than evidence for it.
Answer:Fail to reject H0 (p≈0.32): the data are consistent with a 400 g average but do not prove it.
Takeaway:Failing to reject means 'not enough evidence against the claim', never 'the claim is proved'.
Where You'll See It
Testing a new checkout button
An online store shows 20,000 visitors the old checkout button and 20,000 a new one. The old converts 5.0% and the new 5.4%. A two-sided test of H0: no difference gives z≈1.8 and p≈0.07, so at α=0.05 the store can't yet conclude the new button is better and keeps collecting data.
Approving a new medicine
A drug trial sets H0: the drug works no better than a placebo. Regulators require results that would be very unlikely under that null, with α fixed before any data arrive, so the threshold can't be moved afterwards to fit the results.
Common Mistakes
The slip:Treating 'fail to reject H0' as 'prove H0 true'
Instead:it only means insufficient evidence against it.
The slip:Choosing α after seeing the data
Instead:set the significance level before testing to avoid cherry-picking.
The slip:Confusing the null and alternative
Instead:H0 is the default 'no effect' claim; Ha is what you suspect.
Mini Practice
Try these on your own. Tap Reveal when you want to check.
1.
A juice company claims its bottles hold 350 mL on average. You suspect they hold less. Write the null and alternative hypotheses.
Hint:The null is the claim being challenged; the alternative is what you suspect.
2.
A company says its batteries last μ0=40 hours with σ=4 hours. A sample of n=64 gives xˉ=39 hours. Find z and test H0:μ=40 against Ha:μ<40 at α=0.05.
Hint:First find the standard error σ/n. The one-sided critical value is −1.645.
3.
Cereal boxes should hold 500 g, with σ=10 g. A sample of 25 boxes has xˉ=497 g. Test H0:μ=500 against Ha:μ=500 at α=0.05.
Hint:Compute σ/25 first. For a two-sided test, compare ∣z∣ with 1.96.
4.
Same cereal claim (500 g, σ=10 g), but now 100 boxes are sampled and xˉ=497 g again. Test two-sided at α=0.05.
Hint:A bigger sample shrinks the standard error. Recompute it before finding z.
5.
A coach wants a range of plausible values for her team's average 100 m time. Nobody has claimed a specific value. Hypothesis test or confidence interval?
Hint:Is there a claimed value to challenge, or a value to estimate?
6.
A test gives z=−1.2 at α=0.05. Mia writes: "We fail to reject H0, so this proves the mean really is 500 g." Correct her conclusion.
Hint:What does failing to reject actually tell you about the evidence?
Want the full set?
50 practice questions for this concept — free to try, every one with a complete worked
solution showing the why, not just the answer.
Is a statistically significant result always an important one?
No. Significant only means the data would be unlikely if H0 were true. With a huge sample, even a tiny effect becomes significant: a study of 100,000 students could show that a study app raises scores by 0.2 points out of 100, with a p-value below 0.05. That difference is real but too small to matter. Report how big the effect is, not just whether it passed the test.
Why is α usually 0.05?
Mostly tradition. The statistician Ronald Fisher suggested 1 in 20 in the 1920s as a convenient line for when a result deserves a second look, and it stuck. Fields choose differently when a false alarm would be costly: particle physicists required odds of about 1 in 3.5 million before announcing the Higgs boson in 2012. The right α depends on how bad a false alarm would be.
What happens if you run 20 tests at α=0.05 when nothing is really going on?
You should expect about one false alarm, since 20×0.05=1, and the chance of at least one is 1−0.9520≈0.64. This is how a study testing whether 20 different foods cause headaches can find one by luck alone. Researchers guard against it by deciding in advance which test matters, or by using a stricter cutoff for each test, such as 200.05=0.0025.
You need the Sampling Distribution (how much a statistic varies), the Normal Distribution and Probability first. Hypothesis testing adds a decision rule: assume H0, measure the sample's distance from it in standard errors, and reject only if that distance would be rare. Next, the P-Value measures that rarity precisely, and Type I and Type II Errors describe the two ways the decision can go wrong.