Not sure which lesson to do first? Find your level and get a plan →

Statistics · Grade 9-12

P-Value

The p-value is the probability of observing results at least as extreme as the actual data, calculated assuming the null hypothesis is true.


Quick Answer

The p-value is the probability of observing results at least as extreme as the actual data, calculated assuming the null hypothesis is true. Use it inside a hypothesis test to measure how surprising your result would be if nothing were going on. A p-value of 0.002 means chance alone would produce something this extreme about 2 times in 1,000; a p-value of 0.40 means it would happen often. The smaller the p-value, the stronger the evidence against the null, but it is not the probability that the null is true.

Why This Matters

Every experiment shows some difference between groups, so researchers need a common scale for "is this more than luck?". The p-value is that scale: anyone can check that p=0.003 is strong evidence and p=0.30 is none. Misread it as the chance the null is true, or as a measure of how big the effect is, and a weak study turns into a confident headline.

Intuitive Explanation

Your friend says she can tell two brands of cola apart by taste. You give her 10 blind cups and she names 9 correctly. If she were only guessing, each cup would be a coin flip, and the chance of 9 or 10 correct is 10+11024=111024≈0.011, about 1 time in 93. That number is the p-value. Results this good are rare for a guesser, which is evidence she really can taste the difference.

Why "at least as extreme" and not "exactly 9"? When there are many possible outcomes, every exact one is unlikely: even exactly 50 heads in 100 fair flips has probability only about 0.08. So the p-value adds up every outcome as far as or farther than yours from what the null predicts. If the alternative allows either direction, as in "the coin is unfair", both tails count; if it names one direction, as in "she does better than guessing", only that tail does.

The p-value is P(data this extreme∣H0), which is not the same as P(H0∣data). Reversing a condition changes the probability: nearly every professional basketball player is tall, but very few tall people are professional basketball players. A p-value of 0.011 does not mean a 1.1% chance she is guessing; that would also depend on how common real taste-testers are. Nor does a tiny p-value mean a big effect: with huge samples, trivial differences produce tiny p-values.

Think of it like this. A p-value is the referee's reaction to a suspiciously lucky run of dice: "if these dice are fair, a run this good or better turns up about 1 game in 500." It rates how odd the data are under the innocent story. It does not say how likely the innocent story is; that also depends on how many players actually cheat.

Core idea. Assume the null is true, then ask how often chance alone would give data at least as extreme as yours. That tail probability measures surprise: small means hard to explain by chance. It does not directly tell you how likely the null is or how big the effect is.


When to Use

Use a p-value when a test has a null hypothesis and an observed test statistic, and the question is how unusual that result would be by chance: how likely is a result this extreme if the coin is fair? Find the tail probability beyond the observed value, on one side or both depending on the alternative. To make the reject-or-not call, compare it with α (statistical significance); to say how big the effect is, use a confidence interval instead.

Tip. Ask: Do you need the chance, assuming the no-effect claim is true, of results at least as extreme as the ones observed?

How to Recognize It

Before computing or reading a p-value, check which claim it assumes and which results count as at least as extreme.

  1. Is there a null hypothesis to assume true, such as a fair die or no difference between groups?

    Yes means a p-value can be computed. With no claim to assume and no data to judge, the chance of an event is Basic Probability.

  2. Do results in both directions count against the null, such as too many or too few heads?

    Yes means a two-sided p-value that adds both tails. If only one direction matters and that was decided beforehand, use one tail.

  3. Is the answer a number between 0 and 1 rather than a verdict?

    Yes is the p-value. Comparing that number with a cutoff such as 0.05 to declare a result is Statistical Significance.

P-Value vs Statistical Significance vs Hypothesis Testing vs Conditional Probability

The p-value is a conditional probability that people often read backward, and it sits inside a test that ends in a verdict. Ask whether you need the surprise number, the verdict, the whole procedure, or which way a conditional probability points.

P-Value

Meaning
The p-value is the probability of results at least as extreme as those observed, computed assuming the null hypothesis is true.
Key test
Probability of the data given the null, never of the null.
Formula
P(X≥xobs∣H0)
Example
18 or more sixes in 60 rolls of a fair die: p≈0.007.

Statistical Significance

Meaning
Statistical significance is the verdict that a p-value fell below a threshold α set before the study.
Key test
A yes-or-no call made from the p-value.
Formula
p<α
Example
Since 0.007<0.05, the excess of sixes is significant at the 5% level.

Hypothesis Testing

Meaning
Hypothesis testing is the full procedure: state the hypotheses, compute a statistic and its p-value, then decide.
Key test
The whole process that the p-value sits inside.
Example
Null: the die is fair; alternative: sixes come up more than 16 of the time.

Conditional Probability

Meaning
Conditional probability is the chance of one event given that another has happened, and swapping the two events changes the answer.
Key test
Explains why P(data∣H0) is not P(H0∣data).
Formula
P(A∣B)≠P(B∣A)
Example
Most people with measles have spots, but most people with spots do not have measles.

Formula & Notation

How to read it: The p-value is denoted p. The significance level threshold is α. We reject H0 when p<α.

Worked Examples

Example 1 — P-value by counting: a taste test

Easy

Ines claims she can tell tap water from bottled water. In 8 blind trials she is right 7 times. Find the p-value for H0: she is guessing (p=0.5), against Ha: she does better than guessing (p>0.5).
Show the steps and the answer
  1. List the outcomes at least as extreme as 7 correct: 7 or 8 correct.

    Ha points toward doing better, so only the high tail counts.

  2. Count them: (87)=8 ways to get exactly 7 right and (88)=1 way to get all 8.

    Under H0 each of the 28=256 right/wrong patterns is equally likely.

  3. Add: p=8+1256=9256≈0.035.

    The p-value is the total probability of the observed result and everything more extreme.

  4. Interpret: a pure guesser would do this well about 3.5% of the time.

    That is fairly rare, so the result is moderate evidence that Ines can taste a difference.

Answer: p=9256≈0.035

Takeaway: To find a p-value, assume H0, list every outcome at least as extreme as yours in the direction of Ha, and add their probabilities.

Example 2 — Exactly 8 versus 8 or more

Standard

A basketball player who has always made 50% of her free throws says she has improved. In her next 10 shots she makes 8. (a) If she is still a 50% shooter, what is the probability she makes exactly 8? (b) What is the p-value for H0:p=0.5 against Ha:p>0.5? At α=0.05, which number decides the test?
Show the steps and the answer
  1. Compute (a): P(X=8)=(108)/210=45/1024≈0.044.

    This is the basic probability of one outcome, with no tail involved.

  2. Compute (b): P(X≥8)=45+10+11024=561024≈0.055.

    The p-value must also count 9 and 10 makes, which would be even stronger evidence of improvement.

  3. Compare each with 0.05: 0.044 is below it, 0.055 is above it.

    Using the single-outcome probability would wrongly make the result look significant.

  4. Decide using (b): 0.055>0.05, so do not reject H0.

    The test is about how surprising results this good or better are, and that is what the p-value measures.

Answer: (a) About 0.044. (b) p≈0.055. The p-value decides, so at α=0.05 the 8 makes are not quite enough evidence of improvement.

Takeaway: A p-value is a tail probability, not the probability of the exact result; with many possible outcomes, any single one is unlikely.

Example 3 — Find the mistake: reversing the condition

Application

A greenhouse study compares plants grown with and without a new fertilizer and gets p=0.03 for H0: the fertilizer has no effect. Ana writes: "There is only a 3% chance that the fertilizer is useless, so we are 97% sure it works." Find the mistake and correct it.
Show the steps and the answer
  1. Write what 0.03 measures: P(a difference at least this large∣no effect)=0.03.

    The p-value is computed by assuming the null is true, so it cannot also be the probability that the null is true.

  2. Name the error: Ana turned P(data∣H0) into P(H0∣data).

    Those two probabilities can be far apart, as a simple count shows.

  3. Count: suppose 900 useless and 100 working fertilizers are tested, and the test detects 80% of working ones. About 900×0.05=45 useless and 100×0.8=80 working ones give p<0.05.

    Among those 125 "significant" results, 45/125=36% come from useless fertilizers, nowhere near 5%.

Answer: If the fertilizer had no effect, a difference at least this large would happen in about 3% of experiments. That is evidence against "no effect"; it is not a 3% chance that the fertilizer is useless.

Takeaway: A p-value assumes the null and grades the data; it never gives the probability that the null is true.

Where You'll See It

  • Surprising coin-flip streak

    If a supposedly fair coin lands heads 9 of 10 times, the p-value answers 'how often would a fair coin do something this extreme?' — a tiny p-value makes you doubt the coin is fair.

  • Is the new teaching method better?

    After comparing two classes, a p-value of 0.02 means 'if the methods were truly equal, results this different would happen only about 2% of the time,' which is surprising enough to suspect a real difference.

  • Spotting a loaded die

    Rolling a die 60 times and getting twenty 6's, the p-value tells you how rare that many sixes would be for a fair die; a very small value is evidence the die is loaded.

Common Mistakes

  1. The slip: The p-value is the probability that the null hypothesis is true.

    Why it's tempting: A small p-value makes the null seem improbable, so it feels like P(null is true).

    Instead: The p-value is P(data this extreme | null is true), not P(null | data); it assumes the null and measures the data's surprise, not the null's probability.

  2. The slip: p = 0.049 is a real effect while p = 0.051 is nothing at all.

    Why it's tempting: The 0.05 cutoff makes the two sides feel like opposite categories.

    Instead: The p-value is continuous; 0.049 and 0.051 are essentially the same strength of evidence, and the 0.05 line is an arbitrary convention, not a wall between 'real' and 'fake.'

  3. The slip: A small p-value means the effect is large or important.

    Why it's tempting: 'Significant' sounds like 'big,' and small p-values feel impressive.

    Instead: A small p-value only signals the effect is unlikely to be pure chance; effect size and practical importance must be judged separately.


Mini Practice

Try these on your own. Tap Reveal when you want to check.

  1. You suspect a coin favors heads. It lands heads in all 5 of 5 flips. Assuming the coin is fair, what is the p-value?

    Hint: Nothing is more extreme than 5 heads out of 5. Find that probability for a fair coin.

  2. A one-sided test with alternative "greater than" gives z=2.0. What is the p-value, and is it significant at α=0.05?

    Hint: Find the area under the standard normal curve to the right of 2.0.

  3. You suspect a die favors sixes. In 8 rolls you get 5 sixes. Find the p-value: the chance of 5 or more sixes in 8 rolls of a fair die.

    Hint: Add the binomial probabilities for 5, 6, 7 and 8 sixes with p=16.

  4. A study of 100,000 students finds a new app raises scores by 0.2 points out of 100, with p=0.001. Does the tiny p-value mean the app has a big effect?

    Hint: Does a p-value measure how surprising the data are, or how large the effect is?

  5. Which is a p-value: (a) the chance of drawing red from a bag with 3 red and 2 blue marbles, or (b) the chance that a spinner claimed to land red half the time gives 14 or more reds in 20 spins, after you saw 14?

    Hint: Which probability assumes a claim and asks how surprising the observed data are?

  6. A student says: "The p-value is 0.03, so there is a 3% chance that the null hypothesis is true." Correct the statement.

    Hint: Is the p-value calculated assuming the null is true, or is it about whether the null is true?

Want the full set?

50 practice questions for this concept — free to try, every one with a complete worked solution showing the why, not just the answer.

Frequently Asked Questions

If I repeat a study exactly, will I get the same p-value?

Usually not, and the difference can be large. Because the p-value is computed from random data, it varies from sample to sample. When a real effect is modest, one study might give p=0.01p = 0.01 and an identical repeat p=0.30p = 0.30. This is why a single p-value near the cutoff is weak ground, and why scientists value repeated studies and effect sizes reported with confidence intervals.

If the null is true, how often will a p-value come out below 0.05?

About 5% of the time. When the null is true and the test's assumptions hold, the p-value is equally likely to fall anywhere between 0 and 1, so it lands below 0.05 in about 1 study in 20 and below 0.01 in about 1 in 100. Run 20 tests of treatments that do nothing, and you should expect about one false discovery by chance alone.

Can a p-value be exactly 0?

Not really. Software may print 0.000 or show p<0.001p < 0.001 when the true value is tiny, such as 0.0000004, but as long as the observed result could happen under the null at all, the probability is above zero. Report it as p<0.001p < 0.001 rather than p=0p = 0. Extremely small p-values often come from very large samples, where even small effects stand out clearly from chance.

Learning Path

You need Hypothesis Testing (the null and alternative), Basic Probability (counting outcomes or finding areas) and Sampling Distribution (what the statistic does when the null is true). The p-value condenses all of that into one number: how surprising your data are under the null. Next, Statistical Significance turns that number into a decision by comparing it with a preset α.

See Also