A coin is flipped 20 times under H0:p=0.5, and you see 15 heads. The two-sided p-value is approximately what (binomial tail, accept ≈0.04)?
Example 3
medium
For a one-sided test with z=−2.33 (lower tail), the p-value is approximately what (use P(Z<−2.33)≈0.01)?
Example 4
hard
A study reports p =0.04. Without more info, what does NOT this tell us about the effect size?
Example 5
hard
Five tests at family-wise α=0.05 via Bonferroni: each test uses α= what?
Example 6
challenge
An exact one-sided permutation p-value uses the proportion of permutations with statistic ≥tobs. If 20 of 1000 permutations meet this, the p-value is ____.
Example 7
medium
A two-sided z-test has observed z=1.5. Upper-tail beyond 1.5 is ≈0.0668. Find the two-sided p-value.
Example 8
medium
Suppose under H0 the test statistic T has CDF F. The observed value is tobs=2 with F(2)=0.9772. What is the one-sided (upper) p-value?One-sided (upper) p-value = 1 − F(2) = 1 − 0.9772 = 0.0228
Example 9
hard
A t-statistic is t=2.5 on 20 df. The upper-tail area beyond 2.5 is about 0.011. Give the two-sided p-value.
Example 10
hard
Compute the two-sided p-value when z=−1.96.
Example 11
challenge
Under H0 a continuous test statistic has p-value distribution Uniform(0,1). Find P(p≤0.01∣H0).
Example 12
hard
A two-tailed z-test gives z=−2.65. The p-value is approximately 0.008. If α=0.05, should we reject H0?
Example 13
hard
A test gives a p-value of 0.12. Interpret this and state the decision at α=0.05.
Practice Problems
Try each one, then open the solution to compare your method.
Example 1
easy
Under what assumption is a p-value computed?
Example 2
easy
A small p-value provides evidence ____ the null hypothesis.
Example 3
easy
What threshold is most commonly used to call a p-value 'small'?
Example 4
easy
True or false: the p-value is the probability that the null hypothesis is true.
Example 5
easy
A p-value of 0.01 means the observed (or more extreme) data would occur how often if H0 were true?
Example 6
easy
If p =0.30 at α=0.05, do we reject H0?
Example 7
easy
A larger test statistic (further from 0) generally produces a ____ p-value.
Example 8
easy
Fill in: the p-value measures how ____ the data are, assuming the null is true.
Example 9
medium
A two-sided z-test gives z=2.5. The upper-tail area beyond 2.5 is about 0.006. What is the two-sided p-value?Two-sided p-value: both tails beyond |z|
Example 10
medium
Two studies report p =0.049 and p =0.051 at α=0.05. How different is the actual evidence?
Example 11
medium
A p-value is 0.20. A student concludes 'there is a 20% chance the null is true.' What is the error?
Example 12
medium
A test of a new teaching method gives p =0.002 but the average score gain is only 0.3 points. What should you report alongside the p-value?
Example 13
medium
If H0 is actually true, what is the distribution of the p-value across many repeated experiments?
Example 14
medium
A one-sided test has z=1.0, with upper-tail area about 0.16. Is this evidence against H0 at α=0.05?One-sided (upper) p-value: tail area beyond z
Example 15
medium
Why does running many tests and reporting only the smallest p-value distort its meaning?
Example 16
medium
A one-sided test of Ha:μ>0 gives z=2.33, with upper-tail area about 0.01. At α=0.05, decide.One-sided p-value: tail area beyond z
Example 17
medium
A two-sided z-test gives z=1.5, with upper-tail area about 0.067. Compute the two-sided p-value.Two-sided p-value: both tails beyond |z|
Example 18
challenge
A two-sided z-test gives z=3. Using that the area beyond ∣z∣=3 is about 0.0027 total, state the p-value and the decision at α=0.05.Two-sided p-value: both tails beyond |z|
Example 19
challenge
Experiment A: p =0.04, n=2,000,000. Experiment B: p =0.04, n=20. Why might A's significant result be less impressive than B's?
Example 20
challenge
A researcher computes p =0.03 and writes 'there is a 97% chance our treatment works.' Identify and correct the two conceptual errors.
Example 21
easy
Under H0, the p-value is the probability of observing data ____ or more extreme than what was seen.
Example 22
easy
At α=0.05, a p-value of 0.02 leads to what decision?
Example 23
easy
At α=0.05, a p-value of 0.20 leads to what decision?
Example 24
medium
For a two-sided test the p-value from z=1.5 is approximately what?
Example 25
medium
A larger sample size, with the same true effect, tends to produce a ____ p-value.
Example 26
medium
A small p-value gives evidence ____ the null hypothesis.
Example 27
medium
A two-sided p-value can be computed from a one-sided p-value by ____.
Example 28
hard
Under H0, the distribution of the p-value is approximately ____.
Example 29
hard
A study tests 40 outcomes at α=0.05 but all H0 are true. Expected number of 'significant' results is what?
Example 30
hard
A 'p-hacked' result is one where the researcher ____.
Example 31
medium
For a chi-square test with χ2=10.5 and 3 df (critical χ0.052=7.815), what is the decision at α=0.05?
Example 32
medium
Two studies of the same effect report p =0.04 and p =0.06. Are their conclusions necessarily different at α=0.05?
Example 33
challenge
A meta-analysis combines k independent p-values via Fisher's method using −2∑ln(pi). Under H0, this statistic follows what distribution?
Example 34
easy
Is a p-value of 0.6 strong evidence against H0?
Example 35
easy
A two-sided z-test gives ∣z∣=1.96. What is the p-value approximately?Two-sided p-value: both tails beyond |z|
Example 36
easy
At α=0.05, the decision when p=0.04 is:
Example 37
easy
A one-sided test gives upper-tail area 0.02 at the observed z. What is the one-sided p-value?One-sided (upper) p-value: upper-tail area is 0.02
Example 38
medium
A p-value of 0.04 is reported. A student says 'the null has 4% probability of being true.' What's wrong?
Example 39
medium
Two studies report p=0.04 and p=0.06, α=0.05. Are these results practically very different?
Example 40
medium
Why do statisticians prefer to report the exact p-value (e.g., 0.012) instead of just 'p < 0.05'?
Example 41
medium
A study has p=0.001 for a tiny effect (Δ=0.1 on a 100-point scale) with n=10000. Is the effect important?
Example 42
medium
A study reports p<0.0001. Why might you still want to see the test statistic value?
Example 43
hard
Researchers run 100 independent tests. Bonferroni correction uses α/m as the per-test threshold. If overall α=0.05, what threshold should each test use?
Example 44
hard
An experiment yields p=0.049. A press release says 'the study proved the treatment works.' Why is this wrong?
Example 45
hard
Two independent studies report p=0.04 for the same hypothesis. Should we combine them by multiplying p-values?
Example 46
medium
True or false: a p-value above α means H0 is true.
Example 47
hard
A scientist runs the same test multiple times, choosing the smallest p-value to report. Why does this inflate Type I error?
Example 48
medium
An A/B test reports p =0.20 after 1000 users. What does this tell us about whether B is better than A?
Example 49
hard
A researcher obtains p=0.03. Would the result be significant at α=0.05? At α=0.01?
Example 50
hard
A hypothesis test gives a p-value of 0.20. What decision would you make at α=0.10 and at α=0.05?
What to focus on
Core idea:Assume the null is true, then ask how often chance alone would give data at least as extreme as yours. That tail probability measures surprise: small means hard to explain by chance. It does not directly tell you how likely the null is or how big the effect is.
Common stuck point:Students often know a procedure related to p-value but skip the recognition step: Do you need the chance, assuming the no-effect claim is true, of results at least as extreme as the ones observed? That leads to a calculation or graph that looks reasonable but answers a different question.
Ask yourself:Do you need the chance, assuming the no-effect claim is true, of results at least as extreme as the ones observed?
Concept recap
The p-value is the probability of observing results at least as extreme as the actual data, calculated under the assumption that the null hypothesis is true. A small p-value (typically below 0.05) suggests the observed data is unlikely under the null, providing evidence against it.
A p-value is the referee's reaction to a suspiciously lucky run of dice: "if these dice are fair, a run this good or better turns up about 1 game in 500." It rates how odd the data are under the innocent story. It does not say how likely the innocent story is; that also depends on how many players actually cheat.