On this concept: Formula

Not sure which lesson to do first? Find your level and get a plan →

Math · Statistics & Probability · Grade 9-12

Two-Sample Tests

Two-sample tests compare a parameter (mean or proportion) between two independent populations: the two-sample t-test for means, the two-proportion z-test for proportions.

t=(xˉ1−xˉ2)−(μ1−μ2)0s12n1+s22n2

Quick Answer

Two-sample tests compare a parameter (mean or proportion) between two independent populations: the two-sample t-test for means, the two-proportion z-test for proportions. Use them when two separate groups (a treatment group and a control group, two classes, two versions of a web page) are measured and you want to know whether the gap between their averages or rates is bigger than chance would produce. The test divides the observed gap by a standard error that combines the variation in both groups.

Why This Matters

Most comparisons in science and business are between two independent groups: patients on a drug versus a placebo, customers who saw ad A versus ad B. A two-sample test turns 'group A scored 4 points higher' into a judgment about whether 4 points means anything, given how much each group varies. The wrong test, such as a t-test on yes/no data or an independent test on paired data, gives the wrong standard error and so the wrong conclusion.

Intuitive Explanation

Two bakeries each sell loaves labelled 500 g. You weigh 30 loaves from each: bakery A averages 503 g, bakery B 498 g. Is A really heavier, or did you happen to pick heavier loaves? It depends on how much loaves vary within each bakery. If they vary by only 1 or 2 g, a 5 g gap is huge; if they vary by 40 g, it is nothing.

The standard error s12/n1+s22/n2 measures exactly this: how far the gap between two sample means would bounce around if the bakeries were truly equal. Both groups contribute, because each mean has its own random wobble and the wobbles add. The test statistic is the observed gap divided by this standard error; with large samples, a gap of about 2 standard errors or more happens less than 5% of the time when there is no real difference.

The groups must be independent: who is in group A tells you nothing about who is in group B. If the 'two groups' are the same people measured twice, the data are paired, and you subtract within each person instead. Check the kind of data too. Averages of measurements (weights, scores) use the two-sample t-test; percentages of yes/no answers (clicked or not, recovered or not) use the two-proportion z-test.

Think of it like this. Comparing two independent groups is like judging two archers by their arrows. Each archer's arrows cluster around a different spot. Whether the archers really aim at different spots depends on how scattered each one's arrows are and how many arrows you have seen. A tight cluster or many arrows makes a small gap convincing.

Core idea. Two-sample tests compare a mean or proportion between two INDEPENDENT groups to see if they truly differ.


When to Use

Use a two-sample test when there are two independent groups of different subjects, such as treatment and control, or version A and version B, and the question asks whether their means or proportions differ. Choose the t-test for averages of measurements and the two-proportion z-test for yes/no rates. If the same subjects are measured twice or matched one-to-one, use the paired t-test; for counts spread over more than two categories, use a chi-square test.

Tip. Ask: Are the two groups made of different, unrelated subjects with no natural pairing between them?

How to Recognize It

Before choosing a two-sample test, look at what was recorded for each subject, how many groups and outcomes there are, and how the groups were formed.

  1. Is the outcome a measurement, such as a time or a score, so that each group has a mean?

    If yes, use the two-sample t-test with standard error s12n1+s22n2. If each subject gives a yes or a no, compare the proportions with the two-proportion z-test instead.

  2. Do the two groups have different sizes, such as 25 students in one class and 31 in another?

    If yes, the data cannot be paired, which confirms a two-sample test. Equal sizes do not prove pairing either; check whether each value has one specific partner, as before-and-after scores do, which would call for the paired t-test.

  3. Are there more than two groups, or more than two outcome categories?

    If yes, a two-sample test is too small for the job. For counts in several categories, use a chi-square test; comparing the means of three or more groups calls for analysis of variance, a later topic.

  4. Were subjects randomly assigned to the two groups, or randomly sampled from two populations?

    If randomly assigned, a significant difference supports a cause-and-effect claim about the treatment. If only sampled, the test shows the populations differ, not why. If neither step was random, the p-value has little meaning.

Two-Sample Tests compared with Paired t-test, Two-proportion z-test, and Chi-square test

All four rows compare groups, so the choice comes down to the shape of the data. Ask whether the groups are linked (paired) or separate, and whether each subject gives a measurement (two-sample t), a yes-or-no answer (two-proportion z), or one of several categories (chi-square).

Two-Sample Tests

Meaning
Use this when you compare a mean or proportion between two independent, unrelated groups.
Key test
Are the two groups made of different, unrelated subjects with no natural pairing between them?
Formula
t=(xˉ1−xˉ2)−(μ1−μ2)0s12n1+s22n2
Example
Method A: xˉ1=82, s1=6, n1=30. Method B: xˉ2=78, s2=5, n2=30. Test if the means differ at α=0.05.

Paired t-test

Meaning
Compares two measurements on the SAME or matched subjects via their differences.
Key test
Use when each subject is measured twice or pairs are matched.
Formula
t=dˉsd/n
Example
Before vs after on the same students

Two-proportion z-test

Meaning
The two-sample test for categorical YES/NO data, using a pooled proportion.
Key test
Use when the outcome is a proportion, not a mean.
Formula
z=p^1−p^2p^(1−p^)(1n1+1n2)
Example
Comparing vote share in two cities

Chi-square test

Meaning
Compares counts across many categories; a 2×2 chi-square equals the two-proportion z-test.
Key test
Use for category counts in a larger table.
Formula
χ2=∑(O−E)2E
Example
A 3×2 frequency table

Formula & Notation

t=(xˉ1−xˉ2)−(μ1−μ2)0s12n1+s22n2
t=(xˉ1−xˉ2)−(μ1−μ2)0s12n1+s22n2 with df from Welch's approximation

How to read it: For proportions: z=(p^1−p^2)−0p^(1−p^)(1n1+1n2) where p^ is the pooled proportion.

Worked Examples

Example 1 — Comparing two methods

Easy

Method A: xˉ1=82, s1=6, n1=30. Method B: xˉ2=78, s2=5, n2=30. Test if the means differ at α=0.05.
Show the steps and the answer
  1. Set up H0:μA=μB against Ha:μA≠μB; the groups are different students, so use a two-sample t-test.

    'Test if the means differ' gives no direction, so the alternative is two-sided.

  2. Standard error: 6230+5230=1.2+0.833=2.033≈1.426.

    Each group's mean wobbles on its own, so the variances of the two means add.

  3. Test statistic: t=82−781.426≈2.81.

    The 4-point gap is about 2.8 standard errors wide.

  4. Compare: with the conservative df=min⁡(30,30)−1=29, the two-sided critical value is 2.045, and 2.81>2.045, so p<0.05.

    Using the smaller sample size minus 1 is a safe choice of df when no software is available.

Answer: Reject H0 — the two methods' means differ

Takeaway: For independent groups, the standard error combines both groups' variances: s12/n1+s22/n2.

Example 2 — Same students twice

Standard

Instead one group of students is tested before AND after a method, and you compare their two scores. Two-sample?
Show the steps and the answer
  1. Check for a link: every 'after' score belongs to one particular student's 'before' score.

    The two columns hold the same people, so they are not independent samples.

  2. See what the two-sample formula would do: count the large differences between students as noise in both columns.

    That inflates the standard error and can hide a real improvement.

  3. Subtract each student's before score from their after score and run a one-sample t-test on those differences with df=n−1.

    This is the paired t-test; it uses only the variation in the changes.

Answer: No — use a paired t-test

Takeaway: The same subjects measured twice call for the paired test; separate, unlinked groups call for the two-sample test.

Example 3 — Find the mistake: a t-test on yes/no data

Application

In a survey, 120 of 200 teens and 90 of 200 adults say they prefer streaming to TV. Raj writes: "These are two groups, so I'll run a two-sample t-test on the means 120 and 90." Find the mistake and correct it.
Show the steps and the answer
  1. Identify the data type: each person answers yes or no, so the summaries are proportions, p^1=120200=0.60 and p^2=90200=0.45.

    120 and 90 are counts, not averages of measurements, and there is no s1 or s2 to plug into a t-test.

  2. Pool under H0:p1=p2: p^=120+90400=0.525.

    If the null is true, both groups share one proportion, best estimated from all 400 people.

  3. Standard error: 0.525(0.475)(1200+1200)=0.00249≈0.0499.

    For yes/no data the spread comes from p^(1−p^), not from a sample standard deviation.

  4. Test statistic: z=0.60−0.450.0499≈3.00, beyond 1.96.

    A 15-point gap is 3 standard errors wide, so p≈0.003.

Answer: Use a two-proportion z-test: z≈3.00, p≈0.003, so teens and adults differ in their preference.

Takeaway: Averages of measurements call for a two-sample t-test; yes/no counts call for the two-proportion z-test with a pooled p^.

Where You'll See It

  • A/B testing a sign-up page

    A website shows page A to 5,000 visitors and page B to another 5,000. If 4.0% sign up on A and 4.8% on B, the two-proportion z-test gives z≈1.95, just short of the 1.96 cutoff, so the company should collect more data before switching pages.

  • Drug versus placebo trial

    Patients are randomly split: 40 get the drug, 40 a placebo. Blood pressure falls by an average of 12 mmHg (s=8) on the drug and 6 mmHg (s=9) on the placebo. Then SE=64/40+81/40≈1.90 and t=6/1.90≈3.15, strong evidence that the drug works.

Common Mistakes

  1. The slip: Running a two-sample test on paired data

    Instead: if subjects are matched, use the paired t-test on differences instead.

  2. The slip: Using a single pooled standard deviation by default

    Instead: the two-sample t-test typically uses s12/n1+s22/n2 with separate variances.

  3. The slip: Choosing a t-test for proportion data

    Instead: compare proportions with the two-proportion z-test, not the t-test for means.


Mini Practice

Try these on your own. Tap Reveal when you want to check.

  1. Group 1: xˉ1=75, s1=10, n1=25. Group 2: xˉ2=69, s2=10, n2=20. Find the standard error of the difference and the t statistic.

    Hint: Use s12n1+s22n2.

  2. A two-sample t-test gives t=2.00 with df=19. The two-sided critical value at α=0.05 is 2.093. What do you conclude?

    Hint: Compare the size of t with the critical value.

  3. In one class 60 of 100 students passed; in another, 45 of 100 passed. Find the pooled proportion and the two-proportion z statistic.

    Hint: Pool both classes: total passes over total students. Then SE=p^(1−p^)(1100+1100).

  4. A survey asks 120 boys and 130 girls a yes-or-no question about a later school start. You compare the proportions saying yes. Two-sample t-test or two-proportion z-test?

    Hint: Is the response a measurement or a yes/no answer?

  5. Omar compares 15 students' quiz scores before and after a review session by running a two-sample t-test with n1=n2=15. Correct his choice of test.

    Hint: Are the before and after scores from different people?

Want the full set?

50 practice questions for this concept — free to try, every one with a complete worked solution showing the why, not just the answer.

Frequently Asked Questions

What is the difference between a pooled and an unpooled standard error?

Pooling means combining both groups into one estimate. The test of H0:p1=p2H_0: p_1 = p_2 pools, because if the null is true both groups share one rate, so the combined proportion estimates it best. The usual two-sample tt-test does not pool: it keeps s1s_1 and s2s_2 separate, since two groups can easily have different spreads even when their means are equal. Pooling means is safe only when the spreads are nearly equal.

Why does the standard error add the two variances instead of subtracting them?

Both sample means wobble by chance, and the gap between them inherits both wobbles. If one mean happens to land high and the other low, the gap grows; the errors do not cancel just because you subtract the means. For independent groups, variances add: Var(xˉ1−xˉ2)=σ12n1+σ22n2\text{Var}(\bar{x}_1 - \bar{x}_2) = \frac{\sigma_1^2}{n_1} + \frac{\sigma_2^2}{n_2}. The paired test avoids part of this by removing person-to-person differences.

How many degrees of freedom does a two-sample t-test use?

Calculators and software use Welch's approximation, which gives a value, often not a whole number, between the smaller of n1−1n_1 - 1 and n2−1n_2 - 1 and the total n1+n2−2n_1 + n_2 - 2. When working by hand, a safe shortcut is the smaller of n1−1n_1 - 1 and n2−1n_2 - 1. It gives a slightly wider interval and a larger p-value, so it never overstates the evidence.

If two confidence intervals overlap, is the difference between the groups not significant?

Not necessarily. Two 95% intervals can overlap somewhat while the difference is still significant at the 5% level, because the standard error of the difference is smaller than the sum of the two margins. With equal standard errors, the intervals can overlap by up to about 29% of either interval's width and the difference still be significant. Test the difference directly with the two-sample procedure.

Learning Path

Two-Sample Tests

You are here

Next →

Nothing after this yet
You need Sampling Distribution and the Central Limit Theorem (why a sample mean is roughly normal), plus Confidence Interval and Hypothesis Testing (how a standard error becomes a decision). Two-sample tests apply those tools to the gap between two groups. They lead directly to analysing randomized experiments and A/B tests, and to chi-square tests and ANOVA when there are more than two groups or categories.

See Also