Experimental Design Examples in Statistics

Start with the recap, study the fully worked examples, then use the practice problems to check your understanding of Experimental Design.

This page combines explanation, solved examples, and follow-up practice so you can move from recognition to confident problem-solving in Statistics.

Concept Recap

Experimental design is the careful planning of experiments to establish cause-and-effect relationships by controlling variables, using comparison groups, and randomly assigning subjects to treatment and control conditions to isolate the effect of interest.

Want to know if a new fertilizer helps plants grow? You can't just use it on some plants and see if they grow - maybe they would've grown anyway! You need identical plants, give fertilizer to some (treatment) but not others (control), and keep everything else the same.

Read the full concept explanation →

How to Use These Examples

  • Read the first worked example with the solution open so the structure is clear.
  • Try the practice problems before revealing each solution.
  • Use the related concepts and background knowledge badges if you feel stuck.

What to Focus On

Core idea: Experimental Design checks whether the study design supports a fair comparison before interpreting the outcome.

Common stuck point: Students often know a procedure related to experimental design but skip the recognition step: Did the study use a design feature that makes the groups comparable before the outcome is measured? That leads to a calculation or graph that looks reasonable but answers a different question.

Sense of Study hint: Ask: Did the study use a design feature that makes the groups comparable before the outcome is measured?

Common Mistakes to Watch For

Before you work through the examples, skim the mistake guide so you know which shortcuts and sign errors to avoid.

Correlation vs Causation Mistakes

Worked Examples

Example 1

medium
A teacher tests two study apps. She splits students into low-GPA and high-GPA groups, then randomly assigns each app within each group. Identify the design.

Answer

Randomized block design\text{Randomized block design}

First step

1
Two blocks (low- and high-GPA), randomization within each block.

See the full worked solution + why-it-works coaching

SetupKey insightWhy it worksCommon pitfallConnection

Unlock answer keys One Family plan — every worked solution, all subjects

Example 2

medium
Identify control, treatment, response, and explanatory: 50 mice get drug X, 50 get saline, and we measure tumor size 30 days later.

Example 3

hard
Compare the validity of: (A) randomized experiment with n=50n=50 vs (B) observational study with n=5000n=5000. Which better supports causal claims?

Example 4

challenge
A randomized trial finds treatment reduces symptoms by 20% (p=0.03p=0.03). Skeptics argue the result is due to chance, bias, or confounding. Address each.

Example 5

medium
A researcher tests three doses (low, medium, high) of a fertilizer on 6060 identical plants. List one valid randomization scheme.

Example 6

hard
Design a randomized controlled experiment to test whether a new app improves quiz scores. Specify the population, treatment, control, randomization unit, and a key control measure.

Example 7

challenge
A company runs an experiment to test a website change but only randomizes by day (treatment on odd days, control on even days). Why is this design weaker than randomizing visitors, and identify one confounder it cannot control.

Example 8

easy
A farmer wants to test whether a new fertiliser improves crop yield. She applies the new fertiliser to Field A and uses the old fertiliser on Field B. She finds Field A produces more. Can she conclude the new fertiliser is better? Identify the flaw in the experiment.

Example 9

medium
Describe the key components of a well-designed experiment to test whether a new study method improves exam scores. Include: treatment and control groups, random assignment, and what should be kept constant.

Practice Problems

Try these problems on your own first, then open the solution to compare your method.

Example 1

easy
A gardener gives fertilizer to half her identical plants (treatment) and nothing to the other half (control), then compares growth. Is this an experiment?

Example 2

easy
In a fertilizer experiment, which group receives no fertilizer to serve as a baseline?

Example 3

easy
Why should subjects be randomly assigned to treatment and control groups in an experiment?

Example 4

easy
A scientist changes both temperature and lighting at once and growth improves. Why can't she identify the cause?

Example 5

easy
To fairly test a new teaching method, what should be true of the two classes being compared before the method is applied?

Example 6

easy
A study gives a new drug to volunteers and reports they improved, with no comparison group. What key design feature is missing?

Example 7

easy
In an experiment, the factor the researcher deliberately changes is called the what?

Example 8

easy
Replicating an experiment on many subjects rather than one helps reduce the effect of what?

Example 9

medium
A company tests a new website layout by randomly showing layout A or B to visitors and measuring purchases. Name the design and the three core experimental features present.

Example 10

medium
A researcher wants to test a drug but only has 20 subjects. She gives all 20 the drug and compares their 'after' health to their 'before' health. What is the design flaw?

Example 11

medium
To test fertilizer, a farmer puts fertilizer on the sunny south field and none on the shady north field. Why is this a poor experiment, and what is the confounded variable?

Example 12

medium
An experiment compares a new painkiller to a sugar pill, with neither patients nor doctors knowing who got which. Name two design features used and what each controls.

Example 13

medium
A study randomly assigns 100 students to a tutoring group and 100 to no tutoring, but the tutoring group also happens to get free lunch. Identify the threat and how to fix the design.

Example 14

medium
A clinic measures blood pressure before and after a yoga program with no control group and reports a drop. Give two alternative explanations the design cannot rule out.

Example 15

medium
Design a fair experiment to test whether a music app improves studying. State the treatment, control, and randomization step in one sentence each.

Example 16

medium
A taste test gives brand X in a fancy cup and brand Y in a plain cup; X wins. Why is the design flawed and what variable is confounded?

Example 17

challenge
An experiment has 3 doses (low, medium, high) plus a placebo, with 25 subjects each. To claim a dose-response effect, what design features must hold, and why is randomization still required even with a placebo present?

Example 18

challenge
Two designs test the same drug: (A) randomized double-blind placebo-controlled trial, (B) compare patients who chose the drug to those who chose not to. Explain precisely why only A supports a causal claim.

Example 19

challenge
A researcher runs a well-randomized experiment but lets each subject pick their preferred measurement time, and outcomes are self-reported. Identify two residual biases that randomization does NOT fix and the design fixes for each.

Example 20

medium
An experiment tests a fertilizer with a treatment and control group but applies them on different days with different weather. What design principle is violated and how do you fix it?

Example 21

easy
An observational study notes that coffee drinkers tend to sleep less. Can we conclude that coffee causes less sleep?

Example 22

easy
A drug trial assigns patients to drug or placebo without telling them which. What is this called?

Example 23

easy
If both subjects and researchers measuring outcomes are kept unaware of assignment, the study is ___-blind.

Example 24

easy
A fertilizer experiment uses 30 plots randomly assigned: 15 to fertilizer A, 15 to fertilizer B. What is the sample size per treatment?

Example 25

medium
Subjects vary by sex, and sex may affect outcome. We randomize within sex groups. This is called ___.

Example 26

medium
Each subject is measured before and after taking a drug. What kind of design is this?

Example 27

medium
A study finds students who attend tutoring score higher, but students choose to attend. Why is this not causal evidence?

Example 28

medium
An A/B test randomly shows version A to 1000 users and version B to 1000 users. Click rates: A = 12%, B = 14%. Why might this difference still be due to chance?

Example 29

medium
A study assigns the first 50 patients to treatment and the next 50 to control. Why is this design flawed?

Example 30

medium
What is the purpose of a placebo in a drug trial?

Example 31

medium
A study compares two diets in 60 subjects. With random assignment, what kind of inference is supported by a statistically significant difference?

Example 32

medium
If a study has only 4 subjects per treatment, what statistical issue arises?

Example 33

medium
A factorial experiment crosses two factors at 2 levels each. How many treatment combinations?

Example 34

hard
A trial has 200 subjects and random assignment, but 30% in the treatment arm drop out. Why does this threaten conclusions?

Example 35

hard
A within-subject crossover trial assigns each subject to both treatments in random order. Why is this typically more powerful than a parallel-group design with the same nn?

Example 36

hard
A school randomly assigns classrooms (not students) to two curricula. What unit must inference treat as the experimental unit?

Example 37

hard
A pharma trial reports p=0.04p=0.04 from 20 outcomes tested. Why might this not be convincing evidence of a real effect?

Example 38

easy
An observational study records what people already do and looks for patterns. An experiment differs from an observational study primarily because the researcher does what?

Example 39

easy
Why are subjects randomly assigned to groups in an experiment, rather than allowing them to choose?

Example 40

easy
In an experiment with a 'treatment group' and 'control group,' which group receives the intervention being tested?

Example 41

easy
Why is replication (many subjects per group) important in an experiment?

Example 42

easy
Identify the response variable: 'Does fertilizer X increase tomato yield?' compared across fertilizer types.

Example 43

medium
A company tests two checkout-page designs by randomly showing 50%50\% of visitors version A and 50%50\% version B and comparing conversion. Name three key experimental features present.

Example 44

medium
A study finds that students who get more sleep get higher grades. Why can't we conclude that sleep CAUSES higher grades from this study?

Example 45

medium
An experiment tests a teaching method but gives it to morning classes and uses afternoon classes as control. What variable is confounded with method?

Example 46

medium
In a matched-pairs design, why are subjects paired and given different treatments within each pair?

Example 47

medium
A pharma trial randomizes patients within hospitals (so each hospital has both treated and control patients). What is this design called?

Example 48

medium
Two factors in a factorial design: drug (A vs B) and dose (low vs high). How many treatment combinations are there?

Example 49

medium
Researchers test a new drug. The control group receives a sugar pill. What is the sugar pill called and what does it control?

Example 50

hard
A study claims a vitamin reduces colds based on a sample of 8080 volunteer adults — all given the vitamin. After a year, 40%40\% report fewer colds than 'usual.' Identify two design weaknesses.

Example 51

hard
A teacher wants to compare two reading programs in her 44 classes. Why is randomly assigning programs to classes (rather than to individual students) limited as a causal study?

Example 52

hard
A drug trial randomizes patients but some in the treatment arm refuse the drug and switch to placebo. Why is 'intention-to-treat' analysis the standard, not 'as-treated'?

Example 53

hard
A study with 2020 subjects per group finds a small, non-significant difference between treatment and control. A larger replication with 20002000 per group finds the same small difference, but now significant. What does this say about sample size and effect detection?

Example 54

hard
Why is a within-subjects (crossover) design statistically more powerful than a between-subjects design, when feasible?

Example 55

challenge
A randomized trial finds that on average the treatment group did better. A critic notes that the trial only included healthy young adults. What concept limits generalizing the result to all patients?

Example 56

challenge
Three treatments are tested across 99 plots. The plots vary in soil quality. Suggest a design that uses blocking to improve precision.

Example 57

medium
A doctor tests a new headache medicine. She gives the medicine to patients who ask for it and compares their recovery to patients who didn't ask. Identify at least two problems with this experimental design.

Example 58

hard
Design a double-blind experiment to test whether caffeine improves reaction time. Specify: (a) how participants are assigned, (b) what the treatment and control conditions are, (c) what 'double-blind' means and why it matters, (d) the response variable.
← Back to Experimental Design Common Mistakes Practice Mode

Background Knowledge

These ideas may be useful before you work through the harder examples.

correlation vs causationdata collection