Inference Concepts

A confidence interval is a range of values, calculated from sample data, constructed so that the procedure captures the true population parameter a specified percentage of the time (e.g., 95%). It quantifies the uncertainty inherent in using a sample to estimate a population value.

"Instead of saying 'the average is 50,' you say 'I'm 95% confident the average is between 47 and 53.' The interval acknowledges uncertainty from sampling."

Why it matters: Confidence intervals quantify uncertainty. They're essential for making decisions based on sample data.

The margin of error is the maximum expected difference between a sample statistic and the true population parameter, typically expressed as a plus-or-minus value. It equals half the width of a confidence interval and decreases as sample size increases.

"When a poll says '52% $\pm$ 3%,' that 3% is the margin of error. It means the true value is probably within 3 percentage points of 52%, so between 49% and 55%."

Why it matters: Margin of error helps you interpret poll results and survey findings with appropriate uncertainty.

Hypothesis testing is a formal statistical procedure for using sample data to decide between two competing claims about a population parameter. You state a null hypothesis (no effect) and an alternative hypothesis, collect data, compute a test statistic, and determine whether the evidence is strong enough to reject the null.

"Hypothesis testing is like a courtroom trial for data. You start by assuming innocence (null hypothesis: nothing special is happening). Then you look at the evidence (data). If the evidence is strong enough to be very unlikely under the assumption of innocence, you reject it and conclude something real is happening."

Why it matters: Hypothesis testing is the backbone of the scientific method for data analysis. It is used to approve new drugs in clinical trials, test whether a business strategy improves revenue, validate research findings, and make evidence-based decisions in engineering and policy.

The p-value is the probability of observing results at least as extreme as the actual data, calculated under the assumption that the null hypothesis is true. A small p-value (typically below 0.05) suggests the observed data is unlikely under the null, providing evidence against it.

"P-value answers: 'If nothing special is really happening, how surprising is my data?' A tiny p-value (like 0.01) means your results would be very rare if the null were true - so maybe the null is wrong. A large p-value means your results aren't surprising under the null."

Why it matters: P-values are reported in virtually every scientific paper, clinical trial, and A/B test. They are the standard way to quantify evidence in medicine, psychology, economics, and engineering, making them essential for data-driven decision-making.

A result is statistically significant when the p-value falls below a predetermined threshold (alpha, typically 0.05), indicating that the observed effect is unlikely to have occurred by random chance alone. Statistical significance is a binary decision criterion used in hypothesis testing — it does not measure the size or practical importance of the effect.

"Statistical significance is a decision rule: before looking at data, you set a threshold (usually 5%). If your p-value is below this threshold, you declare the result 'significant' - meaning unlikely to be just random noise. It's not about importance; it's about confidence that something real is happening."

Why it matters: Statistical significance is the standard threshold for publishing research findings, approving medical treatments, and making evidence-based decisions across science and industry. However, it is widely misunderstood — significance does not mean the effect is large, important, or practically meaningful.