Overfitting (Intuition) Examples in Math

Start with the recap, study the fully worked examples, then use the practice problems to check your understanding of Overfitting (Intuition).

This page combines explanation, solved examples, and follow-up practice so you can move from recognition to confident problem-solving in Math.

Concept Recap

Overfitting occurs when a model learns the noise in training data instead of just the underlying pattern, performing well on training data but poorly on new data.

The model memorized the training data instead of learning the underlying pattern.

Read the full concept explanation →

How to Use These Examples

  • Read the first worked example with the solution open so the structure is clear.
  • Try the practice problems before revealing each solution.
  • Use the related concepts and background knowledge badges if you feel stuck.

What to Focus On

Core idea: Overfitting: too complex. The model sees patterns that aren't really there.

Common stuck point: Adding more parameters always improves training fit but often hurts prediction.

Worked Examples

Example 1

medium
A model fits 10 data points with a degree-9 polynomial (perfect fit, R^2=1). A simpler linear model has R^2=0.85. Explain which model is better for prediction and why.

Solution

  1. 1
    Degree-9 polynomial: 10 parameters for 10 points — fits every point exactly (interpolates), but captures noise not just signal
  2. 2
    Linear model: 2 parameters — fits general trend, ignoring noise
  3. 3
    For prediction: the polynomial will perform poorly on new data (it memorized the training data's random fluctuations)
  4. 4
    The linear model captures the true underlying relationship better, generalizing to new data despite lower in-sample R^2

Answer

The linear model (R²=0.85) is better for prediction; the polynomial overfits noise and won't generalize.
Overfitting occurs when a model is too complex relative to the data. A perfect in-sample fit often means the model learned random noise rather than the true signal. Model selection requires balancing fit and complexity using cross-validation or penalty criteria (AIC, BIC).

Example 2

hard
A machine learning model is trained on 1000 observations with 50 predictors. Training error is near zero; test error on 200 held-out observations is very high. Diagnose the problem and suggest two remedies.

Practice Problems

Try these problems on your own first, then open the solution to compare your method.

Example 1

easy
A student memorizes all 500 practice problems but performs poorly on the exam, which has new problems. How does this analogy illustrate overfitting?

Example 2

hard
Explain the bias-variance tradeoff: how does increasing model complexity affect bias and variance, and where is the optimal model?
← Back to Overfitting (Intuition) Practice Mode

Background Knowledge

These ideas may be useful before you work through the harder examples.

model fit intuition