Overfitting (Intuition) Math Example 3
Follow the full solution, then compare it with the other examples linked below.
Example 3
easyA student memorizes all 500 practice problems but performs poorly on the exam, which has new problems. How does this analogy illustrate overfitting?
Solution
- 1 Training data = 500 practice problems (memorized perfectly = 100% training accuracy)
- 2 Test data = new exam problems (poor performance = high test error)
- 3 The student learned specific answers, not the underlying concepts (the signal)
- 4 Overfitting analogy: like a model that memorizes training data patterns without learning the true underlying structure
Answer
Memorizing answers = overfitting; fails on new problems because concepts (signal) weren't learned.
Overfitting is like rote memorization — perfect training performance but no generalization. A good model (good student) learns the underlying principles that apply to new situations, not the specific examples it was trained on.
About Overfitting (Intuition)
Overfitting occurs when a model learns the noise in training data instead of just the underlying pattern, performing well on training data but poorly on new data.
Learn more about Overfitting (Intuition) →More Overfitting (Intuition) Examples
Example 1 medium
A model fits 10 data points with a degree-9 polynomial (perfect fit, [formula]). A simpler linear mo
Example 2 hardA machine learning model is trained on 1000 observations with 50 predictors. Training error is near
Example 4 hardExplain the bias-variance tradeoff: how does increasing model complexity affect bias and variance, a