Underfitting (Intuition) Math Example 4
Follow the full solution, then compare it with the other examples linked below.
Example 4
hardA linear model for time-series stock prices has high residuals for all time periods. Identify this as underfitting and explain why stock prices might inherently have an upper bound on regardless of model complexity.
Solution
- 1 High residuals everywhere (train and test): indicates underfitting — the model misses systematic patterns
- 2 Possible cause: stock prices may follow non-linear or regime-switching dynamics that a linear model cannot capture
- 3 Upper bound on : stock prices contain intrinsic randomness (random walk component); even a perfect model cannot explain this irreducible noise
- 4 Implication: in genuinely stochastic systems, is bounded below 1 even with the best possible model — this is noise (irreducible error)
Answer
Underfitting due to non-linear structure. Stock price randomness creates irreducible noise that limits maximum achievable .
Irreducible error is the variance in y that no model can explain — it is inherent to the process. Even a perfect model cannot exceed the signal-to-noise ceiling. This is why expecting R²→1 for inherently noisy processes (weather, stocks) is unrealistic.
About Underfitting (Intuition)
Underfitting occurs when a model is too simple to capture the true pattern in the data, performing poorly on both training data and new data.
Learn more about Underfitting (Intuition) →More Underfitting (Intuition) Examples
Example 1 easy
A scatter plot shows a clear U-shaped relationship between [formula] and [formula]. A linear model i
Example 2 mediumCompare two regression models on the same data: Model 1 (linear): training [formula], test [formula]
Example 3 easyA student uses [formula] (the mean) to predict all exam scores. Training [formula]. Explain why this