Describe the bias-variance tradeoff and how regularization affects it?

• A. Higher bias with underfitting, higher variance with overfitting; regularization reduces variance and can increase bias.
• B. Regularization increases variance.
• C. Regularization does not affect bias or variance.
• D. The bias-variance tradeoff says bias and variance are independent.

Answer: (A)

The main idea is how model complexity affects prediction errors through bias and variance. A too-simple model tends to miss patterns in the data, giving high bias and underfitting. A too-flexible model captures noise from the training data, giving high variance and overfitting. Regularization adds a penalty that discourages complexity, so the model becomes less sensitive to training data fluctuations, reducing variance. This comes at the cost of some extra bias because the constraints pull the solution away from the unconstrained optimum. In practice, regularization often improves generalization by lowering variance, while bias increases only modestly. That’s why the best description is: higher bias with underfitting, higher variance with overfitting; regularization reduces variance and can increase bias.