Question 6What happens when you increase the regularization hyperparameter lambda?

When you increase the regularization hyperparameter lambda, the following happens:

1. Increasing lambda adds a penalty to the cost function. This penalty is proportional to the size of the coefficients of the model.

2. As a result, the model becomes more biased, meaning it will become less likely to overfit the training data. This is because the penalty discourages the model from assigning too much importance to any one feature, thus reducing the complexity of the model.

3. However, if lambda is set too high, the model may become underfit. This means it may become too simple and not capture the underlying pattern in the data, leading to poor performance.

4. Therefore, choosing the right value for lambda is a trade-off between bias and variance. A good way to find an optimal value is through cross-validation.

5. Lastly, increasing lambda also makes the model more stable in terms of the input data. Small changes in the training set will not drastically change the learned parameters.

In summary, increasing the regularization hyperparameter lambda can help prevent overfitting by adding a penalty to the cost function, but if set too high, it can cause the model to underfit.