What is Regularization in ML? What is L1 and L2 Regularization?

Spread the love

What is Regularization in Machine Learning?

Regularization is about adding constraints in the model which reduces a model complexity and prevents it from overfitting.

What is L1 and L2 Regularization in Machine Learning ?

L1 Regularization –

L1 regularization penalizes weights in proportion to the sum of the absolute values of the weights. It drives the weights of the irrelevant features to exactly 0, which removes those features from the model. L1 regularization does automatic feature selection.

L2 Regularization –

L2 regularization penalizes the weights in proportion to the sum of the squares of the weights. L2 regularization helps drive outliers weights ( those with high positive or low negative values) closer to 0 but not quite 0.

Leave a Reply