Both L1 and L2 Regularization are used to prevent overfitting by penalizing the coefficients or weights. Below are the differences between L1 and L2 Regularization.
|L1 Regularization||L2 Regularization|
|Lasso Regression||Ridge Regression|
|Adds the absolute value of the coefficient as a penalty term to the loss function.||Adds the squared value of the coefficient as a penalty term to the loss function.|
|Penalizes the sum of the absolute value of the weights.||Penalizes the sum of squares of weights.|
|Performs Feature Selection.||Does not Perform Feature Selection.|
|Robust to outliers.||Not robust to outliers.|
|L = ∑(Ŷi– Yi)² + λ∑|β|||L = ∑(Ŷi– Yi)² + λ∑β²|