What is Elastic net Regression?
Elastic net is a regularized linear regression model.
What is Regularization?
Linear Regression model can face problems like overfitting when the coefficients or the weights of the feature becomes very large. This problem can be resolved by techniques like regularization in which an additional cost (penalty) is added to the cost function. The regularization parameters reduce the coefficient values and hence the weightage of features which makes the model more generalised and solves problems like overfitting.
Regularization in Linear Regression
There are three types of regularized linear regression models:
1. Lasso Regression (L1 regularization)
Lasso is a regularized regression model which adds the absolute values of coefficients as the penalty factor to the cost function.
L = ∑(Ŷi– Yi)² + λ∑|β|
The first part of the equation is ∑( Ŷi– Yi)² squared sum of error and the second part is λ∑|β| regularization parameter (absolute sum of coefficients).
2. Ridge Regression (L2 regularization)
Ridge is a regularized regression model which adds the squared values of coefficients as the penalty factor to the cost function.
L = ∑(Ŷi– Yi)² + λ∑β²
The first part of the above equation is ∑( Ŷi– Yi)² squared sum of error and the second part is λ∑β² regularization parameter (squared sum of coefficients).
1. Lasso regression can be used for feature selection while ridge regression doesnot support feature selection. 2. Lasso regression can be used in cases where less features are significant as it reduces the value of insignificant features to zero whereas Ridge regression can be used in cases where more no. of features are significant.
3. Elastic net Regression
Elastic net is a modified version of linear regression that adds regularization penalties from both lasso and ridge regression during the training.
L = ∑(Ŷi– Yi)² + λ∑β² + λ∑|β|
The first part of the above equation is ∑( Ŷi– Yi)² squared sum of error and the second part is λ∑β² + λ∑|β| regularization parameter from both ridge and lasso regression.
Advantages of Elastic net Model
Ridge Regression is not capable of feature selection. Since the Elastic net model combines penalties from both l1 and l2 regularization. It has characteristics of both lasso and ridge regression hence it can perform feature selection and regularization at the same time.