If your machine learning model learns the pattern of data very well while training, it can lead to the problem of **Overfitting. **Overfitting is a situation in which models perform very well on the training data and poor on testing data. To resolve this issue, a penalty term can be added to the equation of the model, this process is called **Regularization.**

There are three types of regularization models: Lasso Regression (L1 regularization) , Ridge Regression (L2 regularization) and Elastic net Regression.

## Ridge Regression

**Ridge Regression is a regularization method in which a small bias is added to the linear equation such that the overall complexity of the model is reduced.**- It is also called
**L2 Regularization**since the penalty added is squared (power 2). - The amount of bias added to the model is called the
**Ridge Regression penalty.** - It can handle data that suffers from
**Multicollinearity****problem**.

## Cost Function of Linear Regression

**Cost =**

## Cost Function of Ridge Regression

The cost function of Linear Regression is modified by adding a Regularization term to it.

**Cost =**

where

- slope
**m**is the coefficients (weight) of the independent features. - lambda
**Â Î»**is the penalty term.

## Idea behind Ridge Regression

- Regularization parameter is added to the cost function such that when an optimization algorithm like gradient descent is used to minimize the cost function, the values of weights are reduced.
- Reducing weights decreases the complexity of the model and hence resolves the Overfitting problem.

## Controlling the Regularization Parameter

- Lambda Î» which denotes the penalty term in the equation can be used to control the regularization parameter.
- A high value of lambda increases the regularization parameter and decreases the value of coefficient or weight. It can lead to the problem of Underfitting.
- A low value of lambda reduces the regularization parameter and does not have much effect on the value of coefficient or weight. Hence the problem of Overfitting may not get resolved.
- Finding an optimal value of Î» is a must to achieve a perfect Ridge Regression model.

## Advantages andÂ DisadvantagesÂ of Ridge Regression

- Advantage of ridge regression is that it prevents the model from overfitting by reducing the model complexity.
- Disadvantage is that it is not capable of performing feature selection.