How to analyze Ridge and Lasso Regression in R software?

Ridge Regression It's important to understand the concept of regularization before jumping to ridge regression. 1. Regularization Regularization helps to solve over fitting problem which implies model performing well on training data but performing poorly on validation (test) data. Regularization solves this problem by adding a penalty term to the objective function and control the model complexity using that penalty term. Regularization is generally useful in the following situations: 1. Large number of variables 2. Low ratio of number observations to number of variables 3. High Multi-Collinearity 2. L1 Loss function or L1 Regularization In L1 regularization we try to minimize the objective function by adding a penalty term to the sum of the absolute values of coefficients. This is also known as least absolute deviations method. Lasso Regression makes use of L1 regularization. 3. L2 Loss function or L2 Regularization In L2 regularization we try to minimize the objective function by adding a penalty term to the sum of the squares of coefficients. Ridge Regression or shrinkage regression makes use of L2 regularization. Using cv.glmnet( ) function we can do cross validation. By default alpha = 0 which means we are carrying out ridge regression. lambda is a sequence of various values of lambda which will be used for cross validation. We take the best lambda by using lambda.min and hence get the regression coefficients using predict function. Lasso Regression Lasso stands for Least Absolute Shrinkage and Selection Operator. It makes use of L1 regularization technique in the objective function. Using cv.glmnet in glmnet package we do cross validation. For lasso regression we set alpha = 1. We consider the best value of lambda by filtering out lamba.min from the model and hence get the coefficients. Advantage of lasso over ridge regression Lasso regression can perform in-built variable selection as well as parameter shrinkage. While using ridge regression one may end up getting all the variables but with Shrinked Parameters. Which one is better - Ridge regression or Lasso regression? Both ridge regression and lasso regression are addressed to deal with multicollinearity. Ridge regression is computationally more efficient over lasso regression.