A Gentle Introduction to Lasso Regression in Machine Learning

Spread the love

What is Lasso Regression in Machine Learning ?

Lasso Stands for Least Absolute Shrinkage and Selection Operator Regression ( usually simply called Lasso Regression). It is another regularized version of Linear Regression. Just like Ridge regression, it adds a regularization term to the cost function but it uses the l1 norm of the weight vector instead of half the square of the l2 norm.

An important characteristic of Lasso Regression is that it tends to eliminate the weights of the least important features (i.e. sets them to zero). In other words, lasso regression automatically performs feature selection and outputs a sparse model( i.e. with few non zero feature weights).

How to train a Lasso Regression model in scikit-Learn ?

Let’s read a dataset to work with.

import pandas as pd
import numpy as np
from sklearn import datasets

# Read a dataset
housing = datasets.fetch_california_housing()
X = pd.DataFrame(housing.data, columns=housing.feature_names)
y = housing.target
X.head()

Split the data into a training and test set.

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

Now, let’s train a Lasso Regression model in sklearn.

from sklearn.linear_model import Lasso
from sklearn.metrics import mean_squared_error

# create a lasso regression model
lasso_reg = Lasso(alpha=0.1)
# train it on the training data
lasso_reg.fit(X_train, y_train)
# make prediction on the test set
y_pred = lasso_reg.predict(X_test)
# measure error
mse = mean_squared_error(y_test, y_pred)
rmse = np.sqrt(mse)
rmse
# output
0.7832697618354822

You can also use Stochastic Gradient Descent to train a lasso regression model. You have to just set the penalty parameter to l1.

from sklearn.linear_model import SGDRegressor

sgd_reg = SGDRegressor(penalty='l1')
sgd_reg.fit(X_train, y_train)
y_pred = sgd_reg.predict(X_test)
mse = mean_squared_error(y_test, y_pred)
rmse = np.sqrt(mse)
rmse
# output
1469089561529609.2

Related Posts –

  1. Introduction to Linear Regression in Machine Learning
  2. Introduction to Ridge Regression in Machine Learning

Rating: 1 out of 5.

Leave a Reply