What is Lasso Regression in Machine Learning ?
Lasso Stands for Least Absolute Shrinkage and Selection Operator Regression ( usually simply called Lasso Regression). It is another regularized version of Linear Regression. Just like Ridge regression, it adds a regularization term to the cost function but it uses the l1 norm of the weight vector instead of half the square of the l2 norm.
An important characteristic of Lasso Regression is that it tends to eliminate the weights of the least important features (i.e. sets them to zero). In other words, lasso regression automatically performs feature selection and outputs a sparse model( i.e. with few non zero feature weights).
How to train a Lasso Regression model in scikit-Learn ?
Let’s read a dataset to work with.
import pandas as pd import numpy as np from sklearn import datasets # Read a dataset housing = datasets.fetch_california_housing() X = pd.DataFrame(housing.data, columns=housing.feature_names) y = housing.target X.head()
Split the data into a training and test set.
from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
Now, let’s train a Lasso Regression model in sklearn.
from sklearn.linear_model import Lasso from sklearn.metrics import mean_squared_error # create a lasso regression model lasso_reg = Lasso(alpha=0.1) # train it on the training data lasso_reg.fit(X_train, y_train) # make prediction on the test set y_pred = lasso_reg.predict(X_test) # measure error mse = mean_squared_error(y_test, y_pred) rmse = np.sqrt(mse) rmse
# output 0.7832697618354822
You can also use Stochastic Gradient Descent to train a lasso regression model. You have to just set the penalty parameter to l1.
from sklearn.linear_model import SGDRegressor sgd_reg = SGDRegressor(penalty='l1') sgd_reg.fit(X_train, y_train) y_pred = sgd_reg.predict(X_test) mse = mean_squared_error(y_test, y_pred) rmse = np.sqrt(mse) rmse
# output 1469089561529609.2
Related Posts –
- Introduction to Linear Regression in Machine Learning
- Introduction to Ridge Regression in Machine Learning