In regression analysis, residuals — the differences between observed and predicted values — are instrumental in assessing model fit and its assumptions. Among the many ways to scrutinize residuals, one of the most effective is by examining the “Studentized residuals.” These residuals, sometimes known as externally studentized residuals, are essentially a type of standardized residual that is particularly sensitive to potential outliers.
In this in-depth article, we will delve into Studentized residuals, their significance, and how to calculate them in R.
Overview:
- What Are Studentized Residuals?
- Why Are Studentized Residuals Important?
- Mathematical Computation
- Calculating Studentized Residuals in R
- Interpreting Studentized Residuals
- Addressing Concerns Based on Studentized Residuals
- Conclusion
1. What Are Studentized Residuals?
Studentized residuals are the residuals from a regression model that have been standardized by an estimate of their standard deviation. Unlike regular standardized residuals, which are divided by the standard deviation of the residuals, Studentized residuals are divided by an estimate of the standard deviation that excludes the residual being standardized.
2. Why Are Studentized Residuals Important?
- Outlier Detection: Studentized residuals are valuable for identifying outliers. A data point with a large studentized residual suggests that it doesn’t fit the model as well as the other observations.
- Model Assessment: Large studentized residuals may also indicate a lack of fit, suggesting that the model may need to be re-specified.
- Uniform Sensitivity: Because they account for the variability of the residual itself, Studentized residuals offer a uniform scale of sensitivity to outliers across all data points.
3. Mathematical Computation:
Given a residual ei​, the studentized residual ti is calculated as:
Where s(ei) is an estimate of the standard deviation of the residuals, excluding the ith observation.
4. Calculating Studentized Residuals in R:
R makes it straightforward to compute Studentized residuals for linear models. Let’s go through the process:
# Sample data
set.seed(123)
x <- rnorm(100)
y <- 1.5 * x + rnorm(100)
# Fit a linear regression model
model <- lm(y ~ x)
# Calculate Studentized residuals
studentized_residuals <- rstudent(model)
# Display the first few residuals
head(studentized_residuals)Using the rstudent() function from base R, you can easily obtain the Studentized residuals for your linear model.
5. Interpreting Studentized Residuals:
- Size: As a rule of thumb, a studentized residual with an absolute value greater than 2 or 3 might be considered unusual.
- Pattern: Like with regular residuals, you ideally want no patterns when plotting studentized residuals against fitted values. Patterns could suggest non-linearity, interactions, or other model specification issues.
6. Addressing Concerns Based on Studentized Residuals:
If you identify potential outliers or influential points based on studentized residuals:
- Data Verification: Before making any decisions, ensure that the data point in question isn’t a result of data entry error or measurement error.
- Model Re-specification: Consider whether the model needs to include non-linear terms, interactions, or other modifications.
- Robust Regression: If outliers are affecting the model adversely, consider robust regression techniques that down-weight the influence of outliers.
- Influential Points: Apart from size, you should check the influence of an observation. Tools like Cook’s distance can help assess the influence of individual data points on the entire model.
7. Conclusion:
Studentized residuals are a powerful diagnostic tool in regression analysis, offering an enhanced way to detect outliers and assess model fit. In R, the process of computing these residuals is straightforward, but interpreting and making decisions based on them requires careful consideration. Always approach potential outliers and model re-specifications judiciously, incorporating domain knowledge and other diagnostic tools.