Linear Regression with scikit-learn

Site Admin · 11 Sep 2026 · 8 views

Linear Regression with scikit-learn

Linear regression models a numeric target as a weighted sum of features, plus a constant term. It is simple, fast, and surprisingly dependable, and it is the right first model for many prediction problems.

The Idea

For features x and target y, the model finds weights w and a bias b such that y is close to the value of w times x plus b. Training picks the weights that minimize the average squared prediction error. That error is called the mean squared error (MSE).

Fit and Predict

from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)
model = LinearRegression()
model.fit(X_train, y_train)
preds = model.predict(X_test)
print(mean_squared_error(y_test, preds))

fit learns the weights; predict applies them. Two lines of model code after a standard split.

Interpret the Coefficients

The weights tell you the direction and strength of each feature. A slope of 2 for a feature means one unit of that feature is associated with two units of target, all else equal. Beware of units: coefficients are only comparable after scaling features to the same range.

for name, coef in zip(model.feature_names_in_, model.coef_):
    print(name, round(coef, 3))

Judge the Right Metric

Report MSE in the target units when the audience is a business, and use R-squared for a relative comparison of model fit. A high R-squared on training data with a poor test score means overfitting; trust the test score.

Limits

Linear regression assumes a roughly linear relationship and is sensitive to outliers, so plot the residuals (errors) against predictions. Funnels or curves in the plot are a signal to add nonlinear features or switch to a tree-based model.