Logistic Regression and Classification

Logistic Regression and Classification

Despite the name, logistic regression is a classifier. It estimates the probability that an example belongs to a class and turns the probability into a label with a threshold. It is the workhorse baseline for binary classification.

From Line to Probability

A linear model can output any number, but probabilities live between 0 and 1. Logistic regression pushes the linear result through the sigmoid function to squeeze it into that range. Above a threshold, the model predicts one class; below it, the other.

Train a Classifier

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)
clf = LogisticRegression()
clf.fit(X_train, y_train)
preds = clf.predict(X_test)
print(accuracy_score(y_test, preds))

Read the Probabilities, Not Just the Labels

Call predict_proba to see the actual probability:

probs = clf.predict_proba(X_test)[:, 1]
print(probs[:5])

Ranking by probability beats thresholding when you have limited budget: predict the most confident cases first. Labels hide this information; probabilities expose it.

Accuracy Is Not Enough

If 95 percent of examples are class 0, a model that always predicts 0 scores 95 percent accuracy while learning nothing. For imbalanced data, use the confusion matrix, precision, recall, and F1 score during evaluation.

Class Weights and Tuning

Ask for class_weight="balanced" to make rare classes count more. Regularization strength C controls complexity; a lower value shrinks weights and fights overfitting. Use cross-validation rather than guessing values.

Key Points

• Logistic regression predicts class probabilities.
• Fit, predict, and score mirror the regression workflow.
• Rank by probabilities when budget is limited.
• Use precision, recall, and F1 for imbalanced data.