AI & ML: Classification Explained

Predicting a Category

Classification predicts which group a sample belongs to. Email spam or not, a loan approved or refused, an image showing a cat or a dog. Unlike regression, the output is a discrete choice from a fixed set of classes.

How a Classifier Works

Most classifiers compute a confidence score for every class and pick the highest one. The raw inputs are features such as pixel brightness or message length. Inside the model, thousands of small numeric decisions push the scores around until one class wins.

# features: keyword_count, links, uppercase_ratio
# scores the model outputs
spam: 0.87
not_spam: 0.13

The model predicts spam because 0.87 beats 0.13. Training teaches the model to produce high scores for the correct class through thousands of examples.

Popular Classifiers

• Logistic regression - a linear model squashed into a probability between 0 and 1.
• Decision trees - a series of if then rules learned from data, interpretable by humans.
• Random forests - many trees voting together, robust and hard to overfit.
• Support vector machines - find a boundary with the largest margin between classes.
• Neural networks - flexible models that shine on images, audio, and text.

There is no single best classifier. Tabular data often falls to random forests, while images and language belong to deep networks. Start simple, then grow only when the simple model stops improving.

Multiclass and Imbalance

Classification is not limited to two classes. A document can be one of politics, sports, or science, and the model outputs a score per class. Real data is often imbalanced: fraud happens rarely, so a model that always predicts normal achieves high accuracy while catching nothing. Accuracy alone is misleading there, which is why precision and recall are tracked alongside it.

Key Points

• Classification predicts a discrete category from a fixed set of classes.
• Models score every class and predict the highest scoring one.
• Random forests and neural networks cover most practical cases.
• Imbalanced data makes accuracy misleading; check precision and recall.