What is classification in AI/ML?

Classification is a natural language processing task used in machine learning to assign labels to
data items. With an input training dataset, classification algorithms can identify categories and
classify subsequent data accordingly. So they essentially identify and recognize patterns in the
training data and use the findings to find similar patterns in future data.

Let's look at an example: Spam detection is a common use of classification. A spam filter looks for
patterns in an email and determines whether it should be categorized as spam or not. This works
by looking for associated spam characteristics, such as the patterns in images or links, and
classifying the rest of the email accordingly.

To give another example, financial institutions use classification algorithms to detect fraudulent
transactions. By looking for patterns that are associated with fraud, they can automatically flag
potential fraudulent activities and prevent them from happening.

Types of classification algorithms

Support Vector Machines (SVM)

Support Vector Machines (SVM) are robust and effective machine learning algorithms that
transform your data into a linear decision space. The algorithm then determines the optimal
hyperplane in this linear decision space, which separates your training data into different classes,
such as legitimate or spam email.

SVM is particularly good at separating similar things. For instance, all the examples in a group
might be similar, while some examples in a different group might be very different from each other.
K-Nearest Neighbors (KNN)

The K-nearest neighbors algorithm is one of the most basic classification algorithms. It compares
each example of a particular class to all examples of that class by proximity.

This is useful for two reasons:

1. It’s easy to understand and implement.

2. It’s surprisingly accurate.

It's easy to implement because we can just take the mean of each class, which is easy to calculate.
KNN can also be used to solve classification problems with multiple classes, since it's not sensitive
to the number of attributes that can be used.

Decision Trees

Decision trees are a very common way to represent and visualize possible outcomes of a decision
or an action based on probabilities. In a nutshell, a decision tree is a hierarchical representation
of the outcome space where every node represents a choice or an action, with leaves representing
states of the outcome.

Decision trees are highly human-readable since they generally follow a particular structure such
as:

 Start with a root node (an issue, question, or assumption).

 Find the next route.
 Repeat this process until you’ve completed a route and received an outcome.

Now, we can visualize decision trees as a treelike structure where each leaf represents a state or an
outcome, and the internal branches represent the various paths that could lead to that leaf. This is
demonstrated in the following figure.
In our example above, consider whether or not the weather conditions are OK to play a football
game. The first step would be to start at the root node labeled “Weather”, representing current
weather conditions. We then move onto a branch, and the next nodes, depending on whether it’s
sunny, overcast, or rainy. Finally, we continue down the decision tree until we have an outcome:
could play or couldn’t play.

However, this simplicity is deceptive because decision trees can be used across a broad range of
applications. For example, they are widely used in:

 Medical diagnosis: A patient’s symptoms and medical history give rise to a number of
possible diagnoses and treatments.
 Product or service selection: Different brands of cars, types of insurance products, etc.
 Conducting market research: Evaluating different solutions for a problem you’re trying to
solve.

While decision trees and neural networks each represent a different way of classifying (or grouping)
data into clusters that share common characteristics (or features), there are some key differences
between them.

Essentially, decision trees work best for simple cases with few variables, while neural networks
perform better when the data has more complex relationships between features or values (i.e., it’s
“dense”).

As such, decision trees are often used as the first line classification method in simple data science
projects. However, they may not scale well when faced with large amounts of high-dimensional
data, i.e., it’s difficult to interpret meaningful results from their analysis.
Logistic regression

Logistic regression is actually a non-linear extension of linear regression that allows us to handle
classes. This is achieved by classifying predictions into a given class based on a probability
threshold.

Consider the following example of the probability of passing an exam versus hours of studying.
Suppose we have a variable Y representing the probability of passing an exam and a variable X
representing hours spent studying. In that case, we can fit a line through these points using a
regression predictor.

We may then classify the point into one of two categories: pass or fail, based on how close its line
is to a threshold. While this is a simple example, logistic regression is used in many real-world
situations, such as when determining creditworthiness along a spectrum of categories, as well as
other multi-label classification problems.

Artificial Neural Networks (ANNs)

Artificial Neural Networks have been one of the most commonly used machine learning algorithms.
As the name suggests, they emulate biological neural networks in computers.

The way ANNs work is by training a set of parameters on data. These parameters are then used to
determine the model’s output, which may be an input or an action.

For example, if we have a model for predicting whether or not someone will buy a car based on
previous purchases and demographics, we can give it new data to predict what’s next. We can use
multiple inputs, including things like age, marital status, income, and so on to drive its predictions.
The simplest form of artificial neural networks is the perceptron. Nowadays, deep neural networks
are used for more complex problems like image categorization, where there are countless possible
inputs (e.g., different road conditions for a self-driving car).

The end goal of the training is to tune the parameters (weights) for each node so that they best
represent the data it has seen. When all nodes are trained, they should be able to make decisions
based on their parameters.