Machine Learning algorithms are generally categorized based upon the type of output variable and the type of problem that needs to be addressed. These algorithms are broadly divided into three types i.e. Regression, Clustering, and Classification. Regression and Classification are types of supervised learning algorithms while Clustering is a type of unsupervised algorithm.
When the output variable is continuous, then it is a regression problem whereas when it contains discrete values, it is a classification problem. Clustering algorithms are generally used when we need to create the clusters based on the characteristics of the data points. This article focuses on giving a brief introduction to clustering, classification and list some differences between the two.
Classification is a type of supervised machine learning algorithm. For any given input, the classification algorithms help in the prediction of the class of the output variable. There can be multiple types of classifications like binary classification, multi-class classification, etc. It depends upon the number of classes in the output variable.
Types of Classification algorithms
Logistic Regression: – It is one of the linear models which can be used for classification. It uses the sigmoid function to calculate the probability of a certain event occurring. It is an ideal method for the classification of binary variables.
K-Nearest Neighbours (kNN): – It uses distance metrics like Euclidean distance, Manhattan distance, etc. to calculate the distance of one data point from every other data point. To classify the output, it takes a majority vote from k nearest neighbors of each data point.
Decision Trees: – It is a non-linear model that overcomes a few of the drawbacks of linear algorithms like Logistic regression. It builds the classification model in the form of a tree structure that includes nodes and leaves. This algorithm involves multiple if-else statements which help in breaking down the structure into smaller structures and eventually providing the final outcome. It can be used for regression as well as classification problems.
Random Forest: – It is an ensemble learning method that involves multiple decision trees to predict the outcome of the target variable. Each decision tree provides its own outcome. In the case of the classification problem, it takes the majority vote of these multiple decision trees to classify the final outcome. In the case of the regression problem, it takes the average of the values predicted by the decision trees.
Naïve Bayes: – It is an algorithm that is based upon Bayes’ theorem. It assumes that any particular feature is independent of the inclusion of other features. i.e. They are not correlated to one another. It generally does not work well with complex data due to this assumption as in most of the data sets there exists some kind of relationship between the features.
Support Vector Machine: – It represents the data points in multi-dimensional space. These data points are then segregated into classes with the help of hyperplanes. It plots an n-dimensional space for the n number of features in the dataset and then tries to create the hyperplanes such that it divides the data points with maximum margin.
- Email Spam Detection.
- Facial Recognition.
- Identifying whether the customer will churn or not.
- Bank Loan Approval.
Clustering is a type of unsupervised machine learning algorithm. It is used to group data points having similar characteristics as clusters. Ideally, the data points in the same cluster should exhibit similar properties and the points in different clusters should be as dissimilar as possible.
Clustering is divided into two groups – hard clustering and soft clustering. In hard clustering, the data point is assigned to one of the clusters only whereas in soft clustering, it provides a probability likelihood of a data point to be in each of the clusters.
Types of Clustering algorithms
K-Means Clustering: – It initializes a pre-defined number of k clusters and uses distance metrics to calculate the distance of each data point from the centroid of each cluster. It assigns the data points into one of the k clusters based on its distance.
Agglomerative Hierarchical Clustering (Bottom-Up Approach): – It considers each data point as a cluster and merges these data points on the basis of distance metric and the criterion which is used for linking these clusters.
Divisive Hierarchical Clustering (Top-Down Approach): – It initializes with all the data points as one cluster and splits these data points on the basis of distance metric and the criterion. Agglomerative and Divisive clustering can be represented as a dendrogram and the number of clusters to be selected by referring to the same.
DBSCAN (Density-based Spatial Clustering of Applications with Noise): – It is a density-based clustering method. Algorithms like K-Means work well on the clusters that are fairly separated and create clusters that are spherical in shape. DBSCAN is used when the data is in arbitrary shape and it is also less sensitive to the outliers. It groups the data points that have many neighbouring data points within a certain radius.
OPTICS (Ordering Points to Identify Clustering Structure): – It is another type of density-based clustering method and it is similar in process to DBSCAN except that it considers a few more parameters. But it is more computationally complex than DBSCAN. Also, it does not separate the data points into clusters, but it creates a reachability plot which can help in the interpretation of creating clusters.
BIRCH (Balanced Iterative Reducing and Clustering using Hierarchies): – It creates clusters by generating a summary of the data. It works well with huge datasets as it first summarises the data and then uses the same to create clusters. However, it can only deal with numeric attributes that can be represented in space.
Also Read: Data Mining Algorithms You Should Know
- Segmentation of consumer base in the market.
- Analysis of Social network.
- Image segmentation.
- Recommendation Systems.
Difference Between Clustering and Classification
- Type: – Clustering is an unsupervised learning method whereas classification is a supervised learning method.
- Process: – In clustering, data points are grouped as clusters based on their similarities. Classification involves classifying the input data as one of the class labels from the output variable.
- Prediction: – Classification involves the prediction of the input variable based on the model building. Clustering is generally used to analyze the data and draw inferences from it for better decision making.
- Splitting of data: – Classification algorithms need the data to be split as training and test data for predicting and evaluating the model. Clustering algorithms do not need the splitting of data for its use.
- Data Label: – Classification algorithms deal with labelled data whereas clustering algorithms deal with unlabelled data.
- Stages: – Classification process involves two stages – Training and Testing. The clustering process involves only the grouping of data.
- Complexity: – As classification deals with a greater number of stages, the complexity of the classification algorithms is higher than the clustering algorithms whose aim is only to group the data.
The methodology of classification and clustering is different, and the outcome expected from their algorithms differs as well. In a nutshell, both classification and clustering are used to tackle different problems. This article provided a brief introduction to classification and clustering.
We also read a bit about the different types of algorithms used in each case along with a few applications. The algorithms listed in this article are not exhaustive. i.e. It is not a complete list and there exists many other algorithms which can be used to tackle such problems.
If you are curious to learn data science, check out our PG Diploma in Data Science which is created for working professionals and offers 10+ case studies & projects, practical hands-on workshops, mentorship with industry experts, 1-on-1 with industry mentors, 400+ hours of learning and job assistance with top firms.