R has emerged as one of the leading software packages for data and statistical analysis today. It is an open-source environment preferred for its strong computing, visual, and graphics capabilities. If you are an engineering student, a business analytics professional, or someone with a keen interest in data science, learning the R programming language can help you in a lot of ways.
In this article, we will cover some basic concepts of machine learning through this integrated suite. More specifically, we will discuss how to calculate a confusion matrix in R.
But before we move on to the technicalities, let us first understand why we have chosen R for this purpose. It is because of the following benefits that this programming language is gaining popularity among statisticians and data scientists worldwide:
- Reproducible: With R, you can reproduce reports and write reusable code
- Shareable: It has a low learning curve, which opens up avenues for collaboration
- Repeatable: Anyone can not only understand what you did but also repeat the steps to create the same functions on their machines
The use of R language in the field of machine learning has also picked up due to the above reasons. You don’t need to be an expert programmer to make sense of its syntax. And so, we are introducing some fundamentals to you in the next section.
Table of Contents
What is a Confusion Matrix?
A confusion matrix, or an error matrix, deals with the standard classification problem in statistics. It comprises a specific table layout that facilitates data analysts to visualize how an algorithm performs. This particularly applies to supervised learning algorithms.
To elaborate further, a confusion matrix follows a N x N format, where N refers to the number to target classes. You can use this table or matrix to evaluate a classification model’s performance. This is possible because the matrix compares the predicted values with the target values.
In a nutshell, you can describe how your machine learning model, a classifier, in this case, works on a set of test data (for which you already have the true values).
To understand this method, you need to be familiar with the following terms:
- True Positive (TP): Positive values are correctly predicted
- False Positive (FP): Negative values are incorrectly predicted as positive
- False Negative (FN): Positive values predicted as negative
- True Negative (TN): Negative values predicted as actual negative values
Let us look at some examples to gain more clarity.
Confusion Matrix Examples
- True Positive
When you had predicted India to win the Cricket World Cup, and it won.
- False Positive
When you had expected India to win, but it lost.
- False Negative
When you had predicted that France would not win, but it won.
- True Negative
When you projected that India would ‘not win’ the Cricket world cup and it lost the series in real life.
As we move further, you should remember that all predicted values are described as: Positive, Negative, True, and False.
How to Calculate the Confusion Matrix in R?
Consider a scenario where you have a list of expected or known values and another list of predictions from your machine learning model. In R, you can calculate the confusion matrix using a simple function from the caret library: confusionMatrix(). It can not only calculate the matrix but also return a detailed report for the results.
You can follow the below-mentioned steps to practice the process of data mining:
- Test the given dataset with the expected outcomes.
- Predict the rows of your test dataset.
- Determine the total counts of correct and incorrect predictions for each class.
Once you have done this, you will find the numbers organized in the following fashion:
- Every row of the matrix will correspond with a predicted class and every column will be linked to an actual class.
- The total number of correct and incorrect classifications are reflected in the table, along with the sums for each class.
Suppose you have 10 persons divided into two classes, male and female. You have to arrange the information as a confusion matrix when you know that 2 men were classified as women, while 1 woman was classified as a man.
women 3 1
men 2 4
Here, the correct values are organized in a diagonal line from the top left to the bottom-right of the matrix (3 + 4). The results tell us that there more errors with predicting male members as women than predicting females as men. The algorithm made 7 correct predictions out of 10 possible outcomes, which means it has a 70% accuracy.
Guide to Making and Calculating a Confusion Matrix in R
As you can observe, the confusion matrix function is a useful tool for examining the possible outcomes of your predictions. So, before you begin creating your matrix, you first need to have a “cut” of your probability values. In other words, you need to mark a threshold to turn your probabilities into class predictions.
To do this, you can use the ifelse() function. For example:
ifelse (probability_prediction > 0.50,
You can also write the table() function to make a contingency table in base R. However, the confusionMatrix() function is known to yield valuable ancillary statistics.
The next step is to calculate the confusion matrix and other associated stats. Here, you would need the predicted and actual outcomes. Take, for instance, the statement given below:
Now, you should proceed with turning your numeric predictions into a vector of class predictions, sat p_class. Suppose you want to use a cutoff of 0.50.
Also, while making predictions, don’t forget to name the positive and negative classes with separate indicators. Let’s call the positive classes “T” and name the negative ones as “L”. This is done to match classes with the original data.
Now that you have a p_class and actual values in the test dataset, you can start making your confusion matrix, calling the confusionMatrix() function.
Alternatively, you may want to be sure about the accuracy of your data mining model. In such cases, it is advisable to use a threshold of 0.10, not 0.90. thereafter, you can continue with the same steps as you did in the earlier exercise.
With your new predicted classes, you can repeat this call:
pred <- ifelse(probability > threshold, “T”, “L”)
Finally, you can use the confusionMatrix() function in caret:
With this, we conclude this tutorial on the confusion matrix function for machine learning in R. Hope you found it helpful!
If you are curious to learn about R, data science, check out our PG Diploma in Data Science which is created for working professionals and offers 10+ case studies & projects, practical hands-on workshops, mentorship with industry experts, 1-on-1 with industry mentors, 400+ hours of learning and job assistance with top firms.
The following reasons introduce us to the benefits of having a confusion matrix and how it deals with performance issues. The confusion matrix in R can be calculated by using the “confusionMatrix()” function of the caret library. This function not only calculates the matrix but also returns a detailed report of the matrix. You must follow some steps in order to calculate your confusion matrix. You can calculate the accuracy rate of a model by using a 2x2 confusion matrix. The following formula will get you the success rate or the accuracy rate:
What is the need of creating a confusion matrix?
1. The confusion matrix is needed to eradicate the issue with classification accuracy. The classification ratio often causes some problems by concealing the necessary details of the model.
2. The confusion matrix gives an insight into the predictions, and type of errors made in the classification model. The correct and faulty predictions are presented in a summarized manner.
3. The errors and their types are classified to give you an understanding of the performance of your model.
How to calculate the confusion matrix in R?
1. Test your dataset.
2. Predict its total number of rows.
3. Predict the total correct and incorrect predictions for every class.
Once you have mined your data, you will get the numbers organized in rows. The rows will be connected to the predicted class while the columns will be connected to the actual class. The correct values will be in a diagonal line. Add all the values and you will get the accuracy percentage of your matrix.
How to measure the performance in a confusion matrix?
Accuracy = (TP+TN)/(TP+TN+FP+FN)
Where, TP = True Positive ,TN = True Negative,FP = False Positive, FN = False Negative
The error rate of your model can also be calculated with the rate calculating formula which is:
Accuracy = (TP+TN)/(TP+TN+FP+FN) = 1-Accuracy
The concept of the error rate is very simple. Suppose your model has an accuracy rate of 80% then the error rate of your model will be 20%.
The following reasons introduce us to the benefits of having a confusion matrix and how it deals with performance issues.
The confusion matrix in R can be calculated by using the “confusionMatrix()” function of the caret library. This function not only calculates the matrix but also returns a detailed report of the matrix. You must follow some steps in order to calculate your confusion matrix.
You can calculate the accuracy rate of a model by using a 2x2 confusion matrix. The following formula will get you the success rate or the accuracy rate: