Blog_Banner_Asset
    Homebreadcumb forward arrow iconBlogbreadcumb forward arrow iconArtificial Intelligencebreadcumb forward arrow iconGaussian Naive Bayes: What You Need to Know?

Gaussian Naive Bayes: What You Need to Know?

Last updated:
22nd Feb, 2021
Views
Read Time
8 Mins
share image icon
In this article
Chevron in toc
View All
Gaussian Naive Bayes: What You Need to Know?

Gaussian Naive Bayes

Naïve Bayes is a probabilistic machine learning algorithm used for many classification functions and is based on the Bayes theorem. Gaussian Naïve Bayes is the extension of naïve Bayes. While other functions are used to estimate data distribution, Gaussian or normal distribution is the simplest to implement as you will need to calculate the mean and standard deviation for the training data.

Top Machine Learning and AI Courses Online

What is the Naive Bayes Algorithm?

Naive Bayes is a probabilistic machine learning algorithm that can be used in several classification tasks. Typical applications of Naive Bayes are classification of documents, filtering spam, prediction and so on. This algorithm is based on the discoveries of Thomas Bayes and hence its name. 

The name “Naïve” is used because the algorithm incorporates features in its model that are independent of each other. Any modifications in the value of one feature do not directly impact the value of any other feature of the algorithm. The main advantage of the Naïve Bayes algorithm is that it is a simple yet powerful algorithm.

Ads of upGrad blog

It is based on the probabilistic model where the algorithm can be coded easily, and predictions did quickly in real-time. Hence this algorithm is the typical choice to solve real-world problems as it can be tuned to respond to user requests instantly. But before we dive deep into Naïve Bayes and Gaussian Naïve Bayes, we must know what is meant by conditional probability.

Trending Machine Learning Skills

Conditional Probability Explained

We can understand conditional probability better with an example. When you toss a coin, the probability of getting ahead or a tail is 50%. Similarly, the probability of getting a 4 when you roll dice with faces is 1/6 or 0.16. 

If we take a pack of cards, what is the probability of getting a queen given the condition that it is a spade? Since the condition is already set that it must be a spade, the denominator or the selection set becomes 13. There is only one queen in spades, hence the probability of picking a queen of spade becomes 1/13= 0.07.

The conditional probability of event A given event B means the probability of event A occurring given that event B has already occurred. Mathematically, the conditional probability of A given B can be denoted as P[A|B] = P[A AND B] / P[B]. 

FYI: Free nlp course!

Let us consider a little complex example.  Take a school with a total of 100 students. This population can be demarcated into 4 categories- Students, Teachers, Males and Females. Consider the tabulation given below:

FemaleMaleTotal
Teacher81220
Student324880
Total4050100

Here, what is the conditional probability that a certain resident of the school is a Teacher given the condition that he is a Man. 

To calculate this, you will have to filter the sub-population of 60 men and drill down to the 12 male teachers. 

So, the expected conditional probability P[Teacher | Male] = 12/60 = 0.2

P (Teacher | Male) = P (Teacher ∩ Male) / P(Male) = 12/60 = 0.2

This can be represented as a Teacher(A) and Male(B) divided by Male(B). Similarly, the conditional probability of B given A can also be calculated. The rule that we use for Naïve Bayes can be concluded from the following notations:

P (A | B) = P (A ∩ B) / P(B)

P (B | A) = P (A ∩ B) / P(A)

The Bayes Rule

In the Bayes rule, we go from P (X | Y) that can be found from the training dataset to find P (Y | X). To achieve this, all you need to do is replace A and B with X and Y in the above formulae. For observations, X would be the known variable and Y would be the unknown variable. For each row of the dataset, you must calculate the probability of Y given that X has already occurred. 

But what happens where there are more than 2 categories in Y? We must compute the probability of each Y class to find out the winning one. 

Through Bayes rule, we go from P (X | Y) to find P (Y | X)

Known from training data: P (X | Y) = P (X ∩ Y) / P(Y) 

P (Evidence | Outcome)

Unknown – to be predicted for test data: P (Y | X) = P (X ∩ Y) / P(X) 

P (Outcome | Evidence)

Bayes Rule = P (Y | X) = P (X | Y) * P (Y) / P (X)

The Naïve Bayes

The Bayes rule provides the formula for the probability of Y given condition X. But in the real world, there may be multiple X variables. When you have independent features, the Bayes rule can be extended to the Naïve Bayes rule.  The X’s are independent of each other. The Naïve Bayes formula is more powerful than the Bayes formula

Gaussian Naïve Bayes

So far, we have seen that the X’s are in categories but how to compute probabilities when X is a continuous variable? If we assume that X follows a particular distribution, you can use the probability density function of that distribution to calculate the probability of likelihoods. 

If we assume that X’s follow a Gaussian or normal distribution, we must substitute the probability density of the normal distribution and name it Gaussian Naïve Bayes. To compute this formula, you need the mean and variance of X.

In the above formulae, sigma and mu is the variance and mean of the continuous variable X computed for a given class c of Y. 

Representation for Gaussian Naïve Bayes

The above formula calculated the probabilities for input values for each class through a frequency. We can calculate the mean and standard deviation of x’s for each class for the entire distribution.

This means that along with the probabilities for each class, we must also store the mean and the standard deviation for every input variable for the class. 

mean(x) = 1/n * sum(x)

where n represents the number of instances and x is the value of the input variable in the data.

standard deviation(x) = sqrt(1/n * sum(xi-mean(x)^2 ))

Here square root of the average of differences of each x and the mean of x is calculated where n is the number of instances, sum() is the sum function, sqrt() is the square root function, and xi is a specific x value.

Predictions with the Gaussian Naïve Bayes Model

The Gaussian probability density function can be used to make predictions by substituting the parameters with the new input value of the variable and as a result, the Gaussian function will give an estimate for the new input value’s probability. 

Naïve Bayes Classifier

The Naïve Bayes classifier assumes that the value of one feature is independent of the value of any other feature. Naïve Bayes classifiers need training data to estimate the parameters required for classification. Due to simple design and application, Naïve Bayes classifiers can be suitable in many real-life scenarios.

Popular AI and ML Blogs & Free Courses

Conclusion

Ads of upGrad blog

The Gaussian Naïve Bayes classifier is a quick and simple classifier technique that works very well without too much effort and a good level of accuracy.

If you’re interested to learn more about AI, machine learning, check out IIIT-B & upGrad’s PG Diploma in Machine Learning & AI which is designed for working professionals and offers 450+ hours of rigorous training, 30+ case studies & assignments, IIIT-B Alumni status, 5+ practical hands-on capstone projects & job assistance with top firms.

Learn ML Course from the World’s top Universities. Earn Masters, Executive PGP, or Advanced Certificate Programs to fast-track your career.

 

Profile

Rohan Vats

Blog Author
Software Engineering Manager @ upGrad. Passionate about building large scale web apps with delightful experiences. In pursuit of transforming engineers into leaders.
Get Free Consultation

Select Coursecaret down icon
Selectcaret down icon
By clicking 'Submit' you Agree to  
UpGrad's Terms & Conditions

Our Popular Machine Learning Course

Frequently Asked Questions (FAQs)

1What is a naive bayes algorithm?

Naive bayes is a classic machine learning algorithm. Having its origin in statistics, naive bayes is a simple and a powerful algorithm. Naive bayes is a family of classifiers based on applying a conditional probability analysis. In this analysis, the conditional probability of an event is computed using the probability of each of the individual events constituting the event. Naive bayes classifiers are often found to be extremely effective in practice, especially when the number of dimensions of the feature set is large.

2What are the applications of naive bayes algorithm?

Naive Bayes is used in text classification, document classification and for document indexing. In naive bayes, each possible feature does not have any weight assigned in the pre-processing phase and the weights are later assigned during training as well as recognition phases. The basic assumption of naive bayes algorithm is that features are independent.

3What is Gaussian Naïve Bayes algorithm?

Gaussian Naive Bayes is a probabilistic classification algorithm based on applying Bayes' theorem with strong independence assumptions. In the context of classification, independence refers to the idea that the presence of one value of a feature does not influence the presence of another (unlike independence in probability theory). Naive refers to the use of an assumption that the features of an object are independent of one another. In the context of machine learning, naive Bayes classifiers are known to be highly expressive, scalable, and reasonably accurate, but their performance deteriorates rapidly with the growth of the training set. A number of features contribute to the success of naive Bayes classifiers. Most notably, they do not require any tuning of the parameters of the classification model, they scale well with the size of the training data set, and they can easily handle continuous features.

Explore Free Courses

Suggested Blogs

Top 9 Python Libraries for Machine Learning in 2024
74338
Machine learning is the most algorithm-intense field in computer science. Gone are those days when people had to code all algorithms for machine learn
Read More

by upGrad

19 Feb 2024

Top 15 IoT Interview Questions & Answers 2024 – For Beginners & Experienced
63850
These days, the minute you indulge in any technology-oriented discussion, interview questions on cloud computing come up in some form or the other. Th
Read More

by Kechit Goyal

19 Feb 2024

Data Preprocessing in Machine Learning: 7 Easy Steps To Follow
146799
Summary: In this article, you will learn about data preprocessing in Machine Learning: 7 easy steps to follow. Acquire the dataset Import all the cr
Read More

by Kechit Goyal

18 Feb 2024

Artificial Intelligence Salary in India [For Beginners & Experienced] in 2024
906124
Artificial Intelligence (AI) has been one of the hottest buzzwords in the tech sphere for quite some time now. As Data Science is advancing, both AI a
Read More

by upGrad

18 Feb 2024

24 Exciting IoT Project Ideas & Topics For Beginners 2024 [Latest]
741539
Summary: In this article, you will learn the 24 Exciting IoT Project Ideas & Topics. Take a glimpse at the project ideas listed below. Smart Agr
Read More

by Kechit Goyal

18 Feb 2024

Natural Language Processing (NLP) Projects & Topics For Beginners [2023]
105156
What are Natural Language Processing Projects? NLP project ideas advanced encompass various applications and research areas that leverage computation
Read More

by Pavan Vadapalli

17 Feb 2024

45+ Interesting Machine Learning Project Ideas For Beginners [2024]
322856
Summary: In this Article, you will learn Stock Prices Predictor Sports Predictor Develop A Sentiment Analyzer Enhance Healthcare Prepare ML Algorith
Read More

by Jaideep Khare

16 Feb 2024

AWS Salary in India in 2023 [For Freshers & Experienced]
903456
Summary: In this article, you will learn about AWS Salary in India For Freshers & Experienced. AWS Salary in India INR 6,07,000 per annum AW
Read More

by Pavan Vadapalli

15 Feb 2024

Top 8 Exciting AWS Projects & Ideas For Beginners [2023]
95409
AWS Projects & Topics Looking for AWS project ideas? Then you’ve come to the right place because, in this article, we’ve shared multiple AWS proj
Read More

by Pavan Vadapalli

13 Feb 2024

Schedule 1:1 free counsellingTalk to Career Expert
icon
footer sticky close icon