Explore Courses
Liverpool Business SchoolLiverpool Business SchoolMBA by Liverpool Business School
  • 18 Months
Bestseller
Golden Gate UniversityGolden Gate UniversityMBA (Master of Business Administration)
  • 15 Months
Popular
O.P.Jindal Global UniversityO.P.Jindal Global UniversityMaster of Business Administration (MBA)
  • 12 Months
New
Birla Institute of Management Technology Birla Institute of Management Technology Post Graduate Diploma in Management (BIMTECH)
  • 24 Months
Liverpool John Moores UniversityLiverpool John Moores UniversityMS in Data Science
  • 18 Months
Popular
IIIT BangaloreIIIT BangalorePost Graduate Programme in Data Science & AI (Executive)
  • 12 Months
Bestseller
Golden Gate UniversityGolden Gate UniversityDBA in Emerging Technologies with concentration in Generative AI
  • 3 Years
upGradupGradData Science Bootcamp with AI
  • 6 Months
New
University of MarylandIIIT BangalorePost Graduate Certificate in Data Science & AI (Executive)
  • 8-8.5 Months
upGradupGradData Science Bootcamp with AI
  • 6 months
Popular
upGrad KnowledgeHutupGrad KnowledgeHutData Engineer Bootcamp
  • Self-Paced
upGradupGradCertificate Course in Business Analytics & Consulting in association with PwC India
  • 06 Months
OP Jindal Global UniversityOP Jindal Global UniversityMaster of Design in User Experience Design
  • 12 Months
Popular
WoolfWoolfMaster of Science in Computer Science
  • 18 Months
New
Jindal Global UniversityJindal Global UniversityMaster of Design in User Experience
  • 12 Months
New
Rushford, GenevaRushford Business SchoolDBA Doctorate in Technology (Computer Science)
  • 36 Months
IIIT BangaloreIIIT BangaloreCloud Computing and DevOps Program (Executive)
  • 8 Months
New
upGrad KnowledgeHutupGrad KnowledgeHutAWS Solutions Architect Certification
  • 32 Hours
upGradupGradFull Stack Software Development Bootcamp
  • 6 Months
Popular
upGradupGradUI/UX Bootcamp
  • 3 Months
upGradupGradCloud Computing Bootcamp
  • 7.5 Months
Golden Gate University Golden Gate University Doctor of Business Administration in Digital Leadership
  • 36 Months
New
Jindal Global UniversityJindal Global UniversityMaster of Design in User Experience
  • 12 Months
New
Golden Gate University Golden Gate University Doctor of Business Administration (DBA)
  • 36 Months
Bestseller
Ecole Supérieure de Gestion et Commerce International ParisEcole Supérieure de Gestion et Commerce International ParisDoctorate of Business Administration (DBA)
  • 36 Months
Rushford, GenevaRushford Business SchoolDoctorate of Business Administration (DBA)
  • 36 Months
KnowledgeHut upGradKnowledgeHut upGradSAFe® 6.0 Certified ScrumMaster (SSM) Training
  • Self-Paced
KnowledgeHut upGradKnowledgeHut upGradPMP® certification
  • Self-Paced
IIM KozhikodeIIM KozhikodeProfessional Certification in HR Management and Analytics
  • 6 Months
Bestseller
Duke CEDuke CEPost Graduate Certificate in Product Management
  • 4-8 Months
Bestseller
upGrad KnowledgeHutupGrad KnowledgeHutLeading SAFe® 6.0 Certification
  • 16 Hours
Popular
upGrad KnowledgeHutupGrad KnowledgeHutCertified ScrumMaster®(CSM) Training
  • 16 Hours
Bestseller
PwCupGrad CampusCertification Program in Financial Modelling & Analysis in association with PwC India
  • 4 Months
upGrad KnowledgeHutupGrad KnowledgeHutSAFe® 6.0 POPM Certification
  • 16 Hours
O.P.Jindal Global UniversityO.P.Jindal Global UniversityMaster of Science in Artificial Intelligence and Data Science
  • 12 Months
Bestseller
Liverpool John Moores University Liverpool John Moores University MS in Machine Learning & AI
  • 18 Months
Popular
Golden Gate UniversityGolden Gate UniversityDBA in Emerging Technologies with concentration in Generative AI
  • 3 Years
IIIT BangaloreIIIT BangaloreExecutive Post Graduate Programme in Machine Learning & AI
  • 13 Months
Bestseller
IIITBIIITBExecutive Program in Generative AI for Leaders
  • 4 Months
upGradupGradAdvanced Certificate Program in GenerativeAI
  • 4 Months
New
IIIT BangaloreIIIT BangalorePost Graduate Certificate in Machine Learning & Deep Learning (Executive)
  • 8 Months
Bestseller
Jindal Global UniversityJindal Global UniversityMaster of Design in User Experience
  • 12 Months
New
Liverpool Business SchoolLiverpool Business SchoolMBA with Marketing Concentration
  • 18 Months
Bestseller
Golden Gate UniversityGolden Gate UniversityMBA with Marketing Concentration
  • 15 Months
Popular
MICAMICAAdvanced Certificate in Digital Marketing and Communication
  • 6 Months
Bestseller
MICAMICAAdvanced Certificate in Brand Communication Management
  • 5 Months
Popular
upGradupGradDigital Marketing Accelerator Program
  • 05 Months
Jindal Global Law SchoolJindal Global Law SchoolLL.M. in Corporate & Financial Law
  • 12 Months
Bestseller
Jindal Global Law SchoolJindal Global Law SchoolLL.M. in AI and Emerging Technologies (Blended Learning Program)
  • 12 Months
Jindal Global Law SchoolJindal Global Law SchoolLL.M. in Intellectual Property & Technology Law
  • 12 Months
Jindal Global Law SchoolJindal Global Law SchoolLL.M. in Dispute Resolution
  • 12 Months
upGradupGradContract Law Certificate Program
  • Self paced
New
ESGCI, ParisESGCI, ParisDoctorate of Business Administration (DBA) from ESGCI, Paris
  • 36 Months
Golden Gate University Golden Gate University Doctor of Business Administration From Golden Gate University, San Francisco
  • 36 Months
Rushford Business SchoolRushford Business SchoolDoctor of Business Administration from Rushford Business School, Switzerland)
  • 36 Months
Edgewood CollegeEdgewood CollegeDoctorate of Business Administration from Edgewood College
  • 24 Months
Golden Gate UniversityGolden Gate UniversityDBA in Emerging Technologies with Concentration in Generative AI
  • 36 Months
Golden Gate University Golden Gate University DBA in Digital Leadership from Golden Gate University, San Francisco
  • 36 Months
Liverpool Business SchoolLiverpool Business SchoolMBA by Liverpool Business School
  • 18 Months
Bestseller
Golden Gate UniversityGolden Gate UniversityMBA (Master of Business Administration)
  • 15 Months
Popular
O.P.Jindal Global UniversityO.P.Jindal Global UniversityMaster of Business Administration (MBA)
  • 12 Months
New
Deakin Business School and Institute of Management Technology, GhaziabadDeakin Business School and IMT, GhaziabadMBA (Master of Business Administration)
  • 12 Months
Liverpool John Moores UniversityLiverpool John Moores UniversityMS in Data Science
  • 18 Months
Bestseller
O.P.Jindal Global UniversityO.P.Jindal Global UniversityMaster of Science in Artificial Intelligence and Data Science
  • 12 Months
Bestseller
IIIT BangaloreIIIT BangalorePost Graduate Programme in Data Science (Executive)
  • 12 Months
Bestseller
O.P.Jindal Global UniversityO.P.Jindal Global UniversityO.P.Jindal Global University
  • 12 Months
WoolfWoolfMaster of Science in Computer Science
  • 18 Months
New
Liverpool John Moores University Liverpool John Moores University MS in Machine Learning & AI
  • 18 Months
Popular
Golden Gate UniversityGolden Gate UniversityDBA in Emerging Technologies with concentration in Generative AI
  • 3 Years
Rushford, GenevaRushford Business SchoolDoctorate of Business Administration (AI/ML)
  • 36 Months
Ecole Supérieure de Gestion et Commerce International ParisEcole Supérieure de Gestion et Commerce International ParisDBA Specialisation in AI & ML
  • 36 Months
Golden Gate University Golden Gate University Doctor of Business Administration (DBA)
  • 36 Months
Bestseller
Ecole Supérieure de Gestion et Commerce International ParisEcole Supérieure de Gestion et Commerce International ParisDoctorate of Business Administration (DBA)
  • 36 Months
Rushford, GenevaRushford Business SchoolDoctorate of Business Administration (DBA)
  • 36 Months
Liverpool Business SchoolLiverpool Business SchoolMBA with Marketing Concentration
  • 18 Months
Bestseller
Golden Gate UniversityGolden Gate UniversityMBA with Marketing Concentration
  • 15 Months
Popular
Jindal Global Law SchoolJindal Global Law SchoolLL.M. in Corporate & Financial Law
  • 12 Months
Bestseller
Jindal Global Law SchoolJindal Global Law SchoolLL.M. in Intellectual Property & Technology Law
  • 12 Months
Jindal Global Law SchoolJindal Global Law SchoolLL.M. in Dispute Resolution
  • 12 Months
IIITBIIITBExecutive Program in Generative AI for Leaders
  • 4 Months
New
IIIT BangaloreIIIT BangaloreExecutive Post Graduate Programme in Machine Learning & AI
  • 13 Months
Bestseller
upGradupGradData Science Bootcamp with AI
  • 6 Months
New
upGradupGradAdvanced Certificate Program in GenerativeAI
  • 4 Months
New
KnowledgeHut upGradKnowledgeHut upGradSAFe® 6.0 Certified ScrumMaster (SSM) Training
  • Self-Paced
upGrad KnowledgeHutupGrad KnowledgeHutCertified ScrumMaster®(CSM) Training
  • 16 Hours
upGrad KnowledgeHutupGrad KnowledgeHutLeading SAFe® 6.0 Certification
  • 16 Hours
KnowledgeHut upGradKnowledgeHut upGradPMP® certification
  • Self-Paced
upGrad KnowledgeHutupGrad KnowledgeHutAWS Solutions Architect Certification
  • 32 Hours
upGrad KnowledgeHutupGrad KnowledgeHutAzure Administrator Certification (AZ-104)
  • 24 Hours
KnowledgeHut upGradKnowledgeHut upGradAWS Cloud Practioner Essentials Certification
  • 1 Week
KnowledgeHut upGradKnowledgeHut upGradAzure Data Engineering Training (DP-203)
  • 1 Week
MICAMICAAdvanced Certificate in Digital Marketing and Communication
  • 6 Months
Bestseller
MICAMICAAdvanced Certificate in Brand Communication Management
  • 5 Months
Popular
IIM KozhikodeIIM KozhikodeProfessional Certification in HR Management and Analytics
  • 6 Months
Bestseller
Duke CEDuke CEPost Graduate Certificate in Product Management
  • 4-8 Months
Bestseller
Loyola Institute of Business Administration (LIBA)Loyola Institute of Business Administration (LIBA)Executive PG Programme in Human Resource Management
  • 11 Months
Popular
Goa Institute of ManagementGoa Institute of ManagementExecutive PG Program in Healthcare Management
  • 11 Months
IMT GhaziabadIMT GhaziabadAdvanced General Management Program
  • 11 Months
Golden Gate UniversityGolden Gate UniversityProfessional Certificate in Global Business Management
  • 6-8 Months
upGradupGradContract Law Certificate Program
  • Self paced
New
IU, GermanyIU, GermanyMaster of Business Administration (90 ECTS)
  • 18 Months
Bestseller
IU, GermanyIU, GermanyMaster in International Management (120 ECTS)
  • 24 Months
Popular
IU, GermanyIU, GermanyB.Sc. Computer Science (180 ECTS)
  • 36 Months
Clark UniversityClark UniversityMaster of Business Administration
  • 23 Months
New
Golden Gate UniversityGolden Gate UniversityMaster of Business Administration
  • 20 Months
Clark University, USClark University, USMS in Project Management
  • 20 Months
New
Edgewood CollegeEdgewood CollegeMaster of Business Administration
  • 23 Months
The American Business SchoolThe American Business SchoolMBA with specialization
  • 23 Months
New
Aivancity ParisAivancity ParisMSc Artificial Intelligence Engineering
  • 24 Months
Aivancity ParisAivancity ParisMSc Data Engineering
  • 24 Months
The American Business SchoolThe American Business SchoolMBA with specialization
  • 23 Months
New
Aivancity ParisAivancity ParisMSc Artificial Intelligence Engineering
  • 24 Months
Aivancity ParisAivancity ParisMSc Data Engineering
  • 24 Months
upGradupGradData Science Bootcamp with AI
  • 6 Months
Popular
upGrad KnowledgeHutupGrad KnowledgeHutData Engineer Bootcamp
  • Self-Paced
upGradupGradFull Stack Software Development Bootcamp
  • 6 Months
Bestseller
KnowledgeHut upGradKnowledgeHut upGradBackend Development Bootcamp
  • Self-Paced
upGradupGradUI/UX Bootcamp
  • 3 Months
upGradupGradCloud Computing Bootcamp
  • 7.5 Months
PwCupGrad CampusCertification Program in Financial Modelling & Analysis in association with PwC India
  • 5 Months
upGrad KnowledgeHutupGrad KnowledgeHutSAFe® 6.0 POPM Certification
  • 16 Hours
upGradupGradDigital Marketing Accelerator Program
  • 05 Months
upGradupGradAdvanced Certificate Program in GenerativeAI
  • 4 Months
New
upGradupGradData Science Bootcamp with AI
  • 6 Months
Popular
upGradupGradFull Stack Software Development Bootcamp
  • 6 Months
Bestseller
upGradupGradUI/UX Bootcamp
  • 3 Months
PwCupGrad CampusCertification Program in Financial Modelling & Analysis in association with PwC India
  • 4 Months
upGradupGradCertificate Course in Business Analytics & Consulting in association with PwC India
  • 06 Months
upGradupGradDigital Marketing Accelerator Program
  • 05 Months

What is Bayesian Thinking ? Introduction and Theorem

Updated on 30 November, 2022

6.41K+ views
8 min read

A statistical theorem given by the English statistician and philosopher Thomas Bayes in the 1700s continues to be a guiding light for scientists and analysts across the world. Today, Bayesian thinking finds application in medicine, science, technology, and several other disciplines and continues to influence our worldview and resultant actions strongly. 

Thomas’ Bayes idea was strikingly simple. According to Bayes, the probability of a hypothesis being true depends on two conditions: how reasonable it is based on what we already know (the prior knowledge) and how well it fits new evidence. Thus, Bayesian thinking differs from traditional hypothesis testing in that the former includes the prior knowledge before jumping to conclusions.

With the preliminary introduction in mind, let us dive into a bit more detail about Bayesian statistics.

Bayesian Statistics

In simple terms, Bayesian statistics apply probabilities to statistical problems to update prior beliefs in light of the evidence of new data. The probability expresses a degree of belief in a specific event.

The degree of belief may be based on previous knowledge about the event based on personal assumptions or results of prior experiments. Bayesian statistics use the Bayes’ Theorem to compute probabilities. The Bayes’ Theorem, in turn, describes the conditional probability of an event based on new evidence and prior information related to the event.

With that in mind, let us brush up on the fundamental concept of conditional probability before we understand Bayes’ Theorem in depth.

Conditional Probability

Conditional probability can be defined as the likelihood of an event or outcome based on the occurrence of a previous event or outcome. It is calculated by multiplying the probability of the prior event by the probability of the subsequent or conditional event. 

Let’s take a look at an example to understand the concept better.

  • Event A is that a family planning an outing will go on a picnic. There is an 80% chance that the family will go on the picnic.
  • Event B is that it will rain on the day the family goes out on a picnic. The weather forecast says that there is a 60% chance of precipitation on the picnic day.
  • Hence, the probability (P) that the family goes on the picnic and it rains is calculated as follows:

P (Picnic and rain) = P (Rain | Picnic) P (Picnic) = (0.60) * (0.80) = 0.48

In the above example, conditional probability looks at the two events A and B in relationship with one another, that is, the probability that the family does go to the picnic and it also rains on the same day.

Hence, conditional probability differs from unconditional probability because the latter refers to the likelihood of occurrence of an event regardless of whether any other event or events have taken place or any other conditions are present.

The formula for conditional probability

The formula for conditional probability comes from the probability multiplication rule :

P (A and B) or P (A U B) = P ( B given A) or P (B | A) * P (A) 

In the above equation, P (A and B) is the joint probability, referring to the likelihood of two or more events occurring simultaneously. It is also written as P (A,B).

Here’s how to deduce the conditional probability equation from the multiplication rule:

Step 1: Write down the multiplication rule.

P (A and B) = P (B | A) * P (A) 

Step 2: Divide both sides of the equation by P (A).

P (A and B) / P (A) = P (B | A) * P (A) / P (A)

Step 3: Cancel P (A) on the right side of the equation.

P (A and B) / P (A) = P (B | A)

Step 4: Rewrite the equation.

P (A and B) = P (B | A) / P (A)

Thus, the formula for conditional probability is given as:

P (A and B) = P (B | A) / P (A)

Bayes’ Theorem

Using Bayes’ Theorem, we can update our beliefs and convictions based on new and relevant pieces of evidence. For instance, if we are trying to figure out the probability of a given person having cancer, we would generally assume it to be the percentage of the population that has cancer. However, if we introduce extra evidence, such as the person in question is a regular smoker, we can update our perception (and hence the probability) since the probability of having cancer is higher if an individual is a smoker. Hence, we utilize both our prior knowledge and the additional evidence to improve our estimations.

The formula for Bayes’ Theorem

Source

The above equation is the Bayes’ rule. Now, let us look into the stepwise derivation of the Bayes’ Theorem equation.

Step 1: Consider two events, A and B. A is the event whose probability we want to calculate and B is the additional evidence that is related to A.

Step 2: Write down the relationship between the joint probability and conditional probability of events A and B.

P (A,B) = P (A | B) * P(B) = P (B,A) = P (B | A) * P(A)

Step 3: Set the two conditional probability terms equal to each other.

P (A | B) * P(B) = P (B | A) * P(A)

Step 4: Divide both sides of the equation by P (B).

P (A | B) * P(B) / P (B) = P (B | A) * P(A) / P (B)

Step 5: Cancel P (B) on the left side of the equation.

P (A | B) = P (B | A) * P(A) / P (B)

Thus, we get the formula of Bayes’ Theorem as follows:

P (A | B) = P (B | A) * P(A) / P (B)

Understanding the terms in the Bayes’ Theorem equation

P (A | B) = P (B | A) * P(A) / P (B)

  • P (A | B) is called the posterior probability or the probability we are trying to estimate. Based on the previous example, the posterior probability would be the probability of the person having cancer, given that the person is a regular smoker.
  • P (B | A) is called the likelihood, referring to the probability of detecting the additional evidence, given our initial hypothesis. In the above example, the likelihood is the probability of the person being a smoker, given that the person has cancer.
  • P (A) is the prior probability or the probability of our hypothesis without any additional evidence or information. In the above example, the prior probability is the probability of having cancer.
  • P (B) is the marginal likelihood or the total probability of observing the evidence. In the context of the above example, the marginal likelihood is the probability of being a smoker.

A Simple Example To Understand Bayes’ Theorem

Using some hypothetical numbers in the previous example, we will see the effect of applying the Bayes’ Theorem.

Suppose the probability of having cancer is 0.06, that is, 6% of the people have cancer. Now, say that the probability of being a smoker is 0.20 or 20% of people are smokers, and 30% of people with cancer are smokers. So, P (Smoker | Cancer) = 0.30.

Initially, the probability of having cancer is simply 0.06 (prior). But using the new evidence, we can calculate P (Cancer | Smoker) = P ((Smoker | Cancer) * P (Cancer)) / P (Smoker) = (0.30*0.06) / (0.20) = 0.09.

Learn data science courses online from the World’s top Universities. Earn Executive PG Programs, Advanced Certificate Programs, or Masters Programs to fast-track your career.

Way Forward: Master the Concepts of Statistics for a Career in Data Science or Machine Learning

upGrad’s higher EdTech learning platform has impacted over 500,000 working professionals worldwide with its plethora of courses and immersive learning experiences. With a 40,000+ learner base spread across 85+ countries, upGrad’s industry-relevant courses are guaranteed to advance your career in your field of choice.

Master of Science in Data Science is an 18-months course imparting key skills in Statistics, Predictive Analysis, Machine Learning, Big Data Analytics, Data Visualization, etc.

Program Highlights:

  • Master’s Degree from Liverpool John Moores University and Executive PGP from IIIT Bangalore
  • 500+ hours of content, 60+ case studies and projects, 20+ live sessions, 14+ programming languages and tools
  • Industry networking, doubt resolution sessions, and learning support

Advanced Certificate Program in Machine Learning and Deep Learning is a rigorous 6-months course with peer networking opportunities, hands-on projects, industry mentorship, and 360-degree career assistance.

Program Highlights:

  • Prestigious recognition from IIIT Bangalore
  • 240+ hours of content, 5+ case studies, and projects, 24+ live sessions, coverage of 12 programming languages, tools, and libraries
  • 1:8 group coaching sessions and 1:1 mentorship sessions with industry experts

Conclusion 

Bayesian thinking underpins several areas of human thinking, inquiry, and belief, even though most of us are unaware of it. From cancer screening and global warming to monetary policy and risk assessment and insurance, Bayesian thinking is fundamental. Even the famous British mathematician Alan Turing is believed to have employed the Bayesian approach to crack the German Enigma Code during the Second World War.

Sign up with upGrad and further your knowledge of key statistical concepts and more!

Frequently Asked Questions (FAQs)

1. How can Bayes’ Theorem be used practically?

Bayes’ Theorem has plenty of applications in real life. Here are some instances:
1. To determine the accuracy of a medical test result by considering the general accuracy of the test and the likelihood of any given person having a particular disease.
2. In finance, Bayes’ Theorem can be applied to rate the risk of lending money to prospective borrowers.
3. In artificial intelligence, Bayesian statistics can be used to calculate the next step of a robot when the already accomplished step is given.

2. How many terms are required for building a Bayes model?

Three terms are required to build a Bayes model. The three required terms are two unconditional probability and one conditional probability.

3. What is the difference between Bayes’ theorem and conditional probability?

Conditional probability is the probability of occurrence of an event (say, A) based on the occurrence of some other event (say, B). It is expressed as:
P(A|B) = P(A ⋂ B)/P(B)
Bayes’ Theorem is based on the conditional probability of events and includes two conditional probabilities (say, for events A and B). It is expressed as:
P(A|B) = P(B|A) * P(A)/P(B)