Bayes’s theorem is the fundamental concept behind Bayesian statistics and there are several machine learning and deep learning algorithm depends upon Bayes’s theorem like Naive Bayes, Gaussian Naive Bayes, Bayesian Network etc click here for more categorizations of algorithm So in this tutorial we will learn about basic concept of probability, conditional probability and Bayes’s theorem.
A probability is number between 0 and 1 (including both) that represents a degree of belief in a fact or prediction. The value 1 represent certainly that the fact is true or that prediction will come true. The value 0 represent certainly that the fact is false ie 1==>True and 0==>False.
Intermediate value represent degree of certainty like the value 0.5 represent 50% means the predicted value is likely to happen or not. For example tossing a coin will land head is 1/2 or 0.5 or 50% .
Conditional probability is the probability which is depends on some background information. For example i want to know the probability of a particular person will have a heart attack in the next year and i found some data that the rate of heart attack on particular state is 0.4% but we know that there are some factors exists by which person can decrease the probability of heart attack and for that particular person the heart attack rate is 0.2% which is lower than the state average. This is conditional probability because it based on number of factor that depends upon number of factors that make person’s condition.
The usual notations of conditional probability is p(A|B) which is probability of a given that B is true. A represents the prediction that particular person will have heart attack next year and B is set of conditions by which he reduced his chance.
Conjoint probability is the fancy way to say the probability the two things are true. If i write the p(A and B) means that the probability of A and B are both true.
For example if i toss coins and A means the first coin lands Head and B means the second coin lands Head also the p(A)=p(B)=0.5 and sure enough P(A and B)=P(A)p(B)=0.25 but this formula only works because in this case A and B are independent, that is knowing of outcomes of first event does not change the probability of second or p(B|A)=p(B).
Here is different example where the event are not independent. Suppose that A means that it rains today and B means that is rains tomorrow. If i know that it rained today, it is more likely that it will rain tomorrow so p(B|A)>p(B)
probability of conjunction is: p(A and B) =p(A)p(B|A)—————-(1)
if we interchange them
probability of conjunction is: p(B and A) =p(B)p(A|B)——————(2)
Conjunction is commutative ie, p(A and B) = p(B and A) for any event A and B.
combine both equations (1) and (2)=> p(B)p(A|B)=p(A)p(B|A)—————(3)
From equation (3) it means that when there are two ways to compute the conjunctions if you have p(A) you multiply by the conditional probability p(B|A) and if you have p(B) you have to multiply by p(A|B).
On dividing p(B) of equation(1) we get,
This is called Bayes’s Theorem. It tells that how often A happens given that B happens written p(A|B), when we know how often B happens given that A happens written p(B|A) and how likely A and B are their own way.
- p(A|B) is “probability of A given B”
- P(A) is “probability of A”
- p(B|A) is “probability of B given A”
- p(B) is “probability of B”