Sample Space and Probability#
Element of a Probabilistic Model#
Sample Space: The set of all possible outcomes of an experiment. It is denoted by \(\Omega\).
Event: A subset of the sample space. For example, \(A \subseteq \Omega\).
Probability Law: A function that assigns a non-negative number to an event. \(P(A)\) is the probability of event \(A\).
Probability Axioms#
(Non-negativity): \(P(A) \geq 0\) for all events \(A\).
(Additivity): If \(A\) and \(B\) are disjoint events, then \(P(A \cup B) = P(A) + P(B)\).
(Normalization): \(P(\Omega) = 1\).
Properties of Probability Laws#
If \(A \subseteq B\), then \(P(A) \leq P(B)\).
\(P(A \cup B) = P(A) + P(B) - P(A \cap B)\).
\(P(A \cup B) \leq P(A) + P(B)\).
\(P(A \cup B \cup C) = P(A) + P(A^c \cap B) + P(A^c \cap B^c \cap C)\).
Conditional Probability#
Consider two events \(A\) and \(B\) with \(P(B) > 0\). The conditional probability of \(A\) given \(B\) is defined as:
For a fixed event \(B\), the function \(P(\cdot|B)\) is a probability law. Hence, it satisfies the axioms of probability.
(Non-negativity): \(P(A|B) \geq 0\).
(Additivity): \(P(A \cup C|B) = P(A|B) + P(C|B)\) if \(A \cap C = \emptyset\).
(Normalization): \(P(\Omega|B) = 1\).
In addition, sometimes we know \(P(A|B)\) and \(P(B)\), and we want to find \(P(A \cap B)\). This can be done by rearranging the definition of conditional probability: