Joint and Conditional Probability Study Notes for Civil Engineering

Combinatorial Analysis

To have an understanding in combinations and permutations you must first have a basic understanding in probability.

A probability is a numerical measure of the likelihood of the event. Probability is established on a scale from 0 to 1. A rare even has a probability close to 0; a very common event has a probability close to 1.

In order to solve and understand problems pertaining to probability you must know some vocabulary:

• An experiment, such as rolling a die or tossing a coin, is a set of trials designed to study a physical occurrence.
• An outcome is any result of a particular experiment. For example, the possible outcomes for flipping a coin are heads or tails.
• A sample space is a list of all the possible outcomes of an experiment.
• An event is a subset of the sample space. For example, getting heads is an event.

Counting Principles

1. Theorem (The basic principle of counting): If the set E contains n elements and the set F contains m elements, there are nm ways in which we can choose, first, an element of E and then an element of F.
2. Theorem (The generalized basic principle of counting): If r experiments that are to be performed are such that the first one may result in any of n₁ possible outcomes, and if for each of these n₁ possible outcomes there are n₂ possible outcomes of the second experiment, and if for each of the possible outcomes of the first two experiments there are n₃ possible outcomes of the third experiment, and if …, then there is a total of n­₁.n₂…n_r, possible outcomes of the r experiments.
3. Theorem: A set with n elements has 2ⁿ subsets.
4. Tree diagrams

Permutations

1. Permutation: n!
   The number of permutations of n things taken r at a time: 
2. Theorem: The number of distinguishable permutations of n objects of k different types, where n₁ are alike, n₂ are alike, …, n_k are alike and n=n₁+n₂+…+n_k, is 

Combinations

1. Combination: The number of combinations of n things taken r at a time:  (combinatorial coefficient; binomial coefficient)
2. Binomial theorem: 
3. Multinomial expansion: In the expansion of (x₁+x₂+…+x_k)ⁿ, the coefficient of the term , n₁+n₂+…+n_k=n, is . Therefore, (x₁+x₂+…+x_k)ⁿ = .
   Note that the sum is taken over all nonnegative integers n₁, n₂,…,n_k such that n₁+n₂+…+n_k=n.

The Number of Integer Solutions of Equations

1. There are  distinct positive integer-valued vectors (x₁,x₂,…,x_y) satisfying x₁+x₂+…+x_r = n, x_i>0, i=1,2,…,r.
2. There are  distinct nonnegative integer-valued vectors (x₁,x₂,…,x_y) satisfying x₁+x₂+…+x_r = n.

Axioms of Probability

Sample Space and Events

1. Experiment: An experiment (strictly speaking, a random experiment) is an operation which can result in two or more ways. The possible results of the experiment are known as outcomes of the experiment. These outcomes are known in advance. We cannot predict which of these possible outcomes will appear when the experiment is done.
2. Sample Space: The set containing all the possible outcomes of an experiment as its element is known as sample space. It is usually denoted by S.
3. Event: An event is a subset of the sample space S.

Illustration 1:

Let us consider the experiment of tossing a coin. This experiment has two possible outcomes: heads (H) or tails (T)

• sample space (S) = {H, T}

We can define one or more events based on this experiment. Let us define the two events A and B as:

A: heads appears

B: tails appears

It is easily seen that set A (corresponding to event A) contains outcomes that are favorable to event A and set B contains outcomes favorable to event B. Recalling that n (A) represents the number of elements in set A, we can observe that

n (A) = number of outcomes favorable to event A

n (B) = number of outcomes favorable to event B

n (S) = number of possible outcomes

Here, in this example, n (A) = 1, n (B) = 1, and n (S) = 2.

• Set theory concepts: set, element, roster method, rule method, subset, null set (empty set).
• Complement: The complement of an event A with respect to S is the subset of all elements of S that are not in A. We denote the complement of A by the symbol A’ (A^c).
• Intersection: The intersection of two events A and B, denoted by the symbol A∩B, is the event containing all elements that are common to A and B.
  Two events A and B are mutually exclusive, or disjoint, if A∩B = φ that is, if A and B have no elements in common.
• The union of the two events A and B, denoted by the symbol A∪B, is the event containing all the elements that belong to A or B or both.

• Venn diagram:
• Sample space of an experiment: All possible outcomes (points)
• Events: subsets of the sample space
  Impossible events (impossibility): Φ; sure events (certainty): S.

• DeMorgan’s laws:

Axioms of Probability

1. Probability axioms:
   (1) 0≤P(A)≤1;
   (2) P(S)=1;
   (3) P(A₁∪A₂∪…) = P(A₁)+P(A₂)+… If {A₁, A₂, …} is a sequence of mutually exclusive events.
2. Equally likely outcomes: the probabilities of the single-element events are all equal.
   A number of sample events are said to be equally likely if there is no reason for one event to occur in preference to any other event.

Basic Theorems

(1) 0≤P(A)≤1;

(2) 

(3) Complementary events: 

(4) P(A∪B) = P(A) + P(B) – P(A∩B): inclusion-exclusion principle

(5) If A₁, A₂, A_n is a partition of sample space S, then

(6) if a and a’ are complementary events, then P(A) + P(A’)=1.

Conditional Probability and Independence

Conditional Probability

1. Conditional probability: 

Consider two events A and B defined on a sample space S. The probability of occurrence of event A given that event B has already occurred is known as the conditional probability of A relative to B.

P(A/B) = P(A given B) = probability of occurrence of A assuming that B has occurred

It implies that the outcomes favorable to B become the possible outcomes and hence outcomes favorable to P(A/B) are outcomes common to A and B.

Let n(B)=a and n(A∩B)=b. Suppose that the event B occurs. Then there are exactly a sample points and these points can be considered to be the sample space for the other event A. The event A in this sample space consists of b sample points common to A and B.

Therefore, the probability of A in this sample space = b/a.

Thus the probability of A under the assumption that event B has occurred is:

and similarly 

The above result is known as conditional probability theorem.

2. If in an experiment the events A and B can both occur, then P(A∩B) = P(A)P(B|A) = P(B)P(A|B).

P(A∩B∩C) = P(A∩B)P(C|A∩B) = P(A)P(B|A)P(C|A∩B),

The multiplication rule: P(A₁∩A₂ ∩…∩ A_n) = P(A₁)P(A₂|A₁)P(A₃|A₁∩A₂) … P(A_n|A₁∩A₂∩…∩A_n-1).

3. Partition: Let {B₁, B₂, …, B_n} be a set of nonempty subsets of the sample space S of an experiment. If the events B₁, B₂, …, B_n are mutually exclusive and B₁∪B₂∪…∪B_n = S, the set {B₁, B₂, …, B_n} is called a partition of S.

4. Theorem of total probability: If B₁, B₂, … is a partition of S, and A is any event, then 

Total Probability Theorem: The probability that one of several mutually exclusive events A₁, A₂, .... A_n will happen is the sum of the probabilities of the separate events. i.e.

P(A₁∪A₂∪A_n) = P(A₁)+P(A₂)+.... + P(A_n)

5. Bayes’ Theorem: If B₁, B₂, … is a partition of S, and A is any event, then 

Independence

1. Independent events: If A, B are independent events ↔ P(A∩B) = P(A)P(B).
2. Theorem: If A and B are independent, then  are independent.
3. The events A, B, and C are called independent if P(A∩B) = P(A)P(B), P(A∩C) = P(A)P(C), P(B∩C) = P(B)P(C), P(A∩B∩C) = P(A)P(B)P(C). If A, B, and C are independent events, we say that {A, B, C} is an independent set of events.
4. The set of events {A₁, A₂, …, A_n} is called independent if for every subset  of {A₁, A₂,…, A_n}, .

Important Results:

Following results are easily derived from the definition of independent events.

1. A and B are independent if
   P(B/A) = P(B) and P(A/B) = P(A)
2. If A and B are independent, then P (A∩B) = P (A) P (B)
3. A set of events A₁, A₂,... A_n are said to be pair-wise independent if
   P(A_i n A_j) = P(A_i) P(A_j) ∀i ≠j.
4. The events A₁, A₂, ... A_n are mutually independent (or simply independent) if and only if the multiplication rule:
   P(A₁∩A₂∩ .... A_k) = P(A₁) P(A₂) .... P(A_k) ... (1)
   holds for every t triples of events, k = 2, 3, .... n. If (1) holds for k = 2 and may not hold for k = 3, 4, ....... n then events A₁, A₂, ... A_n are said to be pair wise independent. Thus mutually independent events are pair wise independent but converse is not true.
5. If A and B are mutually independent events such that P (A) ≠ 0 and P (B) ≠ 0, then the events A and B have at least one common sample point (i.e. they cannot be mutually exclusive). Or in general, mutually exclusive events are dependent events.

Use the following information for Example 1-4:

A box contains 2 coins. One of them is a fair (F) coin and another one is a biased (UF) coin which always gives head as outcome when toss i.e. which has head on both sides. A coin is selected at random and it is tossed 'n' times. If all first 'n' outcomes results in a head,

Then answer the following question using the convention (notations) given below:-

The probability that (n+1)^th toss also results in a head given as P(n+1,H)

If first 'n' toss results in a head than the probability of selection of fair coin under above condition is denoted by P(n,F)

If first 'n' toss results in a head than the probability of selection of unfair coin under above condition is represented by P(n,UF)

Example 1:

If P(n,UF) = 128/129 then n is :-

A. 6 B. 7

C. 8 D. 9

Correct Answer: 2

Solution:

Event-1: section of coin

Event-2: tossing of the selected coin to ‘n’ times

P(n,F) – Is the probability of selection of fair coin if first n time head comes

So, using Baye’s theorem:

Let the symbol be as given below

P(n,H) = The prob. That (n+1)^th outcome is also head

Example 2:

P(100,F) is

A.  B. 

C.  D. 

Correct Answer: 4

Solution:

Example 3:

If P(n,H) = 33/34 then n is :-

A. 3 B. 4

C. 5 D. 6

Correct Answer: 2

Solution:

Example 4:

A. 2 B. 1/2

C. 1 D. 

Correct Answer: 3

Solution:

Example 5:

A biased coin has P probability for heads. It is tossed until tail appears for the first time. If the probability that no. of tosses required is even is 2/7 then P is

A. 4/7 B. 2/5

C. 3/5 D. 3/7

Correct Answer: 2

Solution:

Let X be the tosses required

P(X=r) = P^r-1(1-P)

E = Event of no of tosses to be even

P(E) = P[(X=2)U(X=4)------

= (1-P)P+P³(1-P)+P⁵(1-P)--------

= (1-P)P/(1-P²) = P/1+P

P/1+P = 2/7 P = 2/5 Hence (b)

Use the following information for Example 6-12:

There are three pots and 4 coins. All there coins are to be distributed into these pots where any pot can contain any number of coins.

Example 6:

In how many ways can the coins be distributed if all pots and coins are different respectively?

A. 3⁴ B. 4³

^{C. 4}P₃ D. None

Correct Answer: 1

Solution:

As all pots have equal chances for any given coin

hence

3×3×3×3 = 3 (for 4 coins)

Example 7:

In how many ways can they be distributed if all pots and coins are respectively identical?

A. 4 B. 8

C. 1 D. 2

Correct Answer: 1

Solution:

As only 4 cases are possible

(4,0,0) (2,1,1) (3,1,0) (2,2,0)

therefore 4 ways

Example 8:

In how many ways can these coins be distributed if all coins are identical and all pots different

A. 16 B. 81

C. 15 D. 17

Correct Answer: 3

Solution:

It is the case when r objects are to be distributed into n things such that each receives none, one or more

Hence possibilities = n+r-¹C_r-1

= 4+3-¹C₂ = ⁶C₂ = 15

Example 9:

In how many ways coins can be distributed if coins different but all pots are identical?

A. 21 B. 14

C. 42 D. None

Correct Answer: 2

Solution: Since pots are identical, these will be 4 cases but since coin are diff selection of coin should be considered

cases (4,0,0) (3,1,0) (2,2,0) (2,1,1)

case I → No of reactions ⁴C₄ = 1

case II → ⁴C₃×¹C₁ = 4

case III → ⁴C₂ײC₂/2! = 3

case IV → ⁴C₂ײC_1×¹C₁ /2! = 6

Total = 6+3+4+1 = 14

Example 10:

In how many ways can the distribution be done so that no one (pot) receives no coin considering coins different but pots identical.

A. 6 B. 16

B. 42 D. 21

Correct Answer: 1

Solution:

The only possible case is (1,1,2)

Hence possible distributions

⁴C₂ײC₁×¹C₁/2! = 6

Example 11:

No. of ways of distribution for no pot to be empty considering coins to be identical and pots to be different are

A. 6 B. 3

C. 9 D. 27

Correct Answer: 2

Solution:

Since all coins are identical and pots different only case is (1,1,2)

But the pot that contain 2 coins can be selected in ³C₁ ways

therefore

Required ways = ³C₁×1 = 3

Example 12:

Distribution in case of all coins to be identical and 2 pots to be identical is in:

A. 1 B. 10

C. 9 D. 11

Correct Answer: 3

Solution:

(4,0,0) → 2 ways

4 put in 1 diff pot or 2 identical pot

(3,1,0) → 3 ways

Distinct pot can have 2 balls or remain empty

(2,1,1) → 2 ways

Distinct pot can have 2 balls or 1 ball

Total ways = 2+3+2+2 = 9

Example 13:

The following can be said about the graph shown:

A. Represents |xy| = 1

B. x|y| = 1

B. |y| = x+1 cuts the graph at 2 distinct points

D. Represents |x|y = 1

Correct Answer: 3

Solution:

x remains always +ve and y changes sign hence the appropriate curve will be x|y| = 1

Now x=1/|y|

Let |y|= k Then from eqn of line

k²-k-1 = 0

 
Hence we get two distinct points of intersection (c) is correct

Example 14:

A tray containing 15 apples contains 4 rotten apples. The apples are selected at random one by one and examined, examined apples are not put in the tray again. Find the probability that ninth one examined is last spoiled?

A. 11/195 B. 12/195

B. 8/195 D. 16/195

Correct Answer: 3

Solution:

We need to get 3 spoiled apples in first 8 outcomes ------(I)

Let A be the event of getting 9 apples as 4th spoiled

Required P = P((I) ∩ A)

Hence P = ⁴C₃×¹¹C₅/¹⁵C₈×1/7 = 8/195

Example 15:

Given that the sum of 2 no-negative integers quantities is 200 the probability that their product is not less than 3/4 times their greatest product value is:

A. 99/200 B. 101/200

C. 87/100 D. 1/2

Correct Answer: 2

Solution:

Let x and y be the 2 numbers

x+y = 200

xy max = 10000

xy ≥ 3/4X1000 ⇒ xy ≥ 7500

⇒ x(200-x) ≥ 7500

⇒ x2-200x + 7500 ≤ 0

50 ≤ x ≤ 150

so to variable ways

Hence P = 101/200

Example 16:

Let A,B,C be three arbitrary events defined on a sample space S and

P(A) + P(B) + P(C) = P1

P(A∩B) + P(B∩C) + P(C∩A) = P2

P(A∩B∩C) = P3

Then probability of exactly one event occurrence is

A. P1-P2+P3

B. P1-P2+2P3

B. P1-2P2+3P3

D. P1-2P2+P3

Correct Answer: 3

Solution:

P(exactly one) = P(A)+P(B)+P(C)-2P(A∩B)-2P(B∩C)-2P(A∩C)+3P(A∩B∩C)

= P1-2P2+3P3