Lectures on Mathematical Statistics-R. C. Sickles Revised September, 1999

September 21, 1999

1. Probabilities in Finite Sample Space: Elements of Combinational Algebra

Before we examine parametric distributions, and in particular special discrete distributions, we will review methods to compute the probability of an event in a situation where the classical definition of probability applies, i.e. where there are n equally likely elementary events. To do this we need to compute the total number of the elementary events that are favarable to the event in question. To help in these calculations we will develope some counting formulas based on two principles:

1. If an event A can occur in a total of m ways and an event B can occur in a total of n ways, then A ∪ B can occur in a total of m + n ways, provided that A and B can not occur simultaneously.

2. If A can occur in a total of m ways and an event B in a total of n ways, then the event A ∩ B can occur in mn ways.

Example 1. Let A be the drawing of a spade from a deck of 52 cards and B be the drawing of a heart. Each event can be done in 13 ways. Therefore the event of drawing either a spade or a heart can be done in 13+13=26 ways. On the other hand, two cards can be drawn from the deck such that one is a spade and the other a heart in 13×13 = 169 ways, since with each spade 13 hearts can be drawn and there are 13 spades in all.

Remark 1. It is evident that these two principles can be generalized to more than two events. Thus if three mutually events A, B and C can occur in k, m and n ways respectively, then the event A or B or C can occur in k + m + n ways and the event A and B and C in kmn ways.

1.1. Permutations:. We will use the second principle to count the number of arrangements that can be made with a set of n objects. Consider the letters {a, b, c} and ask how many arrangements of these letters can be made. To begin with, any of the three letters can occupy the first position, so that this can be done in 3 ways. Then once the first position is filled, the second position can be filled in 2 ways, and finally there remains only 1 way for the third position to be filled. Hence, the three letters can be arranged in 3 × 2 × 1 = 6 ways in all. Written out, these are

abc acb bac bca cab cba

By induction then, n objects can be arranged in n (n − 1) (n − 2) ⋯ 2 × 1 ≡ n! different ways.

Example 2. Six objects can be arranged in 6!=720 different ways. Each arrangement is commonly called a permutation.

Example 3. A person wishes to go round trip from Chicago to New York to London. She wants to go by air between Chicago and New York, and by sea between New York and London. If there are 6 airlines to choose from between Chicago and New York and 3 shiplines between New York and London, how many different ways can she make the round trip without using the same airline or shipline twice on the same trip? Solution: There are 6 different ways that the woman can get to New York and once there, there are 3 ways of getting to London. On the return trip, there remain 2 ways of getting back to New York, and finally there are 5 unused airlines to choose from in returning to Chicago. Therefore, there are 6×3×2×5 = 180 different ways that she can make the round trip from Chicago to New York to London.

Example 4. Ex. Given the digits 1, 2, 3, 4 and 5, how many four-digit numbers can be formed

(a) If there is no repitition?

(b) If there can be repitition?

(c) If the number must be even and no repitition?

(d) If the digits 2 and 3 must appear in the number and in that order with no repitition?

Solution:

(a) The first digit can be chosen in 5 ways, the second in 4 ways, and so on, so that the number of 4-digit numbers that can be formed is 5! = 120.

(b) The first digit can be chosen in 5 ways, as can also the second, and so on. Therefore, 5^{4} = 625 numbers can be formed in all.

(c) The last digit must be either 2 or 4, this can occur in w ways. After the last place is filled, the remaining places can be filled in 4, 3, 2 ways respectively, since no digit can be used more that once. Hence the number is 4 × 3 × 2 × 2 = 48.

(d) Because 2 and 3 must appear together, then two digits become a single digit. Therefore, the problem is reduced to the number of permutations of four difits, or 4!=24.

We can also enumerate the number of permutations that can be made from n objects if only if r objects are to be used in any permutation. Reasoning as before, the first place can be filled in n ways, the second in n − 1, and so on. When we come to the rth place, there remain n − (r − 1) objects from which to choose, hence the number of permutations of n objects taken r at a time is n (n − 1) ⋯ (n − r + 1). The symbol P_{r}^{n} is usually used to denote this number, namely

P_{r}^{n} = n (n − 1) ⋯ (n − r + 1)

1.2. Combinations:. With permutations, order counts in that ‘ab’ is different from ‘ba’. In many situations, however, order does not matter and we will need counting rules to answer a question such as: ”In how many different ways can r objects be selected from n objects?” Let’s call this number x for the moment. Now the number of permutations that we can have of n objects taken r at a time is P_{r}^{n}. But permutations taken order into account, and this we do not want to do. Therefore, each selection of r different objects will have r! permutations. Hence

P_{r}^{n} = xr!

and solving for x we have

x = \frac{P_{r}^{n}}{r!} = \frac{n!}{(n − r)!r!}

Usually this number is denoted by C (n, r) or \binom{n}{r} and is called the number of combinations of n objects taken r at a time.

Example 5. How many different ways can a committee of 5 selected from 9 people? C (9, 5) = 9!/ (5!4!) = 126 different ways.

Let’s now use these counting formulas to assign probabilities in games of chance where the classical definition of probability applies.

Example 6. In drawing 2 cards from a well-shuffled deck, what is the probability that both will be hearts?

Solution: Two cards can be drawn from 52 cards in C (52, 2) different ways, so that the number of elementary events with equal probability will be C (52, 2). The number of events favorable to both cards being hearts will be C (13, 2), since this is the number of ways that 2 hearts can be drawn from a total of 13. Hence, the required probability is

Pr (both cards hearts) = C (13, 2) /C (52, 2) = 3/51

Example 7. Same facts. What is the probability that one card is a heart and one a spade?

Solution: As above, the number of possible outcomes is C (52, 2) and these are assigned equal probability. Since there are 13 hearts, a heart can be drawn in C (13, 1) ways, and similarly for spades. Therefore, the number of favorable outcomes is C (13, 1) × C (13, 1), and the required probability is given by

Pr (H and S) = C (13, 1) C (13, 1) /C (52, 2) = 12/102

Example 8. In drawing 5 cards, what is the probability that they are either all hearts or all spades?

Solution: The number of equally probable elementary events will be C (52, 5). Now 5 hearts can be drawn in C (13, 5) ways, and the same is true for spades. Because 5 hearts or 5 spades are mutually exclusive events, the probabilities add, and so the required probability is

Pr (5H and 5S) = {C (13, 5) + C (13, 5)}/C (52, 5) = 0.000495

Example 9. If 5 cards are drawn and 4 of them are either hearts or diamonds, what is the probability that 2 of them are hearts?

Solution: This is a conditional probability, and the sample space is reduced to the 26 hearts and diamonds. The number of equally likely elementary events is then C (26, 4). The 2 hearts can be drawn in C (13, 2) ways and for each of these the remaining diamonds can also be drawn in C (13, 2). Hence there are C (13, 2) × C (13, 2) favorable events in all, and the required probabiltiy is

Pr (2H|4 are H or D) = C (13, 2) C (13, 2) /C (26, 4) = 0.407

Example 10. Five cards are drawn, what is the probability that none of them is a ace?

Solution: There are C (52, 5) possible outcomes, of which C (48, 5) are favorable. Hence the probability is

Pr (no aces) = C (48, 5) /C (52, 5) = 0.659

Remark 2.

( \begin{matrix}n \\ r \end{matrix} ) = ( \begin{matrix}n \\ n − r \end{matrix} ) for 1 ≤ r ≤ n

( \begin{matrix}n + 1 \\ r \end{matrix} ) = ( \begin{matrix}n \\ r − 1 \end{matrix} ) + ( \begin{matrix}n \\ r \end{matrix} ) This is called Pascal’s Rule

Remark 3. We can intepret Pascal’s Rule it in a number of different ways.

1. One is that it’s the number of ways that n objects can be divided into two groups, one having r items and one having n − r items. If we follow this line of reasoning then consider the number of ways that n objects can be divided into three groups with n_{1}, n_{2}, n_{3} objects respectively with n_{1} + n_{2} + n_{3} = n. First divide n items into 2 groups containing n_{1} and n_{2} +n_{3} items. The second group

can be divided into 2 groups with n_{2} and n_{3} objects in ( \begin{matrix}n_{2} + n_{3} \\ n_{2} \end{matrix} ) ways. Thus

the 3 groups can be formed in

( \begin{matrix}n \\ n_{1} \end{matrix} ) ( \begin{matrix}n_{2} + n_{3} \\ n_{2} \end{matrix} ) = \frac{n!}{n_{1}! (n_{2} + n_{3})!} \frac{(n_{2} + n_{3})!}{n_{2}!n_{3}!} = \frac{n!}{n_{1}!n_{2}!n_{3}!}

ways. This argument can easily be extended to find the number of ways that n objects can be divided into k groups containing n_{1}, n_{2}, ..., n_{k} objects, respectively. This number is

\frac{n!}{n_{1}!n_{2}! ⋯ n_{k}!}

2. Another is that it can be viewed as the number of permutations of n objects when n_{1} are alike and of on kind, n_{2} are alike and of a second kind, and so on. To see that this is so, consider the case of two groups with n_{1} items in the first group, n_{2} in the second, and n_{1} + n_{2} = n. Let the number of permutations be x. Then for each permutation, if we temporarily distinguish among the objects within the first group and within the second group, there would be n_{1}!n_{2}! permutations in all. Therefore we will have x · n_{1}!n_{2}! = n! since we must have n! permutations in all if all n objects are distinguishable. From this it follows that

x = \frac{n!}{n_{1}!n_{2}!}