Lectures on Mathematical Statistics-R. C. Sickles

October 2, 1999

1 Probability Transformation

Problem 1 Given the density f (x_{1}, x_{2}, ..., x_{N}) of (X_{1}, X_{2}, ..., X_{N}), what is the density g (y) of Y = w (X_{1}, X_{2}, ..., X_{N})? Solution 2 Usually we use the following three techniques to find g (y): (1) Distribution function technique (2) Transformation of variables (3) moment generating function technique.

1.1 Distribution Function Technique

If G (y) = Pr (Y ≤ y) = Pr [w (X_{1}, X_{2}, ⋯, X_{N}) ≤ y], then g (y) = dG (y) /dy. Example 3

f (x) = \{ \begin{matrix}6x (1 − x) & 0 < x < 1 \\ 0 & otherwise \end{matrix}

If y = x^{3},

G (y) = Pr (Y ≤ y) = Pr (X^{3} ≤ y) = Pr (X ≤ y^{1/3})
= ∫_{0}^{y^{1/3}} 6x (1 − x) dx = 3y^{2/3} − 2y

⇒

g (y) = \{ \begin{matrix}2 (y^{−1/3} − 1) & 0 < y < 1 \\ 0 & otherwise \end{matrix}

Theorem 4 If Y = |X|, then

g (y) = \{ \begin{matrix}f (y) + f (−y) & y > 0 \\ 0 & otherwise \end{matrix}

Proof. For y > 0, G (y) = Pr (Y ≤ y) = Pr (|X| ≤ y) = Pr (−y ≤ X ≤ y). Thus

G (y) = F (y) − F (−y)

g (y) = f (y) + f (−y) for y > 0

Example 5

f (x_{1}, x_{2}) = \{ \begin{matrix}6e^{−3x_{1}−2x_{2}} & for x_{1} > 0 and x_{2} > 0 \\ 0 & otherwise \end{matrix}

Let Y = X_{1} + X_{2}, then

G (y) = ∫_{0}^{y} ∫_{0}^{y−x_{2}} 6e^{−3x_{1}−2x_{2}}dx_{1}dx_{2} = 1 + 2e^{−3y} − 3e^{−2y}

g (y) = dG (y) /dy = 6 (e^{−2y} − e^{−3y}) for y > 0

1.2 Transformation of Variable Technique

Theorem 6 Let f (x) be the density function of random variable X. y = u (x) is monotonic in x for f (x) ≠ 0 and is differentiable. Then there is a unique one-to-one mapping from y to x given by x = w (y) and

g (y) = \{ \begin{matrix}f [w (y)] · |w′ (y)| & for u′ (x) ≠ 0 \\ 0 & otherwise \end{matrix}

Proof. First assume that y = u (x) is monotonic increasing,

Pr (a < Y < b) = Pr [w (a) < X < w (b)]

= ∫_{w(a)}^{w(b)} f (x) dx = ∫_{a}^{b} f (w (y)) w′ (y) dy

Thus we get

g (y) = f (w (y)) w′ (y)

Second assume that y = u (x) is monotonic decreasing,

Pr (a < Y < b) = Pr [w (b) < X < w (a)]

= ∫_{w(b)}^{w(a)} f (x) dx = − ∫_{a}^{b} f (w (y)) w′ (y) dy

Thus we get

g (y) = −f (w (y)) w′ (y)

But when y = u (x) is monotonic decreasing, −w′ (y) > 0. So that

g (y) = f (w (y)) |w′ (y)|

Example 7 Ex. X ∼ N (0, 1) and Z = X^{2}. In this example the transformation is not monotonic in x, we will do it is two steps. First, if y = |x|, then

g (y) = \{ \begin{matrix}\frac{2}{\sqrt{2π}} e^{− \frac{1}{2} y^{2}} & for y > 0 \\ 0 & otherwise \end{matrix}

Second, z = y^{2} and

\frac{dy}{dz} = \frac{1}{2}z^{− \frac{1}{2}}

h (z) = \frac{2}{\sqrt{2π}} e^{− \frac{1}{2} z} |\frac{1}{2}z^{\frac{1}{2}} | = \frac{1}{\sqrt{2π}} e^{− \frac{1}{2} z}z^{\frac{1}{2}} for z > 0

Note Γ (1/2) = \sqrt{π}, where

Γ (α) = ∫_{0}^{∞} y^{α−1}e^{−y}dy

Thus

h (z) = \frac{1}{2^{1/2}Γ (1/2)}e^{− \frac{1}{2} z}z^{\frac{1}{2}} for z > 0

is the density function of χ_{1}^{2}.

Example 8

f (x_{1}, x_{2}) = \{ \begin{matrix}1 & for 0 < x_{1}, x_{2} < 1 \\ 0 & otherwise \end{matrix}

y = x_{1} + x_{2}

First we find the joint density of y and x_{2}.

x_{1} = y − x_{2} and ∂x_{1}/∂y = 1

Thus

g (y, x_{2}) = 1 · |1| = \{ \begin{matrix}1 & for 0 < x_{2} < 1 and x_{2} < y < x_{2} + 1 \\ 0 & otherwise \end{matrix}

Next integrate over x_{2}.

h (y) = \{ \begin{matrix}∫_{−∞}^{0} g (y, x_{2}) dx_{2} = 0 & y ≤ 0 \\ ∫_{0}^{y} 1dx_{2} = y & 0 < y < 1 \\ ∫_{y−1}^{1} 1dx_{2} = 2 − y & 1 < y < 2 \\ ∫_{2}^{∞} g (y, x_{2}) dx_{2} = 0 & y ≥ 2 \end{matrix}

Example 9 t-Distribution: Let

T = \frac{X}{\sqrt{u/k}}

where X ∼ N (0, 1), u ∼ χ_{k}^{2} and X and u are independent. What is h (t)? Theorem 10

h (t) = \frac{1}{\sqrt{kπ}} · \frac{[(k − 1) /2]!}{[(k − 2) /2]!} (1 + \frac{t^{2}}{k} )^{−(k+1)/2} for − ∞ < t < ∞

Proof. Since u and X are independent, their joint distribution is

f (x, u) = \frac{1}{\sqrt{2π}} e^{− \frac{1}{2} x^{2}} · \frac{1}{[(k − 2) /2]!2^{k/2}} u^{(k−2)/2}e^{− \frac{1}{2} u}

Let

t = \frac{x}{\sqrt{v/k}} and v = u

Substitute x and u and then integrate out x,

\frac{∂x}{∂t} = \sqrt{\frac{v}{k}}

\frac{∂x}{∂v} = \frac{t}{2k} (\frac{v}{k} )^{−1/2}

\frac{∂u}{∂t} = 0

\frac{∂u}{∂v} = 1

Thus the Jacobian of the transformation from t and v to x and u is

| \begin{matrix}\frac{∂x}{∂t} & \frac{∂x}{∂v} \\ \frac{∂u}{∂t} & \frac{∂u}{∂v} \end{matrix} | = \sqrt{\frac{v}{k}}

So that

g (t, v) = \frac{1}{\sqrt{2π}} e^{− \frac{vt^{2}}{2k}} · \frac{1}{[(k − 2) /2]!2^{k/2}} u^{(k−2)/2}e^{− \frac{1}{2} v} · \sqrt{\frac{v}{k}}

and

h (t) = ∫_{0}^{∞} g (t, v) dv = \frac{1}{\sqrt{kπ}} · \frac{1}{[(k − 2) /2]!}A

where

A = \frac{1}{2} ∫_{0}^{∞} e^{−(1/2)(t^{2}/k+1)v} (\frac{v}{2} )^{(k−1)/2} dv

Let

α = \frac{k − 1}{2} and z = \frac{1}{2} (1 + \frac{t^{2}}{k} ) v

Then

\frac{v}{2} = (1 + \frac{t^{2}}{k} )^{−1} z and (\frac{v}{2} )^{α} = (1 + \frac{t^{2}}{k} )^{−α} z^{α}

Also

d (\frac{v}{2} ) = (1 + \frac{t^{2}}{k} )^{−1} dz

With these substitutions

A = (1 + \frac{t^{2}}{k} )^{1(k+1)/2} ∫_{0}^{∞} z^{α}e^{−z}dz = (1 + \frac{t^{2}}{k} )^{1(k+1)/2} [(k − 1) /2]!

since Γ (α + 1) ≡ α!. thus

∫_{0}^{∞} z^{α}e^{−z}dz = Γ (α + 1) = [\frac{k − 1}{2}]!

so that

h (t) = \frac{1}{\sqrt{kπ}} · \frac{[(k − 1) /2]!}{[(k − 2) /2]!} (1 + \frac{t^{2}}{k} )^{−(k+1)/2} for − ∞ < t < ∞

Remark 11 This density function is symmetric about t = 0 and all those odd moments, if they exist, vanish. Even moments are given by

μ2r^{′} = k^{r} \frac{Γ (r + 1/2) Γ (k/2 − r)}{Γ (1/2) Γ (k/2)}

for r = 0, 1, ..., m < k/2. Moments of order greater or equal to k do not exist. Thus E (T) = 0 for k > 1 and v (T) = k/ (k − 2) for k > 2.