Unit II: Moments and Moment Generating Functions

1. Expectation of a Random Variable

Concept

The Expected Value $E[X]$, often denoted by $\mu$, represents the theoretical average of a random variable if an experiment is repeated an infinite number of times. It is the center of gravity of the probability distribution.

Formulas

1. Discrete Case:

$$ E[X] = \sum_{i} x_i P(X=x_i) $$ Multiply each value by its probability and sum them up.

1. Continuous Case:

$$ E[X] = \int_{-\infty}^{\infty} x f(x) dx $$ Integrate the value $x$ weighted by the density function $f(x)$.

Expectation of a Function $g(X)$

If you want the average of a function of $X$ (like $X^2$ or $2X+1$):

• Discrete: $E[g(X)] = \sum g(x) P(x)$
• Continuous: $E[g(X)] = \int g(x) f(x) dx$

Properties of Expectation

Let $X$ and $Y$ be random variables, and $a, b$ be constants.

1. Linearity: $E[aX + b] = aE[X] + b$
2. Additivity: $E[X + Y] = E[X] + E[Y]$ (Always true).
3. Product Rule: $E[XY] = E[X]E[Y]$ ONLY IF $X$ and $Y$ are independent.

• Property: $Var(aX + b) = a^2 Var(X)$. (Adding a constant doesn't change spread; scaling by $a$ scales variance by $a^2$).

Numerical Example

Problem: A fair die is rolled. Find the Expected Value $E[X]$.

• Possible outcomes: $x = {1, 2, 3, 4, 5, 6}$
• Probabilities: $P(X=x) = 1/6$ for each.
• Calculation:

$$ E[X] = (1 imes 1/6) + (2 imes 1/6) + (3 imes 1/6) + (4 imes 1/6) + (5 imes 1/6) + (6 imes 1/6) $$ $$ E[X] = rac{1+2+3+4+5+6}{6} = rac{21}{6} = 3.5 $$ (Interpretation: If you roll a die many times, the average result will be 3.5).