The Binomial Random Variable

Dated Oct 2, 2017; last modified on Thu, 02 Sep 2021

\(X\) is a binomial random variable if it takes the values \(0, 1, 2, …, n\) and

$$ \mathbb{P}\{X = k\} = { n \choose k } \cdot p^k \cdot (1 - p)^{n-k} $$

Sanity Check: Do the probabilities sum to 1?

$$ \sum_{k=0}^{n} \mathbb{P}\{X = k\} = \sum_{k=0}^{n} { n \choose k } p^k (1 - p)^{n-k} = \left( p + (1 - p) \right)^n = 1 $$

I totally didn’t understand how we got to \(\left( p + (1 - p) \right)^n\). In my notes, I simply noted “Calculus Theorem” and that was it. ლ(ಠ_ಠ ლ)

From Wikipedia :

$$ (x + y)^n = {n \choose 0} x^n y^0 + {n \choose 1} x^{n-1} y^1 + {n \choose 2} x^{n-1} y^2 + … + {n \choose n - 1} x^1 y^{n-1} + {n \choose n} x^0 y^n $$

where \(n \ge 0\) is an integer, and each \( {n \choose k} \) is a positive integer known as a binomial coefficient. \( {n \choose 0} \) is defined as equal to 1.

So much for recalling stuff from freshman math…

The Expectation of the Binomial Random Variable

Naïve Solution

Using the definition of expectation:

$$ \mathbb{E}[X] = \sum_{k=0}^{n} \mathbb{P\{X = k\}} k = \sum_{k=0}^{n} { n \choose k } p^k (1 - p)^{n-k} \cdot k $$

Because \( { n \choose k } = \frac{n!}{k! (n-k)!} \), we can simplify further.

Also, note that when \(k = 0\), the summation term is \(0\), and we can therefore exclude it from the summation.

$$ = \sum_{k=1}^{n} \frac{n!}{(k-1)! (n-k)!} p^k (1-p)^{n-k} $$

Let’s massage the summation such that we can invoke the binomial formula…

$$ = n \cdot \sum_{k=1}^{n} \frac{(n - 1)!}{(k-1)! (n-k)!} p^k (1-p)^{n-k} $$ $$ = np \cdot \sum_{k=1}^{n} \frac{(n - 1)!}{(k-1)! (n-k)!} p^{k-1} (1-p)^{n-k} $$

We’re gonna do what is called a pro gamer move. Change of variables. Let \(i = k - 1\), equivalently \( k = i + 1\)

$$ = np \cdot \sum_{i=0}^{n-1} \frac{(n - 1)!}{i! (n - i - 1)!} p^i (1-p)^{n-i-1} $$ $$ = np \cdot \sum_{i=0}^{n-1} \frac{(n - 1)!}{i! ((n - 1) - i)!} p^i (1-p)^{((n-1)-i)} $$ $$ = np \sum_{i=0}^{n-1} {n - 1 \choose i} p^i (1-p)^{((n-1) - i)} $$

And now applying the binomial formula…

$$ = np (p + (1 - p))^{n-1} = np $$

Tbh, I didn’t see that coming. Too many tricks and eurekas.

Clever Solution

Where \(X_i = 1\) if \(i\)-th toss is heads, and \(X_i = 0\) otherwise:

$$ \mathbb{E}[X] = \mathbb{E}[X_1 + X_2 + … + X_n] $$

Not quite certain why this is so. Maybe it’s because the \(X_i\)’s are disjoint?

$$ = \mathbb{E}[X_1] + \mathbb{E}[X_2] + …. + \mathbb{E}[X_n] $$

Linearity of Expectation is the property that for any random variables \(X\) and \(Y\), and a constant \(a\),

$$ \mathbb{E}[X + Y] = \mathbb{E}[X] + \mathbb{E}[Y] $$ $$ \mathbb{E}[aX] = a \mathbb{E}[X] $$

whenever the right-hand side is well defined (finite?). It doesn’t matter whether \(X\) and \(Y\) are independent.

$$ = n \left( p \cdot 1 + (1-p) \cdot 0 \right) = np $$

References

  1. The Binomial Theorem. en.wikipedia.org .
  2. Expected Value: Basic Properties. en.wikipedia.org .