🐡

Binomial distribution

2023/12/16に公開

The binomial distribution is a probability distribution characterized by a discrete random variable X, with parameters n \in \mathbb{N} and p \in [0, 1].

\mathrm{Bin}(X=x|n, p) = \binom{n}{x} p^x q^{n-x}

where q = 1 - p.


Mean and variance are calculated by moment generating function M(\theta) = E[e^{\theta X}].

\begin{align*} M(\theta) &= E[e^{\theta X}] \\ &= \sum_{x} \mathrm{Bin}(x|n, p) e^{\theta x} \\ &= \sum_{x} \binom{n}{x} p^x q^{n-x} e^{\theta x} \\ &= \sum_{x} \binom{n}{x} (p e^{\theta})^x q^{n-x} \\ &= (p e^{\theta} + q)^n \\ &\quad \because (a+b)^n = \sum_x \binom{n}{x} a^x b^{n-x} \end{align*}

We need first and second order derivative of M(\theta) .

\begin{align*} M'(\theta) &= n(p e^{\theta} + q)^{n-1} p e^{\theta} = np(p e^{\theta} + q)^{n-1} e^{\theta} \\ M''(\theta) &= np (n-1) (p e^{\theta} + q)^{n-2} pe^{\theta} e^{\theta} + np(p e^{\theta} + q)^{n-1} e^{\theta} \\ &= npe^{\theta} \{ (n-1) (p e^{\theta} + q)^{n-2} pe^{\theta} + (p e^{\theta} + q)^{n-1} \} \end{align*}

Mean is

\begin{align*} E[X] &= M'(0) = np(p + q)^{n-1} = np \end{align*}

Variance is

\begin{align*} V[X] &= E[X^2] - E[X]^2 = M''(0) - M'(0)^2 \\ &= np \{ (n-1) (p + q)^{n-2} p + (p + q)^{n-1} \} - (np)^2 \\ &= np \{ (n-1) p + 1 \} - (np)^2 \\ &= np \{ np - p + 1 \} - (np)^2 \\ &= \{ (np)^2 - np^2 + np \} - (np)^2 \\ &= - np^2 + np \\ &= np(1 - p) \\ &= npq \end{align*}

Moment Generating Function

Moment generating function M(\theta) is E[e^{\theta X}].
This is useful to obtain k-order moments E[X^k].

Because,

\begin{align*} M(\theta) &= E[e^{\theta X}] \\ &= E\left[ 1 + \frac{(\theta X)}{1!} + \frac{(\theta X)^2}{2!} + \cdots \right] , \quad \left( \because e^x = 1 + \frac{x}{1!} + \frac{x^2}{2!} + \cdots \right) \\ &= 1 + \frac{\theta}{1!} E[X] + \frac{\theta^2}{2!} E[X^2] + \cdots \end{align*}

Here, we can obtain E[X] = M'(0) and E[X^2] = M''(0).

Discussion