The binomial distribution is a probability distribution characterized by a discrete random variable X, with parameters n \in \mathbb{N} and p \in [0, 1].
\mathrm{Bin}(X=x|n, p) = \binom{n}{x} p^x q^{n-x}
where q = 1 - p.
Mean and variance are calculated by moment generating function M(\theta) = E[e^{\theta X}].
\begin{align*}
M(\theta) &= E[e^{\theta X}] \\
&= \sum_{x} \mathrm{Bin}(x|n, p) e^{\theta x} \\
&= \sum_{x} \binom{n}{x} p^x q^{n-x} e^{\theta x} \\
&= \sum_{x} \binom{n}{x} (p e^{\theta})^x q^{n-x} \\
&= (p e^{\theta} + q)^n \\
&\quad \because (a+b)^n = \sum_x \binom{n}{x} a^x b^{n-x}
\end{align*}
We need first and second order derivative of M(\theta) .
\begin{align*}
M'(\theta) &= n(p e^{\theta} + q)^{n-1} p e^{\theta} = np(p e^{\theta} + q)^{n-1} e^{\theta} \\
M''(\theta) &= np (n-1) (p e^{\theta} + q)^{n-2} pe^{\theta} e^{\theta} + np(p e^{\theta} + q)^{n-1} e^{\theta} \\
&= npe^{\theta} \{ (n-1) (p e^{\theta} + q)^{n-2} pe^{\theta} + (p e^{\theta} + q)^{n-1} \}
\end{align*}
Mean is
\begin{align*}
E[X] &= M'(0) = np(p + q)^{n-1} = np
\end{align*}
Variance is
\begin{align*}
V[X] &= E[X^2] - E[X]^2 = M''(0) - M'(0)^2 \\
&= np \{ (n-1) (p + q)^{n-2} p + (p + q)^{n-1} \} - (np)^2 \\
&= np \{ (n-1) p + 1 \} - (np)^2 \\
&= np \{ np - p + 1 \} - (np)^2 \\
&= \{ (np)^2 - np^2 + np \} - (np)^2 \\
&= - np^2 + np \\
&= np(1 - p) \\
&= npq
\end{align*}
Moment Generating Function
Moment generating function M(\theta) is E[e^{\theta X}].
This is useful to obtain k-order moments E[X^k].
Because,
\begin{align*}
M(\theta) &= E[e^{\theta X}] \\
&= E\left[ 1 + \frac{(\theta X)}{1!} + \frac{(\theta X)^2}{2!} + \cdots \right] ,
\quad \left( \because e^x = 1 + \frac{x}{1!} + \frac{x^2}{2!} + \cdots \right) \\
&= 1 + \frac{\theta}{1!} E[X] + \frac{\theta^2}{2!} E[X^2] + \cdots
\end{align*}
Here, we can obtain E[X] = M'(0) and E[X^2] = M''(0).
Discussion