Generating functions
Generating functions characterize probability distributions.
Probability generating function
\begin{align*}
G_X(\theta) &= E[\theta^X] \\
&= \theta^0 p(0) + \theta^1 p(1) + \dots + \theta^\infty p(\infty) \\
&= \sum_{x=\set{1, 2, \dots, \infty}} \theta^x p(X=x)
\end{align*}
where X \in \mathbb{N}^+ is a random variable, |\theta| \le 1.
This function generates the probability distribution as p(0) = G_X(0), p(1) = G_X'(0), p(2) = \frac{1}{2} G_X''(0), \dots
In general,
\begin{align*}
p(X=x) = \frac{1}{x!} \frac{d^x}{d\theta^x} G_X(0)
\end{align*}
Moment generating function
\begin{align*}
M_X(\theta) &= E[e^{\theta X}] \\
&= E\left[ 1 + \frac{\theta}{1!}X^1 + \dots + \frac{\theta^k}{k!}X^k + \dots \right] \\
&= 1 + \frac{\theta}{1!} E[X^1] + \dots + \frac{\theta^k}{k!}E[X^k] + \dots \\
\end{align*}
under the condition \exist h > 0 , \forall \theta \lt |h| , M_X(\theta) exists.
This function generates moments as E[X] = M_X'(0) , E[X^2] = M_X''(0) .
In general,
\begin{align*}
E[X^k] = M_X^{(k)}(0)
\end{align*}
Characteristic function
\begin{align*}
\varphi_X(\theta) &= E[e^{i \theta X}] \\
&= E[\cos(\theta X) + i \sin(\theta X)]
\end{align*}
The characteristic function always exists, because
\varphi_X(\theta) = M_X(i \theta) = G_X(e^{i \theta}) and |e^{i \theta}| = |\cos(\theta) + i \sin(\theta)| \le 1.
In general,
\begin{align*}
E[X^k] = \frac{1}{i^k} \varphi_X^{(k)}(0)
\end{align*}
Discussion