Doc

Discrete Statistical Distributions

docs/tutorial:stats:discrete

Overview

Discrete random variables take on only a countable number of values. The commonly used distributions are included in SciPy and described in this document. Each discrete distribution can take one extra integer parameter: $L .$ The relationship between the general distribution $p$ and the standard distribution $p_{0}$ is

p (x) = p_{0} (x - L)

which allows for shifting of the input. When a distribution generator is initialized, the discrete distribution can either specify the beginning and ending (integer) values $a$ and $b$ which must be such that

p_{0} (x) = 0 x < a or x > b

in which case, it is assumed that the pdf function is specified on the integers $a + mk \leq b$ where $k$ is a non-negative integer ( $0, 1, 2, \dots$ ) and $m$ is a positive integer multiplier. Alternatively, the two lists $x_{k}$ and $p (x_{k})$ can be provided directly in which case a dictionary is set up internally to evaluate probabilities and generate random variates.

Probability Mass Function (PMF)

The probability mass function of a random variable X is defined as the probability that the random variable takes on a particular value.

p (x_{k}) = P [X = x_{k}]

This is also sometimes called the probability density function, although technically

f (x) = k \sum p (x_{k}) δ (x - x_{k})

is the probability density function for a discrete distribution ^[1] .

[1]

Note that we will be using $p$ to represent the probability mass function and a parameter (a probability). The usage should be obvious from context.

Cumulative Distribution Function (CDF)

The cumulative distribution function is

F (x) = P [X \leq x] = x_{k} \leq x \sum p (x_{k})

and is also useful to be able to compute. Note that

F (x_{k}) - F (x_{k - 1}) = p (x_{k})

Survival Function

The survival function is just

S (x) = 1 - F (x) = P [X > k]

the probability that the random variable is strictly larger than $k$ .

Percent Point Function (Inverse CDF)

The percent point function is the inverse of the cumulative distribution function and is

G (q) = F^{- 1} (q)

for discrete distributions, this must be modified for cases where there is no $x_{k}$ such that $F (x_{k}) = q .$ In these cases we choose $G (q)$ to be the smallest value $x_{k} = G (q)$ for which $F (x_{k}) \geq q$ . If $q = 0$ then we define $G (0) = a - 1$ . This definition allows random variates to be defined in the same way as with continuous rv's using the inverse cdf on a uniform distribution to generate random variates.

Inverse survival function

The inverse survival function is the inverse of the survival function

Z (α) = S^{- 1} (α) = G (1 - α)

and is thus the smallest non-negative integer $k$ for which $F (k) \geq 1 - α$ or the smallest non-negative integer $k$ for which $S (k) \leq α .$

Hazard functions

If desired, the hazard function and the cumulative hazard function could be defined as

h (x_{k}) = \frac{p ( x _{k} )}{1 - F ( x _{k} )}

and

H (x) = x_{k} \leq x \sum h (x_{k}) = x_{k} \leq x \sum \frac{F ( x _{k} ) - F ( x _{k - 1} )}{1 - F ( x _{k} )} .

Moments

Non-central moments are defined using the PDF

μ_{m}^{'} = E [X^{m}] = k \sum x_{k}^{m} p (x_{k}) .

Central moments are computed similarly $μ = μ_{1}^{'}$

The mean is the first moment

μ = μ_{1}^{'} = E [X] = k \sum x_{k} p (x_{k})

the variance is the second central moment

μ_{2} = E [(X - μ)^{2}] = x_{k} \sum x_{k}^{2} p (x_{k}) - μ^{2} .

Skewness is defined as

γ_{1} = \frac{μ _{3}}{μ _{2}^{3/2}}

while (Fisher) kurtosis is

γ_{2} = \frac{μ _{4}}{μ _{2}^{2}} - 3,

so that a normal distribution has a kurtosis of zero.

Moment generating function

The moment generating function is defined as

M_{X} (t) = E [e^{X t}] = x_{k} \sum e^{x_{k} t} p (x_{k})

Moments are found as the derivatives of the moment generating function evaluated at $0.$

Fitting data

To fit data to a distribution, maximizing the likelihood function is common. Alternatively, some distributions have well-known minimum variance unbiased estimators. These will be chosen by default, but the likelihood function will always be available for minimizing.

If $f_{i} (k; θ)$ is the PDF of a random-variable where $θ$ is a vector of parameters ( e.g. $L$ and $S$ ), then for a collection of $N$ independent samples from this distribution, the joint distribution the random vector $k$ is

f (k; θ) = i = 1 \prod N f_{i} (k_{i}; θ) .

The maximum likelihood estimate of the parameters $θ$ are the parameters which maximize this function with $x$ fixed and given by the data:

Where

Standard notation for mean

We will use

\overline{y (x)} = \frac{1}{N} i = 1 \sum N y (x_{i})

where $N$ should be clear from context.

Combinations

Note that

k! = k \cdot (k - 1) \cdot (k - 2) \cdot \dots \cdot 1 = Γ (k + 1)

and has special cases of

and

(n k) = \frac{n !}{( n - k ) ! k !} .

If $n < 0$ or $k < 0$ or $k > n$ we define $(n k) = 0$