The Normal Distribution

For any continuous random variable, point probabilities are identically zero: $P(X = x) = 0$ for every $x \in \mathbb{R}$. This is a consequence of the distribution being absolutely continuous with respect to Lebesgue measure $\lambda$ on $\mathbb{R}$.

The probability density function $f$ is the Radon-Nikodym derivative of the probability measure $P$ with respect to $\lambda$. Probability is recovered by integration over any measurable set $A \in \mathcal{B}(\mathbb{R})$:

The value $f(x)$ is a density, not a probability. It is not bounded above by 1 — its only constraint is $\int_{-\infty}^{\infty} f(x)\,dx = 1$. Probability accumulates only over intervals, not at points.

The normal family is parametrised by two quantities: the mean $\mu \in \mathbb{R}$, which determines the location of the distribution, and the variance $\sigma^2 > 0$, which controls its dispersion. We write $X \sim \mathcal{N}(\mu, \sigma^2)$ to denote that the random variable $X$ has density:

Every member of the family is an affine image of the standard normal $\mathcal{N}(0, 1)$: if $Z \sim \mathcal{N}(0, 1)$, then $X = \mu + \sigma Z \sim \mathcal{N}(\mu, \sigma^2)$. Equivalently, the entire two-parameter family is generated by translations and scalings of a single canonical form. This structure defines a location-scale family.

The density achieves its unique maximum at $x = \mu$, where $f(\mu) = (\sigma\sqrt{2\pi})^{-1}$. Inflection points occur at $x = \mu \pm \sigma$. All odd central moments vanish, and the even central moments satisfy $E[(X - \mu)^{2k}] = (2k-1)!! \cdot \sigma^{2k}$ for non-negative integers $k$, where $(2k-1)!! = 1 \cdot 3 \cdots (2k-1)$ is the double factorial.

Let $X \sim \mathcal{N}(\mu, \sigma^2)$. The standardised variable is defined as:

This is an affine transformation, not a change of distributional shape. Subtracting $\mu$ translates the distribution to the origin; dividing by $\sigma$ rescales the horizontal axis so that one unit corresponds to one standard deviation. The resulting variable $Z$ always follows the standard normal $\mathcal{N}(0, 1)$, regardless of the original parameters.

The practical consequence is that probability calculations for any member of the normal family reduce to a single reference: the standard normal CDF $\Phi(z)$. For any $a, b \in \mathbb{R}$:

Many classical procedures — t-tests, ANOVA, ordinary least squares regression — assume that observations or errors follow a normal distribution. In practice, this assumption is frequently violated. The consequences depend on the nature and degree of departure.

Two primary types of departure are relevant. Skewness, measured by $\gamma_1 = E[(X-\mu)^3]/\sigma^3$, quantifies asymmetry. Excess kurtosis measures the weight of the tails relative to a Gaussian. The normal distribution has $\gamma_1 = 0$ and excess kurtosis $\gamma_2 = 0$; distributions with $\gamma_2 > 0$ are leptokurtic (heavy-tailed), and those with $\gamma_2 < 0$ are platykurtic.

Robustness theory formalises the sensitivity of an estimator or test to distributional misspecification. The sample mean is not robust: a single contaminating outlier can move it arbitrarily far from the population centre. The sample median, by contrast, has a breakdown point of 50%. Always assess distributional assumptions via formal diagnostics — the Shapiro-Wilk test, Q-Q plots, or moment-based tests — before applying procedures that rely on normality.

Theorem (Central Limit Theorem). Let $X_1, X_2, \ldots$ be independent and identically distributed random variables with finite mean $\mu$ and finite variance $\sigma^2 > 0$. Denote the sample mean by $\bar{X}_n = n^{-1}\sum_{i=1}^n X_i$. Then, as $n \to \infty$:

Convergence here is in distribution (weak convergence of probability measures), not almost surely or in probability. The theorem does not require the underlying distribution to be symmetric, unimodal, or of any specific form — only that the first two moments are finite. This generality is its defining strength.

A direct consequence is that the sum $S_n = \sum_{i=1}^n X_i$ is approximately $\mathcal{N}(n\mu, n\sigma^2)$ for large $n$. The approximation improves with $n$ and is known to be uniform over the real line (Berry-Esseen theorem). The simulation at right draws samples from the Uniform distribution — a decidedly non-Gaussian shape — and accumulates their means. The bell curve emerges regardless of the population.

The multivariate normal distribution on $\mathbb{R}^d$ is specified by a mean vector $\boldsymbol{\mu} \in \mathbb{R}^d$ and a symmetric positive-definite covariance matrix $\boldsymbol{\Sigma} \in \mathbb{R}^{d \times d}$. We write $\mathbf{X} \sim \mathcal{N}_d(\boldsymbol{\mu}, \boldsymbol{\Sigma})$, with density:

The geometry of the distribution is encoded entirely in $\boldsymbol{\Sigma}$: contours of equal probability density are ellipsoids whose shape, orientation, and eccentricity are determined by the eigenstructure of $\boldsymbol{\Sigma}$. The exponent $(\mathbf{x} - \boldsymbol{\mu})^\top \boldsymbol{\Sigma}^{-1}(\mathbf{x} - \boldsymbol{\mu})$ is the squared Mahalanobis distance from $\mathbf{x}$ to $\boldsymbol{\mu}$, which accounts for correlations between dimensions.

The role of the correlation parameter $\rho$ in the bivariate case is particularly instructive:

• $\rho = 0$: the components are uncorrelated and, being jointly normal, statistically independent. Contours are circular (after standardisation).
• $\rho > 0$: positive linear association; the ellipse is tilted towards the northeast diagonal.
• $\rho < 0$: negative linear association; the ellipse is tilted towards the northwest diagonal.

A key property: any marginal distribution of a multivariate normal is itself normal, and any conditional distribution $\mathbf{X}_1 \mid \mathbf{X}_2 = \mathbf{x}_2$ is normal with a mean that is a linear function of $\mathbf{x}_2$. This closure under marginalisation and conditioning is what makes the Gaussian tractable in high-dimensional inference.