Mathematical Derivation of the Central Limit Theorem
Introduction
The Central Limit Theorem (CLT) is a cornerstone of statistical theory, providing a foundational framework for understanding the distribution of sample means. It states that, under certain conditions, the sum of a large number of independent and identically distributed (i.i.d) random variables, when normalized, approaches a normal distribution, regardless of the original distribution of the variables. This theorem is pivotal in inferential statistics, as it justifies the use of the normal distribution to approximate sample means, even when the underlying population distribution is unknown or non-normal. The CLT bridges the gap between theoretical probability and practical statistical inference, enabling the construction of confidence intervals and hypothesis tests.
Theoretical Foundations
To derive the CLT, several key assumptions must be satisfied:
- Independence: The random variables must be independent.
- Identical Distribution: All variables must share the same distribution, characterized by finite mean and variance.
- Finite Variance: The population variance must be finite.
- Sample Size: The sample size must be sufficiently large (typically n ≥ 30).
These conditions ensure that the distribution of the sample mean converges to a normal distribution as the sample size increases. The theorem relies on the convergence of the distribution of the sample mean to a normal distribution, a result that is both intuitive and mathematically rigorous.
Mathematical Derivation
The derivation of the CLT involves analyzing the behavior of the sample mean as the number of observations grows. Let $ X_1, X_2, \dots, X_n $ be i.i.d random variables with mean $ \mu $ and variance $ \sigma^2 $. Define the sample mean as:
$$
\bar{X}n = \frac{1}{n} \sum{i=1}^{n} X_i
$$
The first step in the derivation is to compute the mean and variance of $ \bar{X}_n $. The mean of $ \bar{X}_n $ is:
$$
\mathbb{E}[\bar{X}_n] = \mu
$$
The variance of $ \bar{X}_n $ is:
$$
\text{Var}(\bar{X}_n) = \frac{\sigma^2}{n}
$$
This shows that the variance of the sample mean decreases as the sample size increases, aligning with the intuition that larger samples provide more precise estimates of the population mean.
To further analyze the distribution of $ \bar{X}n $, we consider the characteristic function or moment generating function of the sum of the random variables. The characteristic function of $ X_i $ is:
$$
\phi{X_i}(t) = \mathbb{E}[e^{itX_i}]
$$
For the sum $ S_n = \sum_{i=1}^{n} X_i $, the characteristic function becomes:
$$
\phi_{S_n}(t) = \left( \phi_{X_i}(t) \right)^n
$$
Using the properties of characteristic functions, we can derive the distribution of $ S_n $. However, the CLT focuses on the distribution of $ \bar{X}n $, which is obtained by normalizing $ S_n $:
$$
\bar{X}_n = \frac{S_n}{n}
$$
The characteristic function of $ \bar{X}_n $ is:
$$
\phi{\bar{X}n}(t) = \left( \phi{X_i}(t/n) \right)^n
$$
As $ n \to \infty $, the characteristic function converges to the characteristic function of a normal distribution with mean $ \mu $ and variance $ \sigma^2/n $. This convergence is a consequence of the Continuous Mapping Theorem and the Law of Large Numbers.
The proof of the CLT often employs the concept of convergence in distribution. For a sequence of random variables $ \bar{X}_n $, if $ \bar{X}_n $ converges in distribution to a normal distribution as $ n \to \infty $, then the CLT is established. This convergence is rigorously demonstrated using the Slutsky's Theorem, which relates the asymptotic behavior of random variables under different transformations.
Key Properties and Applications
The CLT guarantees that the distribution of $ \bar{X}_n $ approaches a normal distribution with mean $ \mu $ and variance $ \sigma^2/n $, regardless of the original distribution of $ X_i $. This property is critical in statistical inference because it allows for the use of the normal distribution to approximate the distribution of sample means, even when the population distribution is unknown or non-normal.
The theorem also highlights the role of the sample size in determining the accuracy of the approximation. As $ n $ increases, the distribution of $ \bar{X}_n $ becomes increasingly concentrated around $ \mu $, and the variance decreases, leading to narrower confidence intervals. This behavior is formalized through the Central Limit Theorem's convergence criteria, which requires that the random variables satisfy the conditions outlined earlier.
In practical applications, the CLT is used to construct confidence intervals and perform hypothesis testing. For example, a 95% confidence interval for the population mean $ \mu $ can be approximated as:
$$
\bar{X}n \pm z{\alpha/2} \sqrt{\frac{\sigma^2}{n}}
$$
where $