Derivation of the Expected Value Formula

Introduction

In probability theory and statistics, the expected value, often denoted as $ \mathbb{E}[X] $, represents the long-term average value of repetitions of a random experiment. It is a fundamental concept that quantifies the central tendency of a random variable. The derivation of the expected value formula is essential for understanding how to compute the average outcome of a random process, regardless of the distribution of the variable. This derivation begins with the definition of expected value for discrete random variables and extends to continuous cases, leveraging principles such as linearity of expectation and the law of total expectation. By systematically analyzing the properties of random variables and their distributions, we can arrive at a rigorous mathematical formulation of the expected value.

Theoretical Foundations

To derive the expected value formula, it is necessary to establish the foundational concepts of probability distributions and their relationship to random variables. A random variable $ X $ can be classified as either discrete or continuous. For a discrete random variable, the expected value is calculated by summing the product of each possible value $ x $ and its corresponding probability $ P(X = x) $. This is expressed mathematically as:
$$ \mathbb{E}[X] = \sum_{x} x \cdot P(X = x) $$
For continuous random variables, the expected value is defined using an integral over the range of possible values, weighted by the probability density function $ f(x) $:
$$ \mathbb{E}[X] = \int_{-\infty}^{\infty} x \cdot f(x) , dx $$
The linearity of expectation, which states that the expected value of a sum is the sum of the expected values, plays a critical role in simplifying calculations. This property allows for the decomposition of complex random variables into simpler components, facilitating the derivation of expected values for functions of random variables.

Mathematical Derivation

The derivation of the expected value formula begins with the definition of a discrete random variable. Suppose $ X $ is a discrete random variable that takes on values $ x_1, x_2, \ldots, x_n $ with corresponding probabilities $ p_1, p_2, \ldots, p_n $, where $ \sum_{i=1}^{n} p_i = 1 $. The expected value of $ X $ is then computed as the sum of each value multiplied by its probability:
$$ \mathbb{E}[X] = \sum_{i=1}^{n} x_i \cdot p_i $$
This formula encapsulates the idea that the expected value is a weighted average of the possible outcomes, with weights being the probabilities of those outcomes. For example, consider a fair six-sided die, where each outcome $ x_i $ (from 1 to 6) has an equal probability $ p_i = \frac{1}{6} $. The expected value would be:
$$ \mathbb{E}[X] = \sum_{i=1}^{6} i \cdot \frac{1}{6} = \frac{1 + 2 + 3 + 4 + 5 + 6}{6} = \frac{21}{6} = 3.5 $$
This result aligns with the intuitive expectation that the average of the die rolls should be around 3.5.

The derivation for continuous random variables follows a similar structure but involves integration instead of summation. If $ X $ is a continuous random variable with probability density function $ f(x) $, the expected value is given by:
$$ \mathbb{E}[X] = \int_{-\infty}^{\infty} x \cdot f(x) , dx $$
This formulation generalizes the discrete case, allowing for the computation of expected values over any real-valued domain. The integral represents the area under the curve of the density function, weighted by the variable $ x $. For instance, consider a uniform distribution on the interval $[a, b]$, where $ f(x) = \frac{1}{b - a} $ for $ a \leq x \leq b $. The expected value would be:
$$ \mathbb{E}[X] = \int_{a}^{b} x \cdot \frac{1}{b - a} , dx = \frac{1}{b - a} \int_{a}^{b} x , dx = \frac{1}{2} \cdot \frac{b^2 - a^2}{b - a} = \frac{b + a}{2} $$
This result corresponds to the midpoint of the interval, reflecting the symmetry of the uniform distribution.

Applications and Examples

The expected value formula has wide-ranging applications in fields such as finance, physics, and engineering. In finance, the expected value is used to calculate the expected return of an investment, taking into account both the probability of different outcomes and their respective financial impacts. Similarly, in physics, the expected value is employed to determine the average position or momentum of a particle in a quantum mechanical system.

Another illustrative example involves the expected value of a random walk. Suppose a particle moves randomly left or right with equal probability, and the expected value of its position after $ n $ steps can be derived using the formula:
$$ \mathbb{E}[X] = n \cdot \frac{1}{2} $$
This result demonstrates how the expected value accounts for the balance between positive and negative movements. Additionally, the expected value of a function of a random variable can be computed using the law of total expectation, which states that:
$$ \mathbb{E}[g(X)] = \mathbb{E}[g(X \mid Y)] $$
This property is particularly useful when dealing with complex random variables, as it allows for the decomposition of the problem into simpler sub