Partial Derivatives and Gradient Vectors

Introduction

Partial derivatives and gradient vectors are foundational concepts in multivariable calculus, providing a framework for analyzing functions of multiple variables. A partial derivative measures how a function changes as one of its variables is varied while keeping the others constant. This concept extends the single-variable derivative to higher dimensions, enabling the study of functions defined on domains with multiple independent variables. Gradient vectors, on the other hand, encapsulate the directional derivatives of a function at a given point, representing the vector of partial derivatives. Together, these tools are essential for understanding the behavior of functions in multiple dimensions, optimizing systems, and solving problems in physics, engineering, and machine learning.

Definitions and Properties of Partial Derivatives

A partial derivative of a function $ f(x_1, x_2, \dots, x_n) $ with respect to a variable $ x_i $ is the rate of change of $ f $ with respect to $ x_i $, while holding all other variables constant. Mathematically, this is expressed as:
$$ \frac{\partial f}{\partial x_i} = \lim_{h \to 0} \frac{f(x_1, \dots, x_{i-1}, x_i + h, \dots, x_n) - f(x_1, \dots, x_{i-1}, x_i, \dots, x_n)}{h} $$
This definition aligns with the single-variable derivative, adapting the concept to multiple variables. The partial derivative of a function $ f $ with respect to $ x_i $ is also known as the directional derivative in the direction of the $ i $-th coordinate axis.

Key properties of partial derivatives include:

• Linearity: $ \frac{\partial}{\partial x_i}(f + g) = \frac{\partial f}{\partial x_i} + \frac{\partial g}{\partial x_i} $.
• Product Rule: $ \frac{\partial}{\partial x_i}(u v) = \frac{\partial u}{\partial x_i} v + u \frac{\partial v}{\partial x_i} $.
• Chain Rule for Partial Derivatives: If $ f $ is a function of $ u $ and $ v $, and $ u $ is a function of $ x_i $, then:
  $$ \frac{\partial f}{\partial x_i} = \frac{\partial f}{\partial u} \frac{\partial u}{\partial x_i} + \frac{\partial f}{\partial v} \frac{\partial v}{\partial x_i} $$
  These properties allow the computation of derivatives for composite functions and are critical in applied contexts.

Gradient Vectors and Their Interpretation

A gradient vector is a vector whose components are the partial derivatives of a function. For a function $ f: \mathbb{R}^n \to \mathbb{R} $, the gradient $ \nabla f $ is defined as:
$$ \nabla f = \left( \frac{\partial f}{\partial x_1}, \frac{\partial f}{\partial x_2}, \dots, \frac{\partial f}{\partial x_n} \right) $$
This vector points in the direction of maximum increase of $ f $, and its magnitude represents the rate of change in that direction. The gradient is a powerful tool in optimization, as it identifies the direction of steepest ascent for a function.

The gradient vector is closely tied to directional derivatives. The directional derivative of $ f $ in the direction of a unit vector $ \mathbf{u