4.4 Multivariable Extrema, Hessian Matrix, and Lagrange Multipliers

4.4 Multivariable Extrema, Hessian Matrix, and Lagrange Multipliers

Optimization links multivariable calculus, linear algebra, and geometry. On the GRE Mathematics Subject Test, questions test critical point classification via the Hessian matrix, boundary optimization on compact domains, and Lagrange multipliers under geometric constraints.

---

Critical Points of Multivariable Functions

Let $f: D \subset \mathbb{R}^n \to \mathbb{R}$. An interior point $\mathbf{x}_0 \in \operatorname{int}(D)$ is a critical point of $f$ if:

1. $\nabla f(\mathbf{x}_0) = \mathbf{0}$ (all partial derivatives vanish simultaneously), or
2. At least one partial derivative fails to exist at $\mathbf{x}_0$.

Multivariable Fermat's Theorem

If $f$ attains a local extremum at an interior point $\mathbf{x}_0$ and is differentiable there, then $\nabla f(\mathbf{x}_0) = \mathbf{0}$. This is a necessary condition, not a sufficient one: a critical point can be a local minimum, local maximum, or saddle point.

---

The Second Derivative Test in Two Dimensions

Let $(x_0, y_0)$ be a critical point of a $C^2$ function $f(x, y)$ where $\nabla f(x_0, y_0) = \mathbf{0}$. The Hessian matrix is: $H(x, y) = \begin{pmatrix} f_{xx} & f_{xy} \\ f_{yx} & f_{yy} \end{pmatrix}$ By Clairaut's Theorem, $f_{xy} = f_{yx}$. The Hessian discriminant is: $D = \det(H(x_0, y_0)) = f_{xx} f_{yy} - (f_{xy})^2$

Classification Rules

1. Local Minimum: $D > 0$ and $f_{xx} > 0$ (also implies $f_{yy} > 0$).
2. Local Maximum: $D > 0$ and $f_{xx} < 0$ (also implies $f_{yy} < 0$).
3. Saddle Point: $D < 0$. The surface curves upwards in some directions and downwards in others.
4. Inconclusive: $D = 0$. The test provides no information; analyze higher-order terms or directional paths.

---

The Hessian Matrix and Definiteness in $\mathbb{R}^n$

For $f: \mathbb{R}^n \to \mathbb{R}$, the second-order Taylor expansion about a critical point $\mathbf{x}_0$ is: $f(\mathbf{x}_0 + \mathbf{h}) = f(\mathbf{x}_0) + \frac{1}{2} \mathbf{h}^T H(\mathbf{x}_0) \mathbf{h} + R_2(\mathbf{h})$ Because $H(\mathbf{x}_0)$ is real symmetric, all eigenvalues $\lambda_1, \dots, \lambda_n$ are real:

• Positive Definite (all $\lambda_i > 0$): $\mathbf{x}_0$ is a strict local minimum.
• Negative Definite (all $\lambda_i < 0$): $\mathbf{x}_0$ is a strict local maximum.
• Indefinite (mixed signs: $\lambda_i > 0$ and $\lambda_j < 0$): $\mathbf{x}_0$ is a saddle point.
• Semidefinite (some $\lambda_i = 0$): The test is inconclusive.

Sylvester's Criterion

Let $\Delta_k$ denote the $k$-th leading principal minor of $H$:

• Positive definite $\iff \Delta_k > 0$ for all $k = 1, \dots, n$.
• Negative definite $\iff (-1)^k \Delta_k > 0$ for all $k = 1, \dots, n$ (alternating signs $\Delta_1 < 0, \Delta_2 > 0, \Delta_3 < 0, \dots$).

---

Global Extrema on Compact Sets

By the Extreme Value Theorem (EVT), a continuous function on a compact set $K \subset \mathbb{R}^n$ attains an absolute maximum and minimum.

Systematic Optimization Procedure

1. Interior: Identify critical points in $\operatorname{int}(K)$ and evaluate $f$.
2. Boundary: Find extreme values on boundary $\partial K$ via parameterization or Lagrange multipliers, including corner points.
3. Comparison: The largest value is the global maximum; the smallest is the global minimum.

---

The Method of Lagrange Multipliers

Single Constraint

To optimize $f(x, y, z)$ subject to $g(x, y, z) = c$ (with $\nabla g \neq \mathbf{0}$): $\nabla f = \lambda \nabla g, \quad g(x, y, z) = c$ Geometric Meaning: At an optimum, the level set of $f$ is tangent to $g = c$. Their normal vectors are collinear, so $\nabla f = \lambda \nabla g$. The scalar $\lambda$ is the Lagrange multiplier.

Multiple Constraints

To optimize $f(\mathbf{x})$ subject to $g_1(\mathbf{x}) = c_1, \dots, g_k(\mathbf{x}) = c_k$: $\nabla f(\mathbf{x}) = \sum_{i=1}^k \lambda_i \nabla g_i(\mathbf{x})$ provided constraint gradients $\nabla g_1, \dots, \nabla g_k$ are linearly independent.

---

Step-by-Step Worked Problems

Problem 1: Critical Point Classification

Find and classify all critical points of $f(x, y) = x^3 + y^3 - 3xy$.

Solution:

1. Partials: $f_x = 3x^2 - 3y = 0 \implies y = x^2$; $f_y = 3y^2 - 3x = 0 \implies x = y^2$.
2. Substitute: $x = x^4 \implies x(x^3 - 1) = 0$, giving critical points $(0, 0)$ and $(1, 1)$.
3. Second partials: $f_{xx} = 6x, f_{yy} = 6y, f_{xy} = -3 \implies D = 36xy - 9$.
4. Classification:
   • At $(0, 0)$: $D = -9 < 0 \implies$ Saddle point with $f(0, 0) = 0$.
   • At $(1, 1)$: $D = 27 > 0$ and $f_{xx} = 6 > 0 \implies$ Local minimum with $f(1, 1) = -1$.

Problem 2: Lagrange Multiplier Optimization

Find the maximum value of $f(x, y, z) = x + 2y + 3z$ on $x^2 + y^2 + z^2 = 14$.

Solution:

1. Gradients: $\nabla f = \langle 1, 2, 3 \rangle$ and $\nabla g = \langle 2x, 2y, 2z \rangle$.
2. Set $\nabla f = \lambda \nabla g$: $1 = 2\lambda x, 2 = 2\lambda y, 3 = 2\lambda z \implies x = \frac{1}{2\lambda}, y = \frac{1}{\lambda}, z = \frac{3}{2\lambda}$.
3. Substitute: $\frac{1 + 4 + 9}{4\lambda^2} = 14 \implies 4\lambda^2 = 1 \implies \lambda = \pm \frac{1}{2}$.
4. Candidates: $\lambda = 1/2 \implies (1, 2, 3)$ with $f = 14$; $\lambda = -1/2 \implies (-1, -2, -3)$ with $f = -14$.
5. The maximum value is $14$.

---

GRE Exam Traps & Pitfalls

Trap 1: Assuming Inconclusive Tests ($D = 0$) Imply Saddle Points When $D = 0$, the Second Derivative Test is inconclusive. Do not guess saddle point; check higher-order terms.

Trap 2: Ignoring Boundaries in Global Extrema On compact domains, global extrema frequently occur on the boundary rather than at interior critical points.

Trap 3: Division by Zero in Lagrange Multipliers When solving $\nabla f = \lambda \nabla g$, check if setting a variable to zero provides a solution before dividing by it.