Unit 1.6: Optimality Criteria

How do we mathematically prove a point is a minimum? We use Calculus.

A. Single Variable Functions f(x)

1. Necessary Condition (First Order) If x* is an extremum (min or max), the slope must be flat.

• **f'(x) = 0*
• Root of this equation is called a Stationary Point.

2. Sufficient Condition (Second Order) Check the curvature (f''(x)) at the stationary point.

• **If f''(x) > 0: Graph curves UP (Cup shape) → Local Minimum*.
• **If f''(x) < 0: Graph curves DOWN (Frown shape) → Local Maximum*.
• **If f''(x) = 0:* Inconclusive (Inflection point).

---

B. Multivariable Functions f(X)

For f(x₁, x₂, ... xₙ).

1. Necessary Condition The Gradient Vector (∇f) must be zero.

• ∇f = [∂f/∂x₁, ∂f/∂x₂, ... ∂f/∂xₙ]ᵀ = 0
• This gives us n simultaneous equations to solve for X.

2. Sufficient Condition We calculate the Hessian Matrix (H) (Matrix of second derivatives). H = [∂²f / ∂x_i ∂x_j]

• Positive Definite H: All Eigenvalues > 0 → Local Minimum.
• Negative Definite H: All Eigenvalues < 0 → Local Maximum.
• Indefinite H: Some > 0, Some < 0 → Saddle Point.

---

Numerical Example for Criteria

Minimize f(x) = x³ - 3x

Step 1: Necessary Condition

• f'(x) = 3x² - 3
• Set 3x² - 3 = 0
• 3(x² - 1) = 0 => x = +1 or x = -1.
• Stationary Points are at x=1 and x=-1.

Step 2: Sufficient Condition

• f''(x) = 6x

At x = 1:

• f''(1) = 6(1) = 6
• Since 6 > 0 (Positive) → Local Minimum.

At x = -1:

• f''(-1) = 6(-1) = -6
• Since -6 < 0 (Negative) → Local Maximum.