Unit 3.3: Indirect Methods: Penalty Function

Penalty function methods transform a constrained problem into an unconstrained problem by adding a "penalty" term to the objective function. This penalty increases heavily if constraints are violated.

Transformed Function

**Φ(X, r) = f(X) + r P(X)*

Where:

• Φ: New pseudo-objective function.
• f(X): Original objective function.
• r: Penalty parameter (a positive constant).
• P(X): Penalty term (depends on constraint violation).

Types of Penalty Methods

1. Exterior Penalty Method

• Starts from outside the feasible region.
• Adds a penalty only if constraints are violated.
• The solution approaches the optimum from the infeasible side.
• Formula: P(X) = max(0, g(X))²
• (Penalty is 0 if feasible; positive if infeasible).

2. Interior Penalty Method (Barrier Method)

• Starts from inside the feasible region.
• Adds a penalty that goes to infinity as the point approaches the boundary.
• Prevents the solution from ever leaving the feasible region.
• Formula: P(X) = -1/g(X) or P(X) = -log(-g(X))
• (As g(X) → 0, P(X) → ∞).

Algorithm

1. Choose initial X₁ and penalty r₁.
2. Minimize Φ(X, r₁) using any unconstrained method (Steepest Descent/Newton).
3. Update r (Increase for Exterior, Decrease for Interior).
4. Repeat until convergence.