How to Find Global and Local Optima Using Derivatives: A Step-by-Step Guide

In this article we will learn how to use derivatives to find a function's optimal values (minimum or maximum) and the points where they occur. This is important in applied mathematics, statistics, and machine learning, where minimizing an error function yields powerful algorithms.

1 Global Optimal Point

For a function, a global minimum point (A point of global minimum) satisfies that the function value at this point is the smallest among all values in the domain.

2 Local Optimal Point

We define a local minimum point (A point of local minimum) as a point where, within a small neighbourhood around it, the function value is the smallest. If a point is a global minimum, it is also a local minimum. The definitions for local and global maxima are analogous, with the inequality direction reversed.

Critical Points

When a function is smooth (differentiable everywhere), a critical point is where its derivative equals zero, i.e., the tangent line is horizontal. A critical point can be a maximum, a minimum, or a saddle point.

If a point is a critical point, then it is either a maximum/minimum or a saddle point.

Near a saddle point the magnitude of the derivative decreases to zero, but after that the function continues to increase or decrease in the same direction it was moving before. In contrast, at a maximum or minimum the direction of change reverses.

Finding Critical Points

To locate minima or maxima we solve for the critical points by setting the derivative equal to zero.

Example 1

For a quadratic function, its derivative is linear. Solving the derivative equal to zero yields a single critical point, which is both the local and global extremum.

Example 2

For a cubic function, its derivative is quadratic. Solving the derivative equal to zero gives two critical points, which can be identified as local extrema (one maximum and one minimum).