Optimization Foundations and Applications in Machine Learning and Computer Vision

With the widespread adoption of machine learning, especially deep learning, most problems can be expressed as optimization tasks that require constructing a suitable objective function and solving it; this first lecture briefly explains examples of objective function design and shows how familiar algorithms fit into an optimization framework.

The optimization framework is widely used in computer vision, graphics, and machine learning, for tasks such as image stitching, image completion, and 3D mesh generation; several concrete application examples are presented at the end of the article.

In general, an optimization problem can be written as:

where the first term represents the space of possible solutions and the second term represents the constructed objective function.

As a simple example, 2‑D linear regression (the most basic machine‑learning problem) illustrates how to build a reasonable objective function. Given a set of points \( (x_i, y_i) \), we assume a line \( y = wx + b \) and formulate the fitting problem as a least‑squares minimization, whose solution involves matrix inversion and has a clear geometric interpretation.

When outliers are present, the standard least‑squares loss becomes sensitive; using a robust loss such as the Huber function mitigates this effect by behaving like an \( L_2 \) norm for small residuals and like an \( L_1 \) norm for large residuals.

Most machine‑learning models also include a regularization term to control model complexity and avoid over‑fitting. For example, a polynomial fitting objective combines a data‑fitting term with a regularizer, which can be an \( L_2 \) norm, \( L_1 \) norm, nuclear norm, etc., each bringing different properties such as sparsity or low‑rank priors.

Many well‑known algorithms become the solution of a specific optimization problem after an appropriate objective is defined. For instance, K‑means clustering solves a matrix‑factorization‑type objective, and image inpainting can be posed as a Poisson equation derived from a smoothness functional.

Further applications demonstrated include graph‑cut based matting, dynamic‑programming texture synthesis, CRF‑based semantic segmentation, and TV‑L1 based 3‑D reconstruction, each illustrating how a carefully crafted objective function drives the final result.

Author: 笨象 (internal code name); the full video of this lecture is available on the internal “Shell Learning” platform for employees.