Chain Rule for Composite Functions: Single‑ and Multi‑Variable Cases

The previous lesson introduced derivatives and partial derivatives; this article moves on to the chain rule, which is used in back‑propagation to differentiate complex functions.

1. Composite Functions – When a function y = f(u) has its inner variable expressed as u = g(x), the overall function becomes y = f(g(x)), a nested or composite function.

2. Chain Rule

2.1 Single‑Variable Chain Rule – For y = f(u) and u = g(x), the derivative of the composite function f(g(x)) is obtained by multiplying the derivative of the outer function evaluated at the inner function by the derivative of the inner function: dy/dx = (df/du)·(du/dx). This can be visualized as a simple “cancellation” of differential symbols.

The right‑hand side of the formula treats dx, dy, du as independent letters, allowing a fraction‑like simplification that does not extend to squared differentials.

Using this notation, we can remember the chain rule as a “cancellation” of differentials, though it only works for first‑order terms.

As an example, the derivative of the composite function sigmoid(w·x + b) is derived using the chain rule.

2.2 Multi‑Variable Chain Rule – The same principle extends to functions of several variables. If z = u(x, y) and u = g(x, y), the derivative of z with respect to x is obtained by summing the products of partial derivatives: ∂z/∂x = (∂z/∂u)·(∂u/∂x) + (∂z/∂v)·(∂v/∂x), and similarly for y.

Thus, to differentiate z with respect to x, first differentiate the inner functions u and v, multiply by the corresponding partial derivatives of z, and sum the results; the same process applies for y.

Source: Mathematics Fundamentals Series (https://www.naah69.com)