Why You Don't Need Advanced Math to Start Learning Deep Learning

Today, with AI’s dominant role, deep learning appears in almost every popular AI application—semantic understanding, image and speech recognition, natural language processing, and more. Many even equate AI with deep learning.

For aspiring programmers, students, or hobbyists, not knowing deep learning feels like falling behind the times.

Although deep learning traditionally requires mathematics such as calculus, linear algebra, and probability, the essential knowledge is far simpler: understanding derivatives and related function concepts is enough, even for those who have only studied middle‑school math.

Derivatives Explained with a Simple Example

Consider "Wang Xiaoer sells pigs". If he sold 100 pigs this year, 90 last year, and 80 the year before, the growth rate (derivative) is 10 pigs per year. Using the function y = f(x) = 10x + 30, where x is time (years) and y is the number of pigs, the derivative is 10.

If the growth rate changes, the function could be y = f(x) = 5x² + 30. The derivative is no longer constant.

Partial Derivatives for Multiple Variables

When a function has several variables, the partial derivative measures the change rate with respect to one variable while keeping the others constant. Example: y = f(x₁, x₂, x₃) = 5x₁² + 8x₂ + 35x₃ + 30 Here x₁ is time, x₂ is farm area, and x₃ is the number of employees. The partial derivative with respect to x₃ is 35, meaning each additional employee adds 35 pigs.

We denote a partial derivative as ∂y/∂x₃.

Why This Matters for Deep Learning

Deep learning uses neural networks to solve problems that are not linearly separable. The network consists of an input layer, hidden layers, and an output layer.

Analogy: dating stages map to network layers. The "first love" stage is the input layer (many factors like height, personality, etc.). The "hot love" stage is the hidden layer (interaction and adjustment). The "stable period" is the output layer (whether the relationship works). Over‑fitting and under‑fitting correspond to too much or too little training data.

Training data (the "training set") provides correct answers for many images, allowing the network to learn weights. Test data (the "test set") evaluates the model’s accuracy on unseen examples.

Adjusting Parameters via Gradients

Each network parameter has a default value. By adding a small change Δ and observing the effect on the output, we can determine whether to increase or decrease the parameter. This requires the error’s partial derivative with respect to each parameter—essentially the gradient.

Activation Functions

Activation functions introduce non‑linearity, enabling the network to model complex relationships. Common choices are sigmoid and ReLU. The derivative of the sigmoid is simple: f'(x) = f(x) * [1 - f(x)] For ReLU, f(x) = 0 when x < 0, otherwise f(x) = x. Custom variants (e.g., leaky ReLU) are also possible.

Learning Rate and Optimization

The learning rate (学习系数) determines how large a step we take along the gradient. It can change during training. Other essential concepts include SGD (stochastic gradient descent), mini‑batch, and epoch.

In practice, engineers often tune parameters on a pre‑defined network, while researchers may design new algorithms.

Source: 数盟