Master the Three Machine Learning Types and Model Paradigms

With the rapid development of the Internet, our lives are filled with various data such as text, music, images, and video, which not only enrich daily life but also drive technological progress. In this data‑driven era, machine learning is essential because it enables computers to learn from massive data and extract valuable information.

Three Types of Machine Learning

Machine learning is mainly divided into three categories: supervised learning, unsupervised learning, and reinforcement learning . Below each type is defined and its specific methods are listed.

Supervised Learning

Supervised learning learns from labeled training data to predict outputs for unseen data. Each training sample consists of input features and an expected output (label), and the model learns to map inputs to correct outputs.

Supervised learning can be likened to a knowledge‑hungry student receiving feedback from a teacher. The teacher (environment) provides feedback on correctness, helping the student (computer) improve. The ultimate goal is generalization—producing correct answers for unseen problems.

Supervised learning is widely applied in handwriting recognition, speech processing, image recognition, spam filtering, web retrieval, and many other fields.

Specific methods:

Linear Regression (Linear Regression): predicts numeric data by learning a linear equation to estimate numeric labels.

Logistic Regression (Logistic Regression): despite its name, it is used for classification, especially binary classification, predicting a probability that is then mapped to one of several classes.

Decision Trees (Decision Trees): learn decision rules from data to predict labels; applicable to both classification and regression.

Unsupervised Learning

Unsupervised learning seeks hidden structures in data without any labels. The training data consists of input vectors only, with no corresponding target values. The goal is to discover the data’s internal distribution, clusters, relationships, or other useful information.

Unlike supervised learning, unsupervised learning does not rely on external guidance or correct answers; it is akin to a student learning independently. It can help computers identify patterns, clusters, and relationships, and is powerful in video analysis, social network analysis, fault diagnosis, and more.

Specific methods:

K‑Means Clustering (K-Means Clustering): partitions data into K clusters, making points within a cluster as similar as possible while keeping different clusters distinct.

Principal Component Analysis (PCA): a dimensionality‑reduction technique that removes redundant features to simplify data while retaining the most important information.

Autoencoders: learn a compressed representation of data through unsupervised learning, reconstructing inputs from a smaller encoded layer.

Reinforcement Learning

Reinforcement learning is a paradigm where an agent interacts with an environment, learning a strategy through rewards or penalties to maximize long‑term reward.

Similar to supervised learning, it aims to enable computers to respond correctly to unknown problems, but instead of providing correct answers, it uses a reward mechanism to guide learning. This mirrors how infants learn behaviors to receive parental praise. Reinforcement learning is widely used in robot control, game AI, and market strategy optimization.

Specific methods:

Q‑Learning (Q-Learning): a model‑free algorithm that learns the expected utility of taking a specific action in a given state.

Deep Q‑Network (DQN): combines deep learning with Q‑Learning, using a deep neural network to approximate the Q‑value function for high‑dimensional inputs.

Policy Gradients: directly learn policy parameters to maximize cumulative reward, optimizing in policy space and suitable for continuous action spaces.

Generative and Discriminative Models

In machine learning, generative models and discriminative models are two major categories that differ fundamentally in how they handle data and learning tasks.

Generative Models

Generative models learn the data distribution, aiming to understand how data is generated so they can produce new data points.

Characteristics:

They can generate new data instances .

They learn the joint probability distribution of inputs X and outputs Y.

Applications include image generation, text generation, speech synthesis, and more.

Typical examples:

Gaussian Mixture Models (GMMs): probabilistic models representing data as a mixture of multiple Gaussian distributions.

Hidden Markov Models (HMMs): statistical models describing hidden state sequences behind observed sequences.

Generative Adversarial Networks (GANs): consist of a generator and a discriminator that compete, enabling the generation of realistic new data.

For GANs, the generator creates data that looks real, while the discriminator tries to distinguish generated data from real data; through competition, both improve, eventually producing data indistinguishable from real samples.

Discriminative Models

Discriminative models directly map inputs to outputs, focusing on predicting the probability of output labels given input data.

Characteristics:

They learn the conditional probability distribution P(Y|X).

Typically used for classification and regression tasks.

Suitable for clear decision boundaries, such as spam detection or disease diagnosis.

Typical examples:

Support Vector Machines (SVMs): find the optimal boundary between data points for classification and regression.

Logistic Regression : primarily used for binary classification, predicting probabilities via a sigmoid function.

Neural Networks : including deep learning models that learn feature representations to make predictions.

Comparison:

Goal: Generative models aim to learn the overall data distribution to generate new instances; discriminative models aim to learn the mapping from input to output for accurate prediction.

Use cases: Generative models suit tasks requiring new data creation (e.g., image synthesis); discriminative models suit classification and regression where decisions are needed.

Data requirements: Generative models generally need more data to capture the full distribution, while discriminative models can perform well with less data.

Each model has unique strengths and appropriate scenarios; the choice depends on the specific task and data characteristics.

Machine learning not only drives data science forward but also brings revolutionary changes across industries. Understanding and mastering machine learning techniques is essential.