Tagged articles
Gradient Descent
37 articles · Page 1 of 1
This comprehensive guide explains large language models from first principles, covering token prediction, information theory, neural networks, Transformer architecture, training via gradient descent, and the path from base models to chat assistants like ChatGPT, including limitations such as hallucinations.
Attention MechanismBackpropagationGradient Descent
0 likes · 74 min read

This article breaks down the fundamentals of large model training—covering data, parameters, neural networks, loss functions, gradient descent, pre‑training, and fine‑tuning—in plain language so readers can grasp how massive models learn without needing to dive into complex mathematics.
Gradient DescentModel TrainingPretraining
0 likes · 12 min read

This article walks you through the evolution from basic mathematical functions to modern large‑scale models, explaining activation functions, forward and backward propagation, loss calculation, gradient descent, regularization, dropout, word embeddings, RNNs, and the core mechanics of the Transformer architecture.
Attention MechanismGradient DescentRNN
0 likes · 15 min read

This tutorial introduces linear regression fundamentals, explaining features, labels, weights, bias, and mean squared error loss, then demonstrates gradient descent optimization in PyTorch, reveals how unscaled features break convergence, and shows feature standardization fixing the problem.
Gradient DescentPyTorchfeature scaling
0 likes · 6 min read

This article breaks down the core deep‑learning optimizers—gradient descent, Momentum, AdaGrad, RMSProp and Adam—showing why vanilla gradient descent converges slowly, how each method uses exponential moving averages to accelerate training, and why Adam is generally the preferred choice.
AdaGradAdamGradient Descent
0 likes · 8 min read

This article explains how neural networks learn by using labeled training data, describing the role of weights, biases, activation functions, and how gradient descent iteratively adjusts parameters to minimize loss, illustrated with the MNIST digit‑recognition example.
Gradient DescentMNISTdeep learning
0 likes · 16 min read

This comprehensive guide explores linear regression from its fundamental concepts and mathematical foundations to practical Python implementation with scikit‑learn, covering single‑ and multiple‑variable models, assumptions, loss functions, OLS and gradient‑descent solutions, evaluation metrics, advantages, limitations, and real‑world case studies.
Gradient DescentModel EvaluationPython
0 likes · 21 min read

This article explains the structure of biological neurons, introduces the McCulloch‑Pitts (MP) mathematical model, shows how manual weight adjustments work, and walks through the development from single‑layer perceptrons to two‑layer networks and modern deep learning techniques, covering activation functions, training algorithms, and practical examples.
BackpropagationGradient DescentMP model
0 likes · 30 min read

This article walks through the complete training cycle of a simple neural network—from random weight initialization and forward propagation with labeled data, through loss calculation and gradient‑based weight updates, to iterative epochs, average loss, and practical issues like gradient explosion and vanishing.
AIGradient DescentModel Training
0 likes · 11 min read

This article uses a dialogue format to introduce the fundamentals of linear regression, explain how loss functions such as mean squared error quantify prediction errors, and describe gradient descent as an iterative optimization technique for finding the best model parameters, illustrated with simple numeric examples and visual aids.
AI basicsGradient Descentlinear regression
0 likes · 13 min read

This article demonstrates how to build a multivariate linear regression model using scikit-learn's LinearRegression class, visualizes the results, and then compares the implementation, parameter optimization, prediction accuracy, and visualization with a manually coded gradient descent approach.
Gradient DescentLinearRegressionPython
0 likes · 7 min read

This tutorial walks through loading a delivery‑time CSV dataset, defining a multivariate linear regression model, computing mean‑squared error, optimizing parameters with gradient descent, and visualizing the fitted plane alongside the data points in a 3‑D Matplotlib plot.
Gradient DescentNumPyPython
0 likes · 8 min read

This article walks through linear regression by visualizing scattered data points, defining the line y = ax + b, deriving the mean‑squared‑error cost function, computing its gradients with the chain rule, and implementing gradient descent in Python to find the optimal parameters.
Gradient DescentPythoncost function
0 likes · 11 min read

This article explains the backpropagation algorithm—its origins, mathematical basis, step‑by‑step workflow, importance for efficient neural network training, and widespread applications in image recognition, natural language processing, and recommendation systems.
BackpropagationGradient Descentdeep learning
0 likes · 6 min read

This article provides an in‑depth tutorial on neural network algorithms, covering their biological inspiration, significance, advantages and drawbacks, detailed architecture, data preparation, one‑hot encoding, weight initialization, forward and backward propagation, cost functions, regularization, gradient checking, and complete Python code examples.
AIBackpropagationGradient Descent
0 likes · 37 min read

The article reviews recent generative‑AI breakthroughs such as GPT‑5 and AI software engineers, explains that AI systems are deterministic rather than black boxes, and then teaches neural‑network fundamentals—including activation functions, back‑propagation, and a hands‑on MNIST digit‑recognition example with discussion of overfitting and regularization.
Gradient DescentMNISTactivation functions
0 likes · 17 min read

This article uses a narrative of an AI neuron to explain fundamental concepts of neural networks, including neuron structure, weighted sums, activation functions, loss functions, gradient descent, and learning rate, making complex AI topics accessible to beginners.
AI basicsGradient DescentNeural Network
0 likes · 9 min read

This article explains the back‑propagation algorithm in neural networks, starting with a simple single‑neuron example using ReLU, Sigmoid and MSE, then extending to multi‑layer matrix‑based networks, providing detailed Python code, gradient calculations, and comparisons with TensorFlow implementations.
BackpropagationGradient DescentPython
0 likes · 21 min read

This article uses a blindfolded mountain‑climbing analogy to explain how gradient descent trains neural networks, covering cost functions, learning rates, iterative updates, and provides a Python implementation for a simple three‑layer network example.
AIBackpropagationGradient Descent
0 likes · 10 min read

This article introduces nonlinear programming, explains its challenges compared to linear programming, and demonstrates two concrete optimization examples solved with Python's SciPy library using gradient‑descent and SLSQP algorithms, complete with code and result interpretation.
Gradient Descentnonlinear programmingoptimization
0 likes · 5 min read

Andrew Ng’s latest AI newsletter article revisits six foundational machine‑learning algorithms—linear regression, logistic regression, gradient descent, neural networks, decision trees, and k‑means clustering—tracing their historical origins, core concepts, and lasting impact on modern AI applications.
Decision TreesGradient Descentk-means clustering
0 likes · 20 min read

This comprehensive guide explains the mathematical foundation, intuitive intuition, algorithmic steps, tuning strategies, and variants of gradient descent, comparing it with other optimization methods and illustrating its use in machine‑learning models such as linear regression.
Gradient Descentlearning ratelinear regression
0 likes · 14 min read

Using a real‑world housing example, this article explains supervised and unsupervised learning, walks through building a price‑prediction function, introduces gradient descent for optimizing weights, and highlights pitfalls like overfitting, offering a practical introduction to core machine‑learning concepts.
Gradient DescentPythonlinear regression
0 likes · 13 min read

This article explains why algebraic methods often fail to locate critical points in real‑world problems and introduces the steepest descent (gradient descent) algorithm as a simple iterative numerical technique, detailing its initialization, update rules, step‑size considerations, and a concrete example.
Gradient Descentcritical pointsnumerical optimization
0 likes · 3 min read


Code DAODec 6, 2021 · Artificial Intelligence
This article explains the fundamental gradient‑descent optimizers used in neural networks—SGD, Momentum, RMSProp, Adam and their variants—illustrates loss‑surface challenges such as local minima, saddle points and ravines, and shows how techniques like mini‑batching, momentum, adaptive learning rates and scheduling address these issues.
AdamGradient DescentSGD
0 likes · 11 min read


Code DAODec 5, 2021 · Artificial Intelligence
The article explains the principle behind Batch Normalization, why it is essential for training deep neural networks, how it standardizes activations, the role of learnable scale and shift parameters, the computation steps during training and inference, and discusses placement strategies within a model.
Gradient Descentbatch normalizationdeep learning
0 likes · 9 min read

This tutorial walks through the complete process of building a linear regression model in Python from loading a housing price dataset, normalizing features, defining hypothesis, cost and gradient‑descent functions, visualising data and cost convergence, and testing predictions, with full source code provided.
Gradient DescentPythonlinear regression
0 likes · 12 min read

This article explains the intuition behind gradient descent, describing how the cost function surface is visualized, how the learning rate controls step size, and why updating parameters in the correct order is essential for converging to the optimal solution.
Gradient Descentcost functionlearning rate
0 likes · 4 min read

This article explains the fundamentals of machine learning by illustrating how humans recognize cats, extracting features, training models with gradient descent, implementing linear regression, and providing practical JavaScript and SQL code examples for building a simple classifier.
Gradient DescentJavaScriptSQL
0 likes · 12 min read

This article explains the concept of loss functions and gradient descent, illustrates how to find the global optimum for linear regression, discusses the role of learning rate, and provides a complete Python example that generates data, applies gradient descent, and visualizes the results.
Gradient DescentPythonlinear regression
0 likes · 6 min read

This article explains the backpropagation training algorithm for fully connected artificial neural networks, detailing its gradient‑descent basis, mathematical derivation, matrix formulation, and provides a complete Python implementation with mini‑batch stochastic gradient descent, momentum, learning‑rate decay, and experimental results.
BackpropagationGradient DescentMini-Batch
0 likes · 14 min read

This article explains how TensorFlow powers Huajiao Live's recommendation system, introduces the TensorSlow project for demystifying TensorFlow's core, and walks through deep‑learning fundamentals, computational‑graph concepts, forward and backward propagation, loss construction, gradient‑descent optimization, and building a multi‑layer perceptron with Python code examples.
Computational GraphGradient DescentMLP
0 likes · 13 min read

This article introduces deep learning applications at Huajiao Live, explains TensorFlow's computational graph architecture, details core concepts such as placeholders, variables, operations, forward and backward propagation, and provides complete Python-like code examples for building and training a multi-layer perceptron.
Computational GraphGradient DescentMLP
0 likes · 14 min read

This article walks through creating a Python‑based neural network to predict breast cancer using the Wisconsin dataset, covering network architecture, weight and bias initialization, back‑propagation, gradient descent, and the role of activation functions such as sigmoid, tanh, ReLU and Leaky ReLU.
Gradient DescentNeural NetworkPython
0 likes · 13 min read

This tutorial explains neural network fundamentals by defining layers, weights, biases, and sigmoid activation, then walks through building a Python class that implements forward propagation, a sum‑of‑squared‑error loss, and backpropagation using the chain rule and gradient descent to train a simple two‑layer network.
BackpropagationGradient DescentNeural Network
0 likes · 8 min read

This article provides a comprehensive overview of deep learning, covering its definition, fundamental machine‑learning components, illustrative examples such as hot‑dog classification and house‑price prediction, the mathematics of cost functions and gradient descent, back‑propagation via the chain rule, and practical resources and code snippets using Torch.
BackpropagationCode ExamplesGradient Descent
0 likes · 11 min read

This article introduces classic optimization algorithms, distinguishing direct methods that require convexity and closed‑form solutions from iterative first‑ and second‑order methods, and explains their applicability, underlying theory, and key references for solving smooth unconstrained problems.
Gradient DescentNewton's methodalgorithm fundamentals
0 likes · 8 min read
