Tagged articles
Backpropagation
22 articles · Page 1 of 1
This comprehensive guide explains large language models from first principles, covering token prediction, information theory, neural networks, Transformer architecture, training via gradient descent, and the path from base models to chat assistants like ChatGPT, including limitations such as hallucinations.
Attention MechanismBackpropagationGradient Descent
0 likes · 74 min read

This roundup covers essential insights on backpropagation fundamentals, FreeBSD self‑hosting benefits, an open‑source 30B MoE AI model, misuse of cybercrime laws, historic moving sidewalks, party‑planning hacks, deceptive signal‑strength tricks, a 1000‑hp micro motor, Nextcloud performance fixes, and Google Cloud account suspensions, offering a blend of technical depth and practical advice.
AIBackpropagationFreeBSD
0 likes · 11 min read

This article demystifies how Transformers are trained by showing that all their linear layers have learnable weights and biases, and that the attention mechanism—including softmax and dot‑product operations—is fully differentiable and updated via standard back‑propagation.
AttentionBackpropagationPyTorch
0 likes · 7 min read

This article walks through an interactive exploration of a simple two‑hidden‑layer neural network, showing how real‑time visualizations reveal its learned representations, accuracy limits, and why constrained training leads to over‑confident yet unintelligent predictions before introducing backpropagation.
Backpropagationdeep learninghidden layers
0 likes · 10 min read


JavaEdgeFeb 23, 2025 · Artificial Intelligence
This tutorial walks Java developers through the complete workflow of building, training, and evaluating a neural network in PyTorch, covering network definition, data iteration, forward and backward passes, loss calculation, and parameter updates with detailed code examples and Java‑centric analogies.
BackpropagationJavaNeural Network
0 likes · 12 min read

This article explains the structure of biological neurons, introduces the McCulloch‑Pitts (MP) mathematical model, shows how manual weight adjustments work, and walks through the development from single‑layer perceptrons to two‑layer networks and modern deep learning techniques, covering activation functions, training algorithms, and practical examples.
BackpropagationGradient DescentMP model
0 likes · 30 min read

This article explains the backpropagation algorithm—its origins, mathematical basis, step‑by‑step workflow, importance for efficient neural network training, and widespread applications in image recognition, natural language processing, and recommendation systems.
BackpropagationGradient Descentdeep learning
0 likes · 6 min read

This article provides an in‑depth tutorial on neural network algorithms, covering their biological inspiration, significance, advantages and drawbacks, detailed architecture, data preparation, one‑hot encoding, weight initialization, forward and backward propagation, cost functions, regularization, gradient checking, and complete Python code examples.
AIBackpropagationGradient Descent
0 likes · 37 min read

This article explains the back‑propagation algorithm in neural networks, starting with a simple single‑neuron example using ReLU, Sigmoid and MSE, then extending to multi‑layer matrix‑based networks, providing detailed Python code, gradient calculations, and comparisons with TensorFlow implementations.
BackpropagationGradient DescentPython
0 likes · 21 min read

This article uses a blindfolded mountain‑climbing analogy to explain how gradient descent trains neural networks, covering cost functions, learning rates, iterative updates, and provides a Python implementation for a simple three‑layer network example.
AIBackpropagationGradient Descent
0 likes · 10 min read


21CTOMay 9, 2023 · Artificial Intelligence
In a candid MIT Technology Review interview, AI pioneer Geoffrey Hinton discusses his departure from Google, the rapid progress of large language models like GPT‑4, the dangers of AI self‑motivation, and why halting AI development is unrealistic yet urgently needed.
AI riskAI safetyBackpropagation
0 likes · 28 min read


Code DAOMay 16, 2022 · Artificial Intelligence
This article walks through implementing a basic multi‑layer neural network using only NumPy, covering terminology, network architecture, forward and backward propagation, activation functions, loss calculation, parameter updates with SGD, and compares the custom model with a Keras implementation.
BackpropagationNeural NetworkNumPy
0 likes · 17 min read

This article explains how PyTorch's BP Engine dynamically builds the computation graph for back‑propagation, detailing its C++ class structure, thread management, task queues, and key functions such as start_threads, compute_dependencies, execute, and evaluate_function, with illustrative code examples.
BackpropagationC++Engine
0 likes · 17 min read

This article traces the evolution of artificial neural networks from their biological inspiration, explains the McCulloch‑Pitts neuron model, details perceptron architecture and learning rule with a Scikit‑Learn example, and introduces multilayer perceptrons and the back‑propagation algorithm together with common activation functions.
AIBackpropagationPerceptron
0 likes · 19 min read

This article explains PyTorch's autograd system, covering tensor creation, the requires_grad flag, detaching tensors, disabling gradient tracking with no_grad, the Function class and computational graph, and demonstrates forward and backward passes with code examples illustrating gradient computation and Jacobian‑vector products.
AutogradBackpropagationPyTorch
0 likes · 6 min read

This article explains the backpropagation training algorithm for fully connected artificial neural networks, detailing its gradient‑descent basis, mathematical derivation, matrix formulation, and provides a complete Python implementation with mini‑batch stochastic gradient descent, momentum, learning‑rate decay, and experimental results.
BackpropagationGradient DescentMini-Batch
0 likes · 14 min read

The article explains how basic mathematical principles—starting with simple predictors and linear classifiers, then extending to multi‑classifier systems, activation functions, and weight adjustments—underpin neural network architecture, illustrating each step with concrete examples to show how mathematics drives AI model training and performance.
BackpropagationXOR problemactivation function
0 likes · 15 min read

This tutorial explains neural network fundamentals by defining layers, weights, biases, and sigmoid activation, then walks through building a Python class that implements forward propagation, a sum‑of‑squared‑error loss, and backpropagation using the chain rule and gradient descent to train a simple two‑layer network.
BackpropagationGradient DescentNeural Network
0 likes · 8 min read

This article provides a comprehensive overview of deep learning, covering its definition, fundamental machine‑learning components, illustrative examples such as hot‑dog classification and house‑price prediction, the mathematics of cost functions and gradient descent, back‑propagation via the chain rule, and practical resources and code snippets using Torch.
BackpropagationCode ExamplesGradient Descent
0 likes · 11 min read

This article provides a comprehensive guide to Long Short-Term Memory networks, covering their motivation, detailed forward and backward equations, gate mechanisms, training algorithm, gradient checking, and a full C++ implementation, while also introducing the simpler GRU variant.
BackpropagationGRUImplementation
0 likes · 12 min read

This article provides a comprehensive guide to convolutional neural networks, covering their advantages over fully‑connected nets, architectural patterns, detailed forward and backward calculations, ReLU activation, pooling strategies, Python implementation with NumPy, gradient checking, and a practical MNIST application.
BackpropagationImplementationNumPy
0 likes · 22 min read

This article provides a light‑hearted yet thorough overview of deep learning, covering neural network fundamentals, layer construction, back‑propagation, ResNet shortcuts, encoder‑decoder structures, PU‑learning for unlabeled data, GPU acceleration, and practical advice on data size, frameworks, and deployment in financial scenarios.
BackpropagationGPUPu-Learning
0 likes · 27 min read
