Tagged articles
activation functions
11 articles · Page 1 of 1
This article explains neural network fundamentals, covering why logistic regression units form neurons, the problems with Sigmoid activation in hidden layers (vanishing gradients, non-zero centered outputs), why ReLU and its variants are preferred, how Softmax enables multi-class classification with cross-entropy loss, and provides a complete PyTorch implementation example.
Cross-Entropy LossPyTorchReLU
0 likes · 8 min read


Qborfy AIJul 2, 2025 · Artificial Intelligence
This article explains the role of activation functions in neural networks, compares five classic functions with formulas, performance trade‑offs, and gradient behavior, and provides a Python visualization demo plus several practical insights and real‑world examples.
ReLUSwishactivation functions
0 likes · 7 min read

The article explains why deep networks suffer from gradient vanishing—especially when using sigmoid or tanh activations—covers the underlying mathematics, compares activation functions, and presents practical techniques such as proper weight initialization, batch normalization, residual connections, and code examples to visualize the phenomenon.
ResNetactivation functionsbatch normalization
0 likes · 7 min read

This article explains how a simple neural network can recognize handwritten digits by preprocessing images, organizing neurons into input, hidden, and output layers, using weighted sums, biases, sigmoid compression, and matrix multiplication to illustrate the fundamentals of deep learning.
LayersSigmoidactivation functions
0 likes · 16 min read

This article explains the structure of biological neurons, introduces the McCulloch‑Pitts (MP) mathematical model, shows how manual weight adjustments work, and walks through the development from single‑layer perceptrons to two‑layer networks and modern deep learning techniques, covering activation functions, training algorithms, and practical examples.
BackpropagationGradient DescentMP model
0 likes · 30 min read

Activation functions are crucial for neural networks, providing non‑linearity, normalization, and gradient flow; this article reviews common functions such as Sigmoid, Tanh, ReLU, Leaky ReLU, ELU, Noisy ReLU, Softmax, and Swish, comparing their characteristics, advantages, drawbacks, and guidance for selecting the appropriate one.
activation functionsmachine learningmodel optimization
0 likes · 10 min read

The article reviews recent generative‑AI breakthroughs such as GPT‑5 and AI software engineers, explains that AI systems are deterministic rather than black boxes, and then teaches neural‑network fundamentals—including activation functions, back‑propagation, and a hands‑on MNIST digit‑recognition example with discussion of overfitting and regularization.
Gradient DescentMNISTactivation functions
0 likes · 17 min read

This article introduces artificial neural networks, explains various activation functions, describes common ANN models such as BP, RBF, FNN and LM, and provides step‑by‑step implementation of BP neural networks for classification and regression using Keras Sequential and scikit‑learn’s MLPClassifier/MLPRegressor.
BP Neural NetworkKerasactivation functions
0 likes · 6 min read

This article walks through creating a Python‑based neural network to predict breast cancer using the Wisconsin dataset, covering network architecture, weight and bias initialization, back‑propagation, gradient descent, and the role of activation functions such as sigmoid, tanh, ReLU and Leaky ReLU.
Gradient DescentNeural NetworkPython
0 likes · 13 min read

The article mathematically demystifies neural networks by first illustrating a linear predictor for kilometre‑to‑mile conversion and a basic bug classifier, then exposing the limits of single linear boundaries (e.g., XOR), before introducing artificial neurons, activation functions, and multi‑layer weight‑adjustment training.
Artificial NeuronClassificationactivation functions
0 likes · 15 min read

This article explains how LSTM networks overcome RNN limitations by using input, forget, and output gates with sigmoid and tanh activations, describes the core update equations, discusses alternative activation functions and hard‑gate variants, and provides references for deeper study.
LSTMRNNSequence Modeling
0 likes · 10 min read
