This article explains how the Softmax function converts raw neural‑network scores into a proper probability distribution, why this conversion is essential for training and inference in large language models, and how the temperature parameter shapes the model's creativity and diversity.
The article explains the Temperature hyper‑parameter in large language models, shows how it modifies the softmax distribution, provides a Python visualisation script, and demonstrates through experiments that higher values increase creativity while lower values make outputs more deterministic.
This article explains how loss functions evaluate model performance, contrasts regression’s mean squared error with classification’s cross‑entropy, describes one‑hot encoding and softmax outputs, and shows why higher predicted probabilities for the correct class yield lower loss, highlighting applications in image, language, and speech tasks.
This article explains the fundamentals of recurrent neural networks (RNNs), their role in language modeling, various RNN architectures such as bidirectional and deep RNNs, the back‑propagation through time (BPTT) training algorithm, gradient challenges, vectorization techniques, and provides a step‑by‑step code implementation.
