Author

Code Archaeology

Learn from the past to understand the present

Articles

Likes

Views

Comments

Latest from Code Archaeology

67 recent articles

Code Archaeology

Nov 11, 2025 · Artificial Intelligence

Prompt Engineering Essentials: Best Practices & Attack Defense for LLMs

This guide introduces prompt engineering for LLMs, explaining how prompts work with roles like system and user, detailing best practices from Anthropic and OpenAI including clear instructions, context provision, task decomposition, and chain-of-thought, and covering three types of prompt attacks with model, prompt, and system-level defenses.

Best PracticesChain of ThoughtDeepSeek

0 likes · 7 min read

Prompt Engineering Essentials: Best Practices & Attack Defense for LLMs

Code Archaeology

Oct 27, 2025 · Artificial Intelligence

Understanding LLM Output & Sampling: Temperature, Top-k, Top-p with DeepSeek Examples

This article explains how LLMs generate tokens via probability distributions, details three sampling strategies — temperature, top-k, and top-p — with concrete DeepSeek API examples, outlines four methods for structured output, and discusses why hallucination is an inherent feature of probabilistic generation rather than a bug.

DeepSeekLLMhallucination

0 likes · 8 min read

Understanding LLM Output & Sampling: Temperature, Top-k, Top-p with DeepSeek Examples

Code Archaeology

Oct 24, 2025 · Artificial Intelligence

DeepSeek's Three Pillars: MoE, MLA, and Multi-Token Prediction Demystified

This article explains DeepSeek's three core innovations—DeepSeekMoE with fine-grained experts and shared experts, Multi-Head Latent Attention (MLA) for compressed KV-cache, and Multi-Token Prediction (MTP) for faster training convergence—detailing how each reduces compute costs and memory overhead while maintaining model performance.

DeepSeekKV-Cache OptimizationLLM Architecture

0 likes · 5 min read

DeepSeek's Three Pillars: MoE, MLA, and Multi-Token Prediction Demystified

Code Archaeology

Oct 20, 2025 · Artificial Intelligence

LLaMA 2 & 3 Deep Dive: Architecture Innovations (RMSNorm, RoPE, GQA) and Post-Training Pipeline

This article explains Meta's LLaMA model series, detailing LLaMA 2's architectural innovations—RMSNorm, SwiGLU, rotary positional encoding, and grouped-query attention—and LLaMA 3's post-training pipeline including supervised fine-tuning, reward modeling, rejection sampling, and direct preference optimization.

DPOGrouped-Query AttentionLLM Architecture

0 likes · 10 min read

LLaMA 2 & 3 Deep Dive: Architecture Innovations (RMSNorm, RoPE, GQA) and Post-Training Pipeline

Code Archaeology

Oct 13, 2025 · Artificial Intelligence

GPT vs BERT: How In-Context Learning Replaced Fine-Tuning in LLMs

This article traces the evolution from GPT-1's pre-training and fine-tuning paradigm to GPT-3's in-context learning, contrasting GPT's decoder-only generative approach with BERT's encoder-only bidirectional understanding, and explains zero-shot, one-shot, and few-shot learning.

BERTFine-TuningGPT

0 likes · 7 min read

GPT vs BERT: How In-Context Learning Replaced Fine-Tuning in LLMs

Code Archaeology

Oct 10, 2025 · Artificial Intelligence

Transformer Explained: Self-Attention, Positional Encoding & Encoder-Decoder Code

This article explains the Transformer architecture from "Attention Is All You Need", covering self-attention, multi-head attention, layer normalization, positional encoding, and the encoder-decoder structure with PyTorch code, showing how Transformers enable parallel sequence processing without recurrence.

Attention Is All You NeedEncoder-DecoderLayer Normalization

0 likes · 10 min read

Transformer Explained: Self-Attention, Positional Encoding & Encoder-Decoder Code

Code Archaeology

Sep 30, 2025 · Artificial Intelligence

RNNs, LSTM, and Attention: The Building Blocks Behind LLMs

This article explains how recurrent neural networks handle variable-length sequences, why LSTM's gated memory cells solve long-term dependency issues, and how attention mechanisms improve translation by weighting encoder hidden states, laying the groundwork for Transformer-based LLMs.

Attention MechanismGRULSTM

0 likes · 7 min read

RNNs, LSTM, and Attention: The Building Blocks Behind LLMs

Code Archaeology

Sep 28, 2025 · Artificial Intelligence

How LLMs Turn Text into Numbers: Tokenization & Embeddings Explained

This article explains how natural language is converted into neural network inputs, covering tokenization with BPE using DeepSeek's 128K vocabulary, special tokens, and word embeddings via Word2Vec CBOW/Skip-gram, demonstrated with cosine similarity examples from a 7B model's 4096-dimensional embedding layer.

BPEDeepSeekLLM

0 likes · 12 min read

How LLMs Turn Text into Numbers: Tokenization & Embeddings Explained

Code Archaeology

Sep 22, 2025 · Artificial Intelligence

Convolutional Neural Networks Explained: From Parameter Explosion to ResNet

This article explains convolutional neural networks from first principles, detailing convolution operations, pooling, key parameters, and a PyTorch implementation, while tracing historic models from LeNet to ResNet that shaped modern computer vision.

AlexNetComputer VisionConvolutional Neural Networks

0 likes · 10 min read

Convolutional Neural Networks Explained: From Parameter Explosion to ResNet

Code Archaeology

Sep 19, 2025 · Artificial Intelligence

Neural Networks Explained: Why ReLU Beats Sigmoid & How to Build One in PyTorch

This article explains neural network fundamentals, covering why logistic regression units form neurons, the problems with Sigmoid activation in hidden layers (vanishing gradients, non-zero centered outputs), why ReLU and its variants are preferred, how Softmax enables multi-class classification with cross-entropy loss, and provides a complete PyTorch implementation example.

PyTorchReLUSigmoid

0 likes · 8 min read

Neural Networks Explained: Why ReLU Beats Sigmoid & How to Build One in PyTorch