Skip to content
Tagged articles

Generative Pre-training

2 articles · Page 1 of 1
Code Archaeology
Code Archaeology
Oct 13, 2025 · Artificial Intelligence

GPT vs BERT: How In-Context Learning Replaced Fine-Tuning in LLMs

This article traces the evolution from GPT-1's pre-training and fine-tuning paradigm to GPT-3's in-context learning, contrasting GPT's decoder-only generative approach with BERT's encoder-only bidirectional understanding, and explains zero-shot, one-shot, and few-shot learning.

BERTGPTGenerative Pre-training
0 likes · 7 min read
GPT vs BERT: How In-Context Learning Replaced Fine-Tuning in LLMs
Network Intelligence Research Center (NIRC)
Network Intelligence Research Center (NIRC)
Jul 29, 2023 · Artificial Intelligence

Getting Started with GPT: How Generative Pre‑Training and Discriminative Fine‑Tuning Work

This article explains GPT's two‑stage learning—unsupervised generative pre‑training on large raw corpora followed by discriminative fine‑tuning on labeled tasks—detailing the underlying Transformer decoder architecture, loss functions, and task‑specific input transformations.

GPTGenerative Pre-trainingNLP
0 likes · 5 min read
Getting Started with GPT: How Generative Pre‑Training and Discriminative Fine‑Tuning Work