Overview of Prominent Large Language Models and Instruction Fine‑Tuning Techniques

Since the release of ChatGPT, large language models (LLMs) have proliferated, making it difficult to distinguish their origins, capabilities, and relationships. This article offers a concise taxonomy of well‑known LLMs to clarify the landscape.

1. Basic Language Models – Models pretrained solely on massive text corpora without instruction or downstream fine‑tuning. Most are decoder‑only (GPT‑style), though some follow encoder‑decoder (T5‑style) or hybrid (GLM‑style) designs.

T5 (Google) treats every NLP task as a text‑to‑text problem using an encoder‑decoder Transformer, enabling unified multitask learning. GPT‑3 (OpenAI) scales the decoder‑only architecture to 175 B parameters and introduces in‑context learning for few‑shot performance. LaMDA (Google) focuses on dialogue with 137 B parameters and a quality‑safety‑groundedness metric suite.

Other notable models include Jurassic‑1 (AI21 Labs, 178 B/7 B), MT‑NLG (Microsoft + NVIDIA, 530 B), Gopher (DeepMind, up to 280 B), Chinchilla (DeepMind, 70 B, optimized for compute‑efficient scaling), PaLM (Google, 540 B, trained on Pathways), U‑PaLM (Google, UL2R‑fine‑tuned PaLM variants), OPT (Meta, open‑source series up to 175 B), LLaMA (Meta, 7‑65 B), BLOOM (BigScience, 176 B open‑source), GLM‑130B (Tsinghua + Zhipu AI, bilingual 130 B), and ERNIE 3.0 Titan (Baidu, 260 B, the largest Chinese monolingual model).

2. Instruction‑Fine‑Tuned Language Models – By framing tasks as natural‑language prompts, instruction tuning improves zero‑shot generalisation. Examples include T0 (HuggingFace + 42 researchers, T5‑based multi‑task fine‑tuning), FLAN (Google, LaMDA‑based instruction tuning), Flan‑LM (scaled FLAN on T5, PaLM, U‑PaLM), and multilingual variants BLOOMZ and mT0 .

OpenAI’s evolution from GPT‑3.5 (code‑fine‑tuned Codex, InstructGPT) to ChatGPT (RLHF‑aligned) and GPT‑4 (multimodal, 1‑trillion‑scale) demonstrates the impact of reinforcement learning from human feedback. Alpaca (Stanford) shows that a 7 B LLaMA model can match GPT‑3.5 performance using self‑instruct data, while ChatGLM (Zhipu AI) adapts the GLM‑130B backbone for Chinese‑English dialogue with LoRA‑based fine‑tuning.

The article concludes with an extensive bibliography of primary papers, model cards, and code repositories, encouraging readers to explore the cited resources for deeper technical understanding.