A Technical Roadmap of GPT‑3.5: From Pre‑training to RLHF and Emerging Capabilities

A Technical Roadmap of GPT‑3.5

Recent breakthroughs of OpenAI's ChatGPT have sparked intense interest in the AI community; this article aims to dissect the emergent abilities of ChatGPT, trace their origins, and provide a comprehensive technical roadmap that explains how the GPT‑3.5 model series evolved into their current powerful form.

1. The 2020 GPT‑3 Model and Large‑Scale Pre‑training

GPT‑3 demonstrated three core abilities: (1) language generation via a language‑modeling objective, (2) in‑context learning (few‑shot prompting), and (3) world knowledge (factual and commonsense). All three stem from massive pre‑training on ~3 trillion tokens with 175 billion parameters, where the training data composition (C4, WebText2, Books, Wikipedia) supplies the knowledge.

2. From GPT‑3 to ChatGPT (2020‑2022)

OpenAI released the davinci model (July 2020) and subsequently introduced Codex ( code‑cushman‑001 , code‑davinci‑002 ) and instruction‑tuned variants ( davinci‑instruct‑beta , text‑davinci‑001 ). In 2022, supervised instruction tuning produced text‑davinci‑002 , while reinforcement‑learning‑from‑human‑feedback (RLHF) created text‑davinci‑003 and ChatGPT, both derived from code‑davinci‑002 . These steps added instruction following, zero‑shot generalisation, and alignment.

3. Code‑Davinci‑002 and Text‑Davinci‑002: Capabilities

Respond to human instructions rather than merely completing prompts.

Generalise to unseen tasks when the instruction set is sufficiently large.

Code generation and understanding, a direct result of code‑based training.

Chain‑of‑thought (CoT) reasoning, which appears stronger after code training.

Empirical observations suggest that instruction tuning unlocks abilities already present in the base model, while code training likely injects the reasoning power needed for CoT.

4. What Comes From Pre‑training vs. Fine‑tuning?

Language generation, world knowledge, and in‑context learning are rooted in the original GPT‑3 pre‑training. Instruction tuning (both supervised and RLHF) primarily unlocks instruction following and zero‑shot capabilities without adding new knowledge. Complex reasoning seems to emerge as a side‑effect of extensive code training.

5. Summary of Evolutionary Path

The following table (omitted here) summarises how each ability maps to a specific stage: pre‑training → instruction tuning → code training → RLHF. Notably, RLHF does not add new abilities but aligns existing ones, often incurring an “alignment tax” that reduces raw performance while improving safety and factuality.

6. Current Limitations of GPT‑3.5

Inability to revise its own beliefs reliably when presented with contradictory evidence.

Lack of formal reasoning for strict mathematical proofs or first‑order logic.

No built‑in internet retrieval; knowledge is frozen at the training cutoff.

7. Conclusion

The article concludes that GPT‑3.5’s strengths arise from a combination of massive pre‑training, code‑centric training, instruction tuning, and RLHF alignment. It calls for the open‑source community to reproduce this roadmap, thereby increasing transparency and fostering further research on large language models.