Why the Transformer Core Structure Is the Key to AI Interview Success

What does a Transformer do?

Transformer introduced in “Attention Is All You Need” (2017) replaces recurrent architectures by processing the whole sequence in parallel with self‑attention, allowing the model to capture long‑range dependencies without recurrence.

Overall architecture

Input → Embedding + Positional Encoding → Encoder stack → Decoder stack → Linear → Softmax → Output

Input embedding and positional encoding

Tokens are mapped to dense vectors (e.g., 512‑ or 1024‑dimensional). Because the model has no recurrence, a positional encoding (sinusoidal or learned) is added to each token vector to inject order information.

Encoder stack

The encoder consists of N identical layers (commonly N=6 or 12). Each layer contains:

Multi‑Head Self‑Attention

Position‑wise Feed‑Forward Network

Each sub‑layer is wrapped with a residual connection followed by LayerNorm.

Self‑Attention mechanism

For every token three vectors are computed:

Q (query)

K (key)

V (value)

The attention weight matrix is obtained by the scaled dot‑product of Q and K: Attention(Q,K,V)=softmax((QKᵀ)/√d_k)·V Multi‑head attention splits the embedding dimension into h heads, each performing the above computation independently, then concatenates the results. This enables the model to attend to different relational patterns (e.g., syntactic, semantic, long‑distance) in parallel.

Decoder stack

The decoder mirrors the encoder but adds two mechanisms:

Masked Self‑Attention : applies a causal mask so each position can only attend to earlier positions, enforcing autoregressive generation.

Encoder‑Decoder Attention : queries the encoder’s output, allowing the decoder to incorporate source‑side information while generating.

Each sub‑layer also uses residual connections and LayerNorm.

Output layer

The final decoder representation is projected with a linear layer to the vocabulary size and passed through a Softmax to obtain a probability distribution over tokens. The token with highest probability is emitted as the next word.

Transformer families

Encoder‑Decoder (e.g., T5, BART): both understanding and generation; suited for translation, summarization.

Encoder‑only (e.g., BERT): only the encoder stack; optimized for classification, NER, semantic similarity.

Decoder‑only (e.g., GPT, LLaMA, Qwen): only the decoder stack with causal masking; excels at language generation, few‑shot learning.

Why most large models are Decoder‑only

Unified training objective : next‑token prediction only, simplifying training and scaling data.

Native generation capability : causal masking makes the model inherently autoregressive, ideal for continuation, dialogue, and multi‑turn reasoning.

Easy few‑shot / zero‑shot adaptation : providing a few examples in the prompt steers the model to new tasks without fine‑tuning.

Inference efficiency : only the decoder is needed at runtime, reducing latency and increasing throughput.

Practical answer outline for interviews

Summarize the flow and key distinctions:

“A Transformer converts tokens into embeddings, adds positional encodings, processes them through an encoder stack of multi‑head self‑attention and feed‑forward layers, then (if present) a decoder stack that uses masked self‑attention and encoder‑decoder attention, and finally projects to a vocabulary with a linear‑softmax head. Encoder‑only models (BERT) focus on understanding, encoder‑decoder models (T5) handle both understanding and generation, while modern large‑scale models adopt a decoder‑only design because it unifies the training objective, scales efficiently, and is optimal for generation and few‑shot use.”