ByteDance Interviewer Asks: What Rank r Do You Use for LoRA? I Said 64—He Said I'm Wasting GPU Memory

In a typical AI engineering interview, a candidate is asked about the LoRA rank \(r\) used for fine‑tuning. The interviewee answers "the larger the better" and is immediately corrected, highlighting the risk of guessing without understanding.

Common Mistakes

First pitfall: force‑guessing an answer. The candidate knows that a larger \(r\) means more parameters and tries to sound confident, but this often leads to a wrong answer and a severe impression.

Second pitfall: saying "I don't know" and staying silent. While honest, this wastes the opportunity to demonstrate reasoning and boundary awareness, which interviewers value.

Three‑Step Strategy for Unknown Questions

Honest boundary declaration. Instead of "I don't know," say "I understand up to X, and beyond that I'm uncertain; may I share my current reasoning?" For LoRA, an example opening is: "I have used LoRA for instruction fine‑tuning; I know the rank \(r\) relates to parameter count and over‑fitting risk, but I haven't studied a systematic selection rule. Let me outline my understanding and you can tell me if I'm on the right track."

Derive a logical direction from known facts. Recall that LoRA assumes low‑rank updates: \(A\in\mathbb{R}^{d\times r}\) and \(B\in\mathbb{R}^{r\times k}\). The parameter count is \(d\times r + r\times k\). For example, with \(d=k=4096\):

# LoRA parameter count
# Original weight matrix: d × k parameters (e.g., 4096×4096)
# LoRA decomposition: A (d×r) + B (r×k) parameters, r << d,k
# r=8: 4096×8 + 8×4096 = 65,536 (≈0.4% of original)
# r=64: 4096×64 + 64×4096 = 524,288 (≈3.1% of original)
# Larger r increases parameters and reduces LoRA's lightweight advantage

From this, argue that a smaller \(r\) suffices for simple tasks with limited data, while larger \(r\) may be used for complex tasks but requires regularization to avoid over‑fitting.

Ask a targeted follow‑up question. After the derivation, ask something like "Is my reasoning direction correct? How do you choose \(r\) in practice?" This shows curiosity and turns the interview into a learning dialogue.

Reasoning Beats Memorization

Interviewers understand that candidates cannot memorize every detail; they assess how candidates handle uncertainty. Demonstrating a logical chain from LoRA's low‑rank hypothesis to rank selection shows a bottom‑up understanding that can be transferred to new problems.

When Derivation Is Appropriate

Derivation works for questions that have logical inference space, such as hyper‑parameter selection or design trade‑offs. It does not suit highly specific implementation details that require exact numbers or code, e.g., the exact CUDA kernel of Flash Attention.

Case Study: vLLM PagedAttention

A candidate was asked about the implementation of PagedAttention. Although unfamiliar, he explained the memory‑fragmentation problem of KV‑Cache and hypothesized that the solution mirrors OS paging: allocate fixed‑size blocks on demand. The interviewer confirmed the direction and spent five minutes detailing the actual design, leading to a strong impression and an offer.

Practical Three‑Step Response Template

Boundary positioning (≈10 s). "I understand up to X, beyond that I'm uncertain; may I share my current understanding?"

Logical derivation (1–2 min). Walk through known concepts step by step, linking them to the question.

Closing with a good question (≈10 s). "Is my direction correct? How do you choose the parameter in practice?"

Combining honest boundary setting, reasoned inference, and a thoughtful question can turn an unknown question into a memorable interview moment.