PEFT adaptation methods — LoRA, QLoRA, DPO panel grill
Expected question
"Walk me through LoRA vs QLoRA vs full fine-tuning — when would you pick each, and what breaks in production?"
Variant forms
- "Explain LoRA — why is it cheap?"
- "What are rank and alpha?"
- "LoRA vs QLoRA vs full fine-tune — trade-offs?"
- "Why DPO after SFT?"
- "Can you fine-tune new facts into the model?"
- "Merge LoRA weights vs serve adapters dynamically?"
- "When does LoRA underperform full fine-tuning?"
Where this actually gets asked
High-frequency archetype in 2025–2026 LLM engineer and Staff AI platform loops — reported across public prep guides (fine-tuning clusters, LoRA/QLoRA deep dives). Not attributed to a single company's leaked question set.
Org grounding (O): DomainForge S3/S4 · peft_gpu.json · ADR-019/020
Executive summary
30-second thesis
LoRA freezes the base and trains a low-rank update (W = W₀ + BA). QLoRA stores the base in 4-bit NF4 and trains adapters in higher precision — the workhorse for 7B–70B on one GPU. Full fine-tune is for when low-rank structure isn't enough; in enterprise triage I default to RAG for facts + PEFT for behavior, not stuffing policies into weights.
2-minute answer
I'd start with the decomposition: only B and A are trainable; rank r controls expressiveness, alpha scales the adapter into the forward pass. QLoRA adds NF4 base weights + paged optimizers — memory drops roughly 4× on the base at the cost of dequant overhead and setup complexity.
For when to pick what: QLoRA SFT (DomainForge S3) when JSON schema and intent codes must stick; DPO (S4) when SFT still emits plausible-wrong intents or bad citations in the JSON envelope. Full fine-tune when the task needs large weight movement — I name it, then say why PEFT+RAG is usually the enterprise economics winner.
Deployment fork: merge adapters for lowest inference overhead on a single variant; multi-LoRA serve when many tenant adapters share one base — with max_loras and eviction as the failure mode.
Honesty: published org receipt is T4 fp16 LoRA micro on TinyLlama — valid operator proof, not a claim that every panel depth run is 7B QLoRA+DPO in prod.
Quantitative trade-offs
| Method | Train memory (illustrative H) | Quality vs full FT | When to reverse choice |
|---|---|---|---|
| Full FT | Highest (optimizer on all params) | Best ceiling | PEFT within 1–2 pts on task + 10× cost savings |
| LoRA fp16 | Low (adapters only) | Near-full for many tasks | Need QLoRA to fit base on GPU |
| QLoRA | Very low (4-bit base) | Slight gap vs fp16 LoRA | Task needs precision-sensitive base updates |
| DPO after SFT | + preference data cost | Fixes format/intent errors SFT misses | Preference data too noisy — fall back to SFT + rules |
Deep dive 1: LoRA math (whiteboard)
Write W' = W₀ + (α/r)·BA. Explain intrinsic dimensionality intuition: fine-tuning updates often lie in a low-dimensional subspace. Target modules typically q_proj/v_proj (or all linear layers for quality). Staff+ signal: connect rank choice to ablation, not defaults from a tutorial.
Deep dive 2: QLoRA innovations (name three)
NF4 quantization for normally-distributed weights; double quantization of quant constants; paged optimizers to CPU. Interviewers use this to separate "ran PEFT" from "understands memory story."
Deep dive 3: DPO vs RLHF (alignment stage)
SFT teaches the format; DPO optimizes preferences without a separate reward model pipeline — cheaper than classic RLHF for many enterprise format-repair tasks. DomainForge S4 compares win-rate vs S3 on preference pairs.
Deep dive 4: Facts in weights trap
Fine-tuning memorizes stale SOPs; RAG gives fresh cites. Bitext labels teach intent grammar — not copied into the vector store as memorization targets (ADR-019).
Staff+/Principal rubric
- Senior: names LoRA/QLoRA; mentions RAG vs fine-tune at high level.
- Staff+: rank/alpha, merge vs multi-LoRA serve, separate eval dimensions, DPO after SFT.
- Principal: economics tree, honesty on receipts, registry/promotion gates for adapters, org-wide fact/behavior split.
Follow-up questions to expect
- "What rank would you pick for JSON triage?" — Start r=8–16; ablate on golden schema_pass; watch VRAM on 7B QLoRA.
- "How do you detect catastrophic forgetting?" — Hold-out general tasks + domain eval; compare S0/S3/S4 ladder.
- "Show me a failed fine-tune." — Preference noise, eval leakage, or trying to RAG-replace with PEFT.