PEFT adaptation methods — LoRA, QLoRA, DPO panel grill

Expected question

"Walk me through LoRA vs QLoRA vs full fine-tuning — when would you pick each, and what breaks in production?"

Variant forms

"Explain LoRA — why is it cheap?"
"What are rank and alpha?"
"LoRA vs QLoRA vs full fine-tune — trade-offs?"
"Why DPO after SFT?"
"Can you fine-tune new facts into the model?"
"Merge LoRA weights vs serve adapters dynamically?"
"When does LoRA underperform full fine-tuning?"

Where this actually gets asked

High-frequency archetype in 2025–2026 LLM engineer and Staff AI platform loops — reported across public prep guides (fine-tuning clusters, LoRA/QLoRA deep dives). Not attributed to a single company's leaked question set.

Org grounding (O): DomainForge S3/S4 · peft_gpu.json · ADR-019/020

Executive summary

30-second thesis

LoRA freezes the base and trains a low-rank update (W = W₀ + BA). QLoRA stores the base in 4-bit NF4 and trains adapters in higher precision — the workhorse for 7B–70B on one GPU. Full fine-tune is for when low-rank structure isn't enough; in enterprise triage I default to RAG for facts + PEFT for behavior, not stuffing policies into weights.

2-minute answer

I'd start with the decomposition: only B and A are trainable; rank r controls expressiveness, alpha scales the adapter into the forward pass. QLoRA adds NF4 base weights + paged optimizers — memory drops roughly 4× on the base at the cost of dequant overhead and setup complexity.

For when to pick what: QLoRA SFT (DomainForge S3) when JSON schema and intent codes must stick; DPO (S4) when SFT still emits plausible-wrong intents or bad citations in the JSON envelope. Full fine-tune when the task needs large weight movement — I name it, then say why PEFT+RAG is usually the enterprise economics winner.

Deployment fork: merge adapters for lowest inference overhead on a single variant; multi-LoRA serve when many tenant adapters share one base — with max_loras and eviction as the failure mode.

Honesty: published org receipt is real QLoRA SFT + DPO on Mistral-7B-Instruct-v0.3, rented L4 (peft-20260904T055541Z). That is training config/timing — not an adapter quality/win-rate score, and not always-on production serve.

What I'd refuse: an unregistered adapter in prod, or claiming QLoRA receipts we don't have. Clock: lock LoRA vs QLoRA vs full FT for this constraint.

Migration and rollout

Adapters are artifacts. An unregistered LoRA in prod is the incident.

Pick the method for this constraint (QLoRA vs LoRA vs full FT) and pin the dataset.
Register the adapter before any serve path can see it. No sidecar files on a box.
Merge vs multi-LoRA is a second decision — prove eviction/wrong-adapter before many tenants share a base.
Promote only with a task slice, not a train-loss screenshot. Receipts are config/timing, not win-rate.
Rollback = previous registered adapter or merged weights. Kill switch = unload the adapter id.

Org ownership and operating model

Adaptation / training owns method choice and dataset pin.
Serving owns merge vs multi-LoRA and eviction.
Eval owns the slice that promotes.
Platform owns the registry — unregistered is deny.
Honesty owns not upgrading a rented-L4 receipt into a fleet SLO.

What I'd ask them

Fact change or behavior change?
Serving path — merged weights or multi-LoRA?
Who promotes an adapter?

Quantitative trade-offs

Method	Train memory (illustrative H)	Quality vs full FT	When to reverse choice
Full FT	Highest (optimizer on all params)	Best ceiling	PEFT within 1–2 pts on task + 10× cost savings
LoRA fp16	Low (adapters only)	Near-full for many tasks	Need QLoRA to fit base on GPU
QLoRA	Very low (4-bit base)	Slight gap vs fp16 LoRA	Task needs precision-sensitive base updates
DPO after SFT	+ preference data cost	Fixes format/intent errors SFT misses	Preference data too noisy — fall back to SFT + rules

Deep dive 1: LoRA math (whiteboard)

Write W' = W₀ + (α/r)·BA. Explain intrinsic dimensionality intuition: fine-tuning updates often lie in a low-dimensional subspace. Target modules typically q_proj/v_proj (or all linear layers for quality). Staff+ signal: connect rank choice to ablation, not defaults from a tutorial.

Deep dive 2: QLoRA innovations (name three)

NF4 quantization for normally-distributed weights; double quantization of quant constants; paged optimizers to CPU. Interviewers use this to separate "ran PEFT" from "understands memory story."

Deep dive 3: DPO vs RLHF (alignment stage)

SFT teaches the format; DPO optimizes preferences without a separate reward model pipeline — cheaper than classic RLHF for many enterprise format-repair tasks. DomainForge S4 compares win-rate vs S3 on preference pairs.

Deep dive 4: Facts in weights trap

Fine-tuning memorizes stale SOPs; RAG gives fresh cites. Bitext labels teach intent grammar — not copied into the vector store as memorization targets (ADR-019).

Staff+/Principal rubric

Senior: names LoRA/QLoRA; mentions RAG vs fine-tune at high level.
Staff+: rank/alpha, merge vs multi-LoRA serve, separate eval dimensions, DPO after SFT.
Principal: economics tree, honesty on receipts, registry/promotion gates, org-wide fact vs behavior split. Who inherits: adapter owners. Fail-closed: unregistered adapter never serves. Clock: lock LoRA vs QLoRA vs full FT for this constraint; skip reciting every PEFT paper.

Follow-up questions to expect

"What do you skip in 45 minutes?" Reciting every PEFT paper. Cover method, economics, promotion.
"What rank would you pick for JSON triage?" — Start r=8–16; ablate on golden schema_pass; watch VRAM on 7B QLoRA.
"How do you detect catastrophic forgetting?" — Hold-out general tasks + domain eval; compare S0/S3/S4 ladder.
"Show me a failed fine-tune." — Preference noise, eval leakage, or trying to RAG-replace with PEFT.