Task types panel — classification, regression, JSON, eval regression
Expected question
"You said classification — is that sklearn, or LLM intent? And what about regression in your stack?"
Variant forms
- "Design tabular fraud classification at scale."
- "What's the difference between data drift and concept drift?"
- "How is eval regression different from model regression?"
- "When does an SLM beat a frontier model?"
- "How do you score format adherence vs factual grounding?"
Where this actually gets asked
Crossover round — ML platform engineers moving into LLM, or Principal architects bridging classical MLOps and LLMOps. Tabular classification/regression archetypes are standard in MLOps interviews (SharpSkill, techinterview.org drift/skew guides). LLM intent/JSON classification appears in applied AI support-triage designs.
Org grounding (O): DomainForge triage · golden-eval-registry · Model Plane taxonomy
Executive summary
30-second thesis
Use precise task vocabulary. DomainForge intent classification = LLM/PEFT JSON triage (~27 intents), not tabular softmax. Tabular classification/regression = classical ML platform archetype in my playbook — registry, feature store, batch/online — not a live sklearn service in the org. Eval regression = golden fixture CI gates (ADR-014) — not linear regression.
2-minute answer
For support triage I'd describe three LLM task types: (1) intent classification into a fixed schema, (2) structured JSON generation with format_validate, (3) RAG-grounded answers with chunk_ids. Eval separates schema_pass from faithfulness — lumping them hides regressions.
If they pivot to classical ML: design a unified prediction platform — offline store for training, online store for serving, feature transformations defined once (Feast-style) to kill training–serving skew. Registry holds champion/challenger aliases; promote only with offline eval + shadow/canary evidence.
Drift: data drift = input P(X) shift (PSI common); concept drift = P(Y|X) change. Retrain only when challenger beats champion under governance — not on every alert.
Bridge to LLM plane: adapters register like model versions; gateway records tokens; golden-eval gates both classical and LLM releases.
Taxonomy table (say this in panels)
| Label | What it means here | Live in org? |
|---|---|---|
| Intent classification (LLM) | JSON intent codes from triage | Yes — DomainForge |
| Structured JSON / schema task | schema_pass eval dimension | Yes — S3/S4 |
| Tabular classification | Fraud/churn softmax/logistic | Playbook only |
| Tabular regression | Numeric forecast models | Playbook only |
| Eval regression | Fixture/metric CI gates | Yes — golden-eval |
Deep dive 1: LLM "classification" without sklearn
Intent router + PEFT teach discrete labels in JSON, not a separate classifier head. Retrieval grounds facts; adapter holds action grammar. Panel trap: calling it "multi-class classifier" without mentioning schema and RAG separation.
Deep dive 2: Classical classification/regression design (whiteboard)
Functional: batch + online scoring, model registry, feature parity, monitoring.
Non-functional: p99 latency for online; reproducible offline training; audit promote history.
Name: registry (mlops-llmops/01), skew (02), drift/shadow (03), batch/online (04), ML CI/CD (05).
Deep dive 3: Eval regression story (org proof)
Running golden-eval CI for the first time found a bug in the fixture itself — that's the kind of "eval regression" I mean. Different from predicting house prices.
Staff+/Principal rubric
- Senior: conflates LLM triage with "classification" vaguely.
- Staff+: separates task types; names skew + drift; separate JSON vs faithfulness metrics.
- Principal: honest playbook vs live boundaries; promotion governance; LLM/classical bridge.