Task types panel — classification, regression, JSON, eval regression

Expected question

"You said classification — is that sklearn, or LLM intent? And what about regression in your stack?"

Variant forms

• "Design tabular fraud classification at scale."
• "What's the difference between data drift and concept drift?"
• "How is eval regression different from model regression?"
• "When does an SLM beat a frontier model?"
• "How do you score format adherence vs factual grounding?"

Where this actually gets asked

Crossover round — ML platform engineers moving into LLM, or Principal architects bridging classical MLOps and LLMOps. Tabular classification/regression archetypes are standard in MLOps interviews (SharpSkill, techinterview.org drift/skew guides). LLM intent/JSON classification appears in applied AI support-triage designs.

Org grounding (O): DomainForge triage · golden-eval-registry · Model Plane taxonomy

Executive summary

30-second thesis

Use precise task vocabulary. DomainForge intent classification = LLM/PEFT JSON triage (~27 intents), not tabular softmax. Tabular classification/regression = classical ML platform archetype in my playbook — registry, feature store, batch/online — not a live sklearn service in the org. Eval regression = golden fixture CI gates (ADR-014) — not linear regression.

2-minute answer

For support triage I'd describe three LLM task types: (1) intent classification into a fixed schema, (2) structured JSON generation with format_validate, (3) RAG-grounded answers with chunk_ids. Eval separates schema_pass from faithfulness — lumping them hides regressions.

If they pivot to classical ML: design a unified prediction platform — offline store for training, online store for serving, feature transformations defined once (Feast-style) to kill training–serving skew. Registry holds champion/challenger aliases; promote only with offline eval + shadow/canary evidence.

Drift: data drift = input P(X) shift (PSI common); concept drift = P(Y|X) change. Retrain only when challenger beats champion under governance — not on every alert.

Bridge to LLM plane: adapters register like model versions; gateway records tokens; golden-eval gates both classical and LLM releases.

What I'd refuse: an LLM on a tabular problem we can solve with a calibrated classifier and a cheaper SLO. Clock: lock when not to use an LLM.

Migration and rollout

The migration is stop using an LLM on a tabular decision you can calibrate cheaper.

1. Name the task: class/score vs open text. If it's a decision, start with the cheaper model.
2. Feature parity + registry alias before any LLM "classifier" story.
3. Shadow the challenger; promote only when it beats champion on the owned metric.
4. LLM intent/JSON stays on the schema path — don't mash it into sklearn.
5. Rollback = previous champion alias. Kill switch = freeze the LLM route on tabular.

Org ownership and operating model

• ML platform owns registry, skew, and promote.
• Model owners own task type and the metric.
• Eval owns champion/challenger evidence.
• Product owns when an LLM is allowed to speak vs score.
• FinOps owns the "cheaper SLO" veto.

Taxonomy table (say this in panels)

Deep dive 1: LLM "classification" without sklearn

Intent router + PEFT teach discrete labels in JSON, not a separate classifier head. Retrieval grounds facts; adapter holds action grammar. Panel trap: calling it "multi-class classifier" without mentioning schema and RAG separation.

Deep dive 2: Classical classification/regression design (whiteboard)

Functional: batch + online scoring, model registry, feature parity, monitoring.

Non-functional: p99 latency for online; reproducible offline training; audit promote history.

Name: registry (mlops-llmops/01), skew (02), drift/shadow (03), batch/online (04), ML CI/CD (05).

Deep dive 3: Eval regression story (org proof)

Running golden-eval CI for the first time found a bug in the fixture itself — that's the kind of "eval regression" I mean. Different from predicting house prices.

Staff+/Principal rubric

• Senior: conflates LLM triage with "classification" vaguely.
• Staff+: separates task types; names skew + drift; separate JSON vs faithfulness metrics.
• Principal: honest playbook vs live boundaries; promotion governance; LLM vs classical as a product default with reversal. Who inherits: ML + LLMOps. Clock: lock when not to use an LLM; skip a methods dump.

What I'd ask them

• Labels, imbalance, and leakage risk?
• Latency/cost envelope?
• Is the output a decision (class/score) or open text?

Follow-up questions to expect

• "What do you skip in 45 minutes?" A methods dump. Cover task type, metric, promotion.

Related

• mlops-llmops/04 batch/online prediction
• MODEL_PLANE_PANEL_QUESTION_BANK
• Venkat AI taxonomy tab