Design a real-time wait-time prediction service with an agent and interactive provenance

Expected question

"Build an AI-powered 'Find Me a Table' service for a consumer dining app: ingest real-time signals (reservation platforms, foot traffic, weather, local events), use an autonomous agent to recommend the restaurant with the shortest current wait for a user's location/cuisine/party size, and let the user ask 'Why?' and get a data-backed, non-hallucinated answer."

Variant forms

Interviewers often ask the same design with different framing — recognize the archetype:

"Design a real-time wait-time or ETA prediction service (restaurants, theme parks, DMV, urgent care) with an agent that ranks options."
"Add an explainability layer to an existing recommendation agent — the user must be able to ask 'why this one?' and get a grounded answer."
"Design the feature pipeline behind a live wait-time estimate that has to react to a stadium event ending nearby."
"Your agent recommends restaurant A over B — how do you prove the recommendation isn't hallucinated?"
"Design quantile-regression serving for a heavy-tailed target with a scarce, proxy-only label."

Where this actually gets asked

Asked firsthand: Workday, Principal AI Engineer, system design round (Aug 2026). This is a composite that tests four muscle groups in one hour — streaming ingestion, tabular ML with a label-scarcity problem, deterministic agent orchestration, and per-prediction explainability — and the last one is the differentiator: almost no candidate walks in with a rehearsed answer for grounding a natural-language "why" in the actual feature values that drove one specific prediction. That is where this question separates Staff+ from Principal.

Executive summary

30-second thesis

I'd keep the ranking agent deterministic — the LLM only turns structured evidence into natural language, never into a ranking decision — and I'd persist a provenance record at inference time so "why?" needs no recomputation and every claim in the answer traces back to a real feature value.

2-minute answer

Two-stage model: a historical baseline wait curve per restaurant × hour × party-size, plus a real-time residual correction from live signals (foot traffic, weather, nearby events). I'd pick a GBDT for the residual, not a deep model — tabular signals, low latency, and native SHAP explainability, which here is a hard requirement, not a nicety. Quantile regression gives P50/P90 instead of a point estimate, because wait times are heavy-tailed and a confident wrong number erodes trust faster than an honest range.

The hard part is the label: true wait times are almost never observed. I'd treat seated-timestamp deltas from reservation platforms, user check-in confirmations, and staff-reported waits as trust-weighted proxies, and build the feedback loop as a first-class pipeline, not an afterthought.

The agent is a deterministic loop — retrieve candidates, batch-score, filter hard constraints, rank on utility — because ranking must be reproducible, cheap, and auditable; an LLM-as-ranker gives up all three. Every prediction persists model version, feature values, SHAP attributions, and fired signal flags. "Why?" runs a grounded LLM over only that record, followed by a claim validator that rejects any sentence that doesn't map to a stored field — that's what makes "data-backed, not invented" true instead of aspirational.

What I'd ask them: Point estimate or interval? Free-form "why" dialogue or one auditable justification? Do we have any ground truth, or only proxies? What's the interactive latency budget?

Requirements

Functional

Ingest real-time signals from reservation platforms, foot-traffic sensors, weather, and local events, per restaurant.
Given location, cuisine, and party size, recommend the restaurant with the shortest current wait that satisfies hard constraints (accepts the party size, open, walk-ins if required).
Answer "why?" for any recommendation with a grounded, auditable justification — not an invented one.

Non-functional

Interactive latency: under ~500ms end-to-end for a recommendation (H).
~25K restaurants, millions of MAU, low-thousands QPS at peak dinner hour (H).
Predictions as quantiles (P50/P90), not point estimates.
Every "why?" answer must be traceable to a persisted evidence record — no re-derivation, no free generation.

Core entities

Signal event: one record from one source (reservation update, foot-traffic tick, weather, event feed), normalized to a common schema.
Restaurant feature vector: online-store row joining baseline-curve position with live residual features.
Prediction: P50/P90 wait estimate, model version, timestamp.
Evidence bundle / provenance record: SHAP attributions plus fired signal flags plus feature values tied to one prediction — the unit the interrogation layer reads.
Recommendation: ranked candidate list, chosen restaurant, evidence bundle reference.

API / interface

POST /v1/recommendations
{ "location": {"lat":40.727,"lng":-74.001}, "cuisine": "italian", "party_size": 2 }
→ 200 {
  "restaurant_id": "r_carbone",
  "wait_minutes": {"p50": 8, "p90": 15},
  "trace_id": "tr_...",
  "evidence_id": "ev_..."
}

GET /v1/recommendations/{trace_id}/why
→ 200 {
  "answer": "The Yankees game just ended nearby, driving up midtown waits. Carbone is below its typical Friday crowd and accepts walk-ins for parties of 2.",
  "grounded": true,
  "claims_validated": 3,
  "claims_rejected": 0
}

Staff+ callout: evidence_id is written once, at prediction time, by the prediction service — the interrogation layer only reads it. That ordering is what makes "why" fast and auditable instead of a second, possibly-different inference.

Data Flow

Rendering architecture diagram…

High-level design

Rendering architecture diagram…

Two-stage retrieval keeps the hot path cheap: candidate retrieval by geo/cuisine index first, then batch scoring only the shortlist against the prediction service — never scoring all 25K restaurants per request.

Deep dive 1: the label problem (lead with this)

True wait times are almost never observed directly. Proxies: seated-timestamp deltas from reservation platforms, user check-in confirmations ("was the wait accurate?"), and staff-reported waits where available. Each proxy has different trust and bias — weight labels by source reliability rather than pooling them uniformly, and treat the confirmation loop as a pipeline with its own SLOs, not a nice-to-have. Cold start for new or sparse restaurants backs off to neighborhood / cuisine / price-tier priors (hierarchical pooling).

Deep dive 2: grounded "why" and the claim validator

The naive design generates the "why" answer straight from the LLM with the evidence bundle as context and calls it done. That's not enough: a fluent model can still add a plausible-sounding clause that isn't in the evidence. The claim validator parses the generated answer into discrete claims and checks each one against a field in the provenance record; unsupported claims are dropped or the answer is regenerated. Faithfulness is evaluated as claim-to-field precision (validator pass rate) plus human eval on helpfulness — not a vibe check.

Deep dive 3: SHAP stability and signal exposure

Raw SHAP values can flip between very similar predictions, which makes them confusing and occasionally misleading to expose directly to end users. I'd aggregate correlated features into signal groups before exposing anything, and prefer thresholded "fired flags" (event_ended: Yankees, 0.8mi, T-12min) over raw attribution scores in the user-facing explanation — SHAP stays internal, in the provenance record, as the source of truth the flags are derived from.

Deep dive 4: event-driven load and adversarial labels

A stadium event ending nearby can thermal-spike dozens of restaurants' predictions at once — handle it with event-driven cache invalidation of affected predictions and pre-computed event-impact features, and degrade gracefully to baseline curves if the residual service is overwhelmed. Separately, a restaurant that inflates reported waits to reduce walk-ins is an adversarial label — trust-weight sources and run anomaly detection on reported-vs-inferred divergence rather than trusting self-reported waits at face value.

What's expected at each level

Mid-level: ingestion → model → agent pipeline, weak or missing "why" story.
Senior: quantile predictions, basic SHAP, agent does simple filtering.
Staff+: deterministic agent (no LLM-as-ranker), persisted provenance at inference time, a claim validator as the mechanism that makes "grounded" true rather than merely asserted.
Principal: the label-scarcity problem framed and solved first, SHAP-stability handling for user-facing explanations, adversarial-label defense, and event-driven degradation under load.

Follow-up questions to expect

"Your SHAP attributions flip between similar predictions — do you still show them to users?" Aggregate into signal groups, expose thresholded fired-flags instead of raw values.
"A restaurant games the system by inflating reported waits." Trust-weight sources; anomaly detection on reported-vs-inferred divergence.
"50K people just left a stadium — thermal spike on nearby restaurants." Event-driven cache invalidation, pre-computed event-impact features, graceful degrade to baseline curves.
"How do you evaluate the 'why' answers themselves?" Claim-to-provenance-field precision as the faithfulness metric, plus human eval on helpfulness; log every rejected claim.