Playbook / AI system design / Design a unified query engine across dispersed data sources (email, calendar, docs, chat)

Design a unified query engine across dispersed data sources (email, calendar, docs, chat)

Expected question

"When is my flight to Denver, and did Priya reply about the offsite budget? Design the system that answers this across Gmail, Calendar, Drive, and Slack-like chat — correct, fresh, permission-safe, at consumer scale."

Variant forms

Interviewers often ask the same design with different framing — recognize the archetype:

  • "Design a personal-assistant search layer over email, calendar, docs, and chat for one user."
  • "How do you decide what to index centrally vs. query live from the source API?"
  • "Design ACL enforcement across sources with wildly different permission models."
  • "Our federated answer used a stale calendar event — how do you bound staleness per source?"
  • "Design cross-source result fusion when scores aren't comparable (BM25 vs. cosine vs. structured hits)."
  • "How do you route a multi-part question to the right subset of sources without fanning out to all of them?"

Where this actually gets asked

Reported from Google GenAI interview loops, 2025–26. Distinct from 02 RAG platform at scale: that entry assumes you index everything centrally under one access-control model. This question's core tension is federate vs. index across sources with different freshness needs, different ACL models, and no shared owner — a different design problem than scaling one corpus.

Executive summary

30-second thesis

I'd split sources by what they need: index what's expensive to search and slow-changing with ACL-aware sync, and federate what's small, structured, and freshness-critical straight from the source API — then say out loud which side each source lands on and why.

2-minute answer

A central index is fast but stale by ingest lag, and it has to mirror ACLs, which drifts. Live federation is perfectly fresh and delegates permission enforcement to the source, but is slow under fan-out and expensive per query. Calendar's "next event" and a fresh inbox search are wrong if they lag — those federate. Long-lived docs and chat history are expensive to search live at low latency — those get indexed with ACL-aware sync.

A lightweight router maps the query to a source subset before anything fans out; fan-out-to-all is a cost and latency disaster at consumer scale, and routing misses are the top quality killer, not retrieval quality. For indexed sources, ACLs are stored with the documents, filtered at query time, and re-verified at read time — the index can be stale, and showing a revoked document is a security incident, so the second check is non-negotiable. For federated sources, the user's delegated OAuth means the source enforces permissions directly.

Cross-source scores aren't comparable — BM25 from mail, cosine from docs, a structured calendar hit — so fusion goes through a learned reranker over unified features (source type, recency, entity overlap, sender frequency) rather than arithmetic on raw scores, with reciprocal rank fusion as the zero-training fallback. Synthesis cites claims back to source objects and refuses rather than guesses when retrieval comes up empty — a false negative is recoverable, a hallucinated personal fact is not.

What I'd ask them: Single account or multi-account/workspace? What's the acceptable staleness window per source class? Is a wrong "no answer found" worse than a slow federated call?

Requirements

Functional

  • Answer natural-language, potentially multi-part questions over email, calendar, documents, and chat for one user (or one account/workspace).
  • Route each query to the right source subset instead of fanning out to everything.
  • Cite every claim in the answer back to a specific source object with a deep link.

Non-functional

  • Never show content, or a citation, the user isn't currently authorized to see — re-verified at read time, not trusted from a stale index.
  • Freshness SLO per source class; disclose staleness in the answer when a source exceeds its SLO.
  • Never fuse results across accounts or workspaces without explicit user intent.
  • Refuse rather than fabricate when retrieval returns nothing relevant.

Core entities

  • Source: one connector (mail, calendar, docs, chat) with its own auth model, freshness characteristics, and index-or-federate classification.
  • Indexed item: chunked, embedded content with ACL metadata and lineage back to the source object.
  • Federated query: a typed connector call (e.g. calendar.list, mail.search) issued live at request time.
  • Route decision: the subset of sources selected for one query, logged for quality mining.
  • Citation: a claim in the synthesized answer, linked back to one source object.

API / interface

POST /v1/query
Authorization: Bearer <user-oauth>
{ "text": "when is my flight to Denver and did Priya reply about the offsite budget" }
→ 200 {
  "answer": "Your flight to Denver is Thursday 7:40am (United). Priya hasn't replied to the offsite budget thread yet.",
  "citations": [
    {"source": "calendar", "deep_link": "..."},
    {"source": "mail", "deep_link": "...", "note": "no reply found as of query time"}
  ],
  "sources_queried": ["calendar", "mail"],
  "staleness": {"calendar": "live", "mail": "live"}
}

Staff+ callout: sources_queried is returned even on success — it's the debugging signal for routing misses, which are logged and mined from "no answer found" sessions.

Data Flow

Rendering architecture diagram…

High-level design

Rendering architecture diagram…

Two-lane retrieval keeps freshness-critical sources (calendar next-N-events, recent inbox) off the index entirely — they're queried live, in parallel with the indexed lane, under a per-source timeout.

Deep dive 1: query routing

A lightweight classifier (or small LLM) maps the query to a source subset; fan-out-to-all is a cost and latency disaster at consumer scale. Routing errors are the top quality killer, above retrieval quality itself — log every route decision, mine misses from "no answer found" sessions, and keep a cheap broad-search fallback for low-confidence routes rather than trusting the classifier blindly.

Deep dive 2: permissions across two enforcement models

Indexed sources store ACLs with the document, filter at query time, and re-verify at read time before showing content — the index can be stale, and skipping the second check turns a stale index into a security incident. Federated sources delegate enforcement to the source via the user's OAuth grant, which is itself a strong argument for federating anything permission-volatile instead of trying to keep a mirrored ACL in sync.

Deep dive 3: freshness and staleness disclosure

Change-data-capture per source — webhooks/push where offered, cursor polling elsewhere — with a freshness SLO per source class. When a source's lag exceeds its SLO, the synthesized answer surfaces staleness explicitly ("as of 5 minutes ago") instead of presenting a possibly-outdated fact as current.

Deep dive 4: cross-source fusion and grounded refusal

Scores from different sources aren't comparable — BM25 from mail vs. cosine similarity from docs vs. a structured calendar hit. Normalize through a learned reranker over unified features (source type, recency, entity overlap, personal-salience signals like sender frequency) rather than doing arithmetic across incompatible score scales; reciprocal rank fusion is the reasonable zero-training fallback. Synthesis cites at the claim level with deep links back to source objects, and refuses rather than guesses when retrieval is empty — a false negative is recoverable, a hallucinated personal fact is trust-fatal.

What's expected at each level

  • Mid-level: single-source search per query type, no fusion story.
  • Senior: query routing to multiple sources, basic citation.
  • Staff+: explicit federate-vs-index classification per source with the freshness/permission reasoning stated, read-time ACL re-verification.
  • Principal: learned cross-source fusion over unified features, staleness disclosure as a product decision (not just a metric), and multi-account isolation handled as a hard boundary rather than an afterthought.

Follow-up questions to expect

  • "The confirmation email is in a promotions folder your router skips." Routing recall vs. cost trade-off; a broad fallback pass on low-confidence or empty results.
  • "Calendar API rate-limits you at peak." Per-source budget, short-TTL caching of structured reads, degrade with staleness disclosure rather than failing the query.
  • "How do you evaluate this end-to-end?" Synthetic, privacy-safe personal corpora; routing accuracy, retrieval recall, and answer faithfulness measured as separate metrics; online: abandonment and reformulation rates.
  • "Multi-account / workspace + personal?" Per-account isolation boundaries; never fuse across accounts without explicit user intent.
← Design a distributed GPU training job scheduler with preemption and checkpointing