Design an agent orchestration platform with real tool use

Expected question

"Design a scalable, enterprise-ready agentic AI platform. How do you architect it for availability, fault tolerance, and scale? Walk me through orchestration, tool use, and governance."

Variant forms

Interviewers often ask the same design with different framing — recognize the archetype:

"Design a multi-tenant AI agent system for 50 enterprise clients."
"How would you scale your EDI agent platform to handle 100× the current load?"
"You are building a document-processing agentic pipeline. Enterprise SLA: 99.9% uptime, 10,000 documents/day at peak. Walk me through the architecture."
"Design infrastructure for agents that call Slack, Jira, and internal APIs — with human approval for side effects."
"How do you prevent one customer's runaway agent from exhausting GPU budget for everyone else?"
"Design a coding-agent platform like background agents — traces can run 30+ minutes with tool loops."
"Architect MCP-style tool access so agents cannot bypass policy or exfiltrate credentials."
"Design multi-agent orchestration: triage agent routes to specialist agents with separate tool surfaces."
"How would you debug and replay a failed mission that touched five tools across three services?"

Where this actually gets asked

A Blind post on Google DeepMind's Applied AI Engineer loop names "agent frameworks (LangChain/LangGraph)" explicitly as an evaluation criterion in its ML-system-design round — the best direct evidence found for this topic. Anthropic's infra-framed interview rounds (per Blind write-ups) are described as testing GPU scheduling and protocol design (MCP), which overlaps meaningfully with this question's tool-use surface. Beyond that, treat "design a platform that lets an LLM take real actions — call APIs, modify records, trigger external side effects — safely" as a fast-emerging archetype across every company shipping agentic products in 2026, rather than a single pinned question.

Executive summary

30-second thesis

I'd split orchestration from governance. The model can propose tool calls; an independent gateway decides allow / HITL / deny — and the agent can't walk around it.

2-minute answer

I'd define mission state, tool schemas, budgets, and idempotency keys up front. Planning lives in one plane; policy, credentials, and audit live in another the model never bypasses. Irreversible actions go through approval.

I'd persist trajectories so I can eval and replay incidents — if I can't answer "who called which tool as which tenant" in five minutes, the design isn't done. Prefer a workflow engine for long deterministic waits; keep the agent for bounded ambiguity.

What I'd refuse: one mega-agent with every tool, or policy checks copy-pasted into each agent. That's how you get seven slightly different "refund allowed?" implementations and no fleet answer.

Quantitative trade-offs

Decision	Trade-off and reversal evidence	Evidence class
Agent vs workflow	Agents handle ambiguous branching; workflows win for durable deterministic steps. Reverse when retries/state machines dominate over open-ended reasoning.	H
Sync tools vs async jobs	Sync keeps UX simple; async protects latency budgets for long tools. Reverse if users need immediate confirmation for high-risk mutations.	H
Fail-closed vs fail-open	Side effects fail closed; read-only assistive paths may degrade. Reverse fail-open only with an explicit product risk acceptance.	H

Numbers and thresholds in interview delivery should be labeled H unless the candidate can defend a measured baseline. Open repositories are O; researched public patterns are R. Do not upgrade O/R into employer P adoption.

Migration and rollout

You're migrating side-effect authority, not just a model endpoint. Treat it that way.

Inventory irreversible tools (refunds, sends, writes to CRM/ERP) and map each to risk tier + HITL owner before any agent can call them in prod.
Put the gateway in front of a shadow path: agent proposes, gateway decides, tool does not execute yet — compare allow/deny/HITL rates against the old inline checks.
Dry-run HITL queues with real approvers on a quiet cohort. If the queue UX can't decide in seconds, fix that before widening — a stuck approval queue is an outage dressed as safety.
Flip execute tokens on for low-risk tools first, then money-moving tools. Kill switch: gateway deny-all for a tool/agent/tenant without redeploying orchestrators.
Rollback is "gateway fails closed / previous allowlist," not "hope the agent graph reverts cleanly."

Org ownership and operating model

Agent platform owns mission state, routing, retries, and trajectory storage.
Governance / security owns the tool gateway, policy versions, credential minting, and signed audit — product teams don't get a bypass flag.
Domain product owners own which workflows exist and what "done" means for a mission.
Approvers (ops / finance / security rota) own HITL SLAs for high-risk tools — if nobody is on the rota, that tool stays deny.
FinOps / platform owns budget meters and halt-on-breach; behavioral kill-switch and money kill-switch are separate levers.

Write exception expiry. Permanent "break-glass allow" is how governance becomes optional under deadline pressure.

Requirements

Functional

Decompose a user request, call tools/APIs, synthesize a final response.
Gate side effects — anything beyond read-only generation is not auto-fire by default.
Multiple specialists with narrow tool surfaces beat one agent that can do everything.

Non-functional

Every side-effecting action is auditable: who/what, which policy version, allow/HITL/deny.
One misbehaving agent can't take down the fleet or walk around policy.
Add tools/agents without redeploying the whole orchestration layer.

Core entities

Agent: bounded task + model + allow-listed tools.
Tool call: intent — tool name + args — not yet an execution.
Policy decision: allow / require-human-approval / deny.
Audit record: signed, immutable log of decisions and executions.

API / interface

Auth: user session + agent identity. Side effects need a short-lived gateway execution token.

POST /v1/missions
Authorization: Bearer <user_token>
{"message":"Summarize overnight incidents and notify Slack","thread_id":"thr_..."}
→ 200 {"mission_id":"mis_...","status":"running|awaiting_approval","interrupt":null|{...}}

POST /v1/missions/{mission_id}/resume
{"decision":"approve","approver_id":"u_..."} → 200 {"status":"running"}

POST /v1/gateway/authorize
Authorization: Bearer <agent_token>
{"mission_id":"mis_...","agent_id":"notifier","tool":"slack.notify","args":{...},"risk_signals":{...}}
→ 200 {"decision":"allow","execution_token":"et_...","expires_in_sec":30}
→ 200 {"decision":"approval_required","approval_id":"ap_..."}
→ 403 {"decision":"deny","reason":"policy_violation","rule_id":"..."}

POST /v1/tools/execute
{"execution_token":"et_...","tool":"slack.notify","args":{...}}
→ 200 {"result":{...},"audit_id":"aud_..."}

Staff+ callout: authorize ≠ execute. Tokens are single-use and audited. Resume is a separate HITL API.

Data Flow

Mission plan → agent work → gateway authorize → (optional HITL) → execute with a short-lived token.

Rendering architecture diagram…

If authorize and execute are the same call, you've already lost the ability to insert HITL without rewriting every tool client.

High-level design

Rendering architecture diagram…

The decision that separates a strong answer from a generic one: orchestration and governance are separate systems. Merging "decide what to do" and "decide if it's allowed" into one agent framework feels simpler on day one. It breaks later because routing changes constantly, while policy changes are compliance events — rare, audited, and not something a prompt tweak should silently rewrite.

I've shipped that split as open proof (O): one plane for missions/routing, one independently deployable gateway for policy/HITL/audit. Every side-effecting call goes through the same door. Don't upgrade that into an employer-production claim.

Deep dive 1: the gateway pattern for side-effecting actions

Hard constraint: side effects fail closed through one enforcement point. Inline per-agent checks don't scale and can't answer "what can any agent in this fleet do right now?"

Naive design: each agent has its own "is refund allowed?" check. Every new agent reimplements it. Audit logs fragment. One buggy specialist becomes a quiet exfiltration path.

Default: a shared gateway SDK. All side effects hit the same policy engine and the same signed audit trail. Actions above a risk threshold need human approval — not an agent-flipable config flag.

Honest trade-off on gateway outage: for money-moving tools I'd rather fail closed. For a soft assistive path, a conservative builtin deny/simulator may beat taking the whole fleet down — but that exception is product-accepted in writing, scoped to enforcement unavailable, never to "policy said no and we ignored it."

Design	Blast radius of a bug	Auditability	Scales to N agents
Inline per-agent checks	High — each bug is independent	Fragmented	Poorly
Shared gateway	One enforcement point to get right	Single trail	Yes

Deep dive 2: protocol surfaces (MCP / A2A) without a second path

Hard constraint: a new protocol surface must be a thin adapter over already-governed logic — never a parallel door that skips the gateway.

Most answers stop at "agents call tools." Stronger: the platform itself speaks protocols both ways. Outbound MCP (agent → external server) still goes through authorize. Inbound MCP (external client → your capabilities) must invoke the same orchestrator/gateway objects HTTP would — not a friendlier bypass.

Same idea for agent-to-agent: discover via a real agent-card/handshake, don't hardcode "service B lives at this URL and we trust it." Guessed routing is how you get two systems that happen to talk without a protocol.

Deep dive 3: containing a misbehaving agent

Hard constraint: one runaway agent must not take the fleet's budget or tool surface with it.

You need two independent levers:

Behavioral kill-switch — block a specific agent, tool, or workflow scope immediately.
FinOps halt — stop dispatch when budget meters breach.

One vague pager that says "something's wrong" is not containment. Behavioral failure and financial failure look different in the first five minutes; treat them that way.

Staff+/Principal signal rubric

Mid-level: orchestrator routes to specialists and calls tools directly.
Senior: separates read-only generation from side effects; some approval gate.
Staff+: orchestration vs governance as separate systems; names fail-open/fail-closed for the gateway; explains why shared gateway beats inline checks.
Principal: protocol surfaces as governed adapters (not bolt-ons); independent behavioral and financial containment; migration that proves HITL queues before money-moving tools go live.

Follow-up questions to expect

"How do you version a policy without breaking agents?" Evaluate against the version active at call time, with a deprecation window. Hard cutover is how you break half the fleet overnight.
"What if the audit log is compromised?" Signed packets (HMAC/KMS). Tamper-evident even if the store is owned.
"How does an approver decide in seconds?" Risk signals + short summary in the HITL queue — not a raw tool payload dump. That's UX dressed as systems.
"45-minute scope?" Mission API + gateway + HITL. Defer full MCP/A2A unless they steer there.

What I'd ask them

Which tools are irreversible today, and who is on the HITL rota?
Fail-closed on gateway outage for money-moving tools — product-accepted?
Budget kill switch: per agent, per tenant, or both?