How I'd run a 45-minute Architecture Review Board
Expected question
"You've been asked to chair the AI Architecture Review Board. How do you run the 45 minutes so it isn't a slide theater or a rubber stamp?"
Variant forms
- "How do you keep ARB from becoming a Principal bottleneck?"
- "What has to come to ARB vs what product can ship without us?"
- "Walk a review of a team that wants to bypass the gateway for launch."
- "How do you record a decision so it survives you?"
Executive summary
30-second thesis
I'd review irreversible seams, not every PR. Forty-five minutes: decision on the table, constraints, what we refuse, who owns the leftover risk. If every change needs my emoji, I failed.
2-minute answer
Pre-read or we cancel. One page: what is changing, which invariant it touches (identity, side-effect gateway, access-before-rank, eval promote, tenancy), the default I'd pick, the discarded alternative, rollback.
In the room: I talk last. Product and the on-call owner talk first. I ask what happens when the model is wrong, who can stop the write, and how we know next week. We leave with approve / approve-with-expiry / reject / needs-data — not "looks good, let's sync."
What I'd refuse: ARB as a gallery of architecture diagrams. Also refuse a standing exception with no owner and no date. Exceptions expire or they become the architecture.
Lucid scar: autonomy-first vs HITL was a real fight. The useful artifact was a written risk tier, not a winning speech (P judgment). Public ADRs are how I practice that habit (O).
The 45 minutes
| Min | What |
|---|---|
| 0–5 | Confirm the decision and the invariant it touches. If neither exists, adjourn. |
| 5–20 | Proposer walks constraints, discarded options, failure mode. |
| 20–35 | Cross-exam: identity, write path, eval, rollback, who is on-call. |
| 35–42 | Decision + residual risk owner + expiry if exception. |
| 42–45 | Write the three-line record in the ticket before anyone stands up. |
What must come to ARB
- New irreversible tool or a bypass of the gateway
- New corpus / tenant isolation change
- Promote path that skips an eval gate
- Fine-tune that might create GPAI / training-data duties
- Anything that changes "who can say no"
What does not: prompt tone, a new dashboard tab, a model swap inside an already-gated route.
Skeptical follow-ups
- "Won't this slow launch?" One ungated refund tool is slower than a 45-minute review.
- "What if you're on PTO?" A named deputy with the same invariants. If there is no deputy, the board is a personality cult.
- "How do you measure ARB?" Exception rate, expired exceptions actually closed, incidents that cite a skipped review — not number of meetings.
Staff+/Principal signal rubric
- Senior: has a review checklist.
- Staff+: timebox, written decision, exception expiry.
- Principal: decides what never comes to ARB, and the org still holds the invariants. Time-bound exceptions. You talk last.