Design an EU AI Act operating model for an enterprise agent platform

Expected question

"How would you operate an enterprise agent / RAG platform under the EU AI Act — classification, technical documentation, GPAI duties, and what you refuse to automate?"

Variant forms

• "Is our support agent high-risk? Walk the classification."
• "What goes in the technical file if a regulator shows up?"
• "GPAI vs an application that just calls GPT — who owns which duty?"
• "Design the operating model: Legal, Security, Platform, Product."
• "What would you refuse to automate even if the VP wants it?"
• "How does this differ from NIST AI RMF?"

Where this actually gets asked

Enterprise / regulated Principal and Cloud Architect loops, and any FDE embed into EU customers. Public text is the EU AI Act and Commission GPAI guidance — R. I have a NIST one-pager mapped to live controls (O). I do not have an EU notified-body engagement. Do not invent one.

NIST is US risk language (Govern / Map / Measure / Manage). The Act is classification + obligations + technical file. Different conversation. Don't paste the NIST one-pager and call it Brussels.

Executive summary

30-second thesis

I'd classify the system, not the model: prohibited stays off; high-risk gets a technical file, human oversight, and logging before it touches a decision that hurts a person; GPAI duties sit with the provider unless we fine-tune enough to become one. I will not automate a high-risk decision just because the demo looks clean.

2-minute answer

Start with use case, not vendor. A supplier-exception agent that can block payment is closer to high-risk than a blog drafts tool. I'd write a one-page classification: prohibited / high-risk / limited / minimal / GPAI-provider — named owner, dated, reviewed when the workflow changes.

If it's high-risk: risk management, data governance, logging, human oversight that can actually stop the write, and a technical file a stranger can follow. HITL that rubber-stamps 400 cases an hour is not oversight.

If we only call an API: the provider carries GPAI. If we fine-tune and put our name on the weights, we picked up duties — say that out loud before the PEFT project starts.

What I'd refuse: emotion recognition in the workplace, social scoring, untargeted scrape-to-train, and "the model decided, no human in the loop" on credit, hiring, or safety. Also refuse claiming EU conformity from a NIST slide.

Migration and rollout

You don't "comply the model." You classify workflows and then stand up the file.

1. Inventory production uses. One-page classification each — owner, date, prohibited / high-risk / limited / minimal / GPAI-provider.
2. High-risk: technical file + HITL that can refuse the write, before any autonomy story. Assist-only is the first ship.
3. Fine-tune / put-your-name-on-weights: Legal signs the GPAI-duty change before the PEFT job, not after the blog post.
4. Review the classification when the workflow or model changes — a stale page is a finding.
5. Rollback = previous classification + assist-only. Kill switch = freeze the write, not a NIST slide.

What I'd ask them

• Which workflow would actually be high-risk here — not the model vendor?
• Who signs the classification?
• What will we refuse to automate even if the VP wants it?

Requirements

Functional: classify each production workflow; store the decision; generate a technical-file pack (intended use, data, evals, oversight, logging, residual risk); route high-risk writes through a human who can refuse; record who overrode what.

Non-functional: classification review when the workflow or model changes; logs retained for the Act's window; EU data residency if the customer requires it — residency is not classification.

Core entities

• AI system record — intended purpose, users, countries, decision type
• Classification — prohibited / high-risk / limited / minimal / GPAI
• Technical file — versioned, digest-linked to model + eval suite
• Oversight event — human refuse / accept / escalate
• Residual risk — what we will not automate, signed by product + legal

Deep dive 1: classification is a product decision

The trap is "we use Azure OpenAI so we're fine." The Act cares what the system does to people. I'd score: does this output materially affect someone's rights, money, job, or safety? If yes, assume high-risk until legal says otherwise. If no, still log and label — limited-risk transparency is cheap.

Deep dive 2: technical file without theater

I'd reuse what we already run: eval gates, signed audit, access-before-rank, HITL on irreversible tools (O — pattern, not an EU certificate). The file is: intended use, training/eval data provenance, known failure slices, how a human stops a write, how we roll back a model. If I can't assemble that in a week, we are not ready to sell into the EU.

Deep dive 3: what I refuse to automate

Prohibited is easy. The hard refuse is high-risk with a VP deadline. I'd ship a read/assist path first — draft the exception, human commits. Autonomy is a promotion through gates, not a launch default. Same instinct as the Lucid autonomy reversal (P judgment, not an EU filing).

Org ownership

Follow-up questions to expect

• "What do you skip in 45 minutes?" Claiming EU conformity from a NIST slide. Lock classification, technical file, oversight that can refuse.

Staff+/Principal signal rubric

• Senior: names high-risk vs chatbot.
• Staff+: technical file contents and human oversight that can stop a write.
• Principal: classification as an operating model, GPAI duty split, written list of what we will not automate. Who inherits: product + legal. Fail-closed: high-risk writes don't ship without oversight that can refuse. Clock: lock classification + technical file contents; skip claiming EU conformity from a NIST slide.

Related

• cloud-architecture/05 security & compliance
• NIST AI RMF one-pager
• 24 PRODUCTION_STRICT