Multi-tenant deep path

HM already asked isolation. I answer in two altitudes without contradicting myself. Short board first. Then platform scale. Then the glue script so Lucid and open proof don’t bleed.

Path (before the next platform HM)

Glue script I’d actually say

For agents, isolation fails as RAG bleed and confused-deputy tools, so I put AuthZ before ranking and tenant on every tool arg through a gateway. That’s the control-plane answer. At platform scale I’d also decide shared vs dedicated inference, per-tenant token budgets, and residency — same identity spine, different data and model-plane trade-offs. Open proof you can click: Strict/JWT RAG plus gateway HITL. Lucid is production access-aware RAG, not those binaries. I won’t pretend an in-memory registry is multi-tenant SaaS.

Follow-ups I have to survive (say it, don’t outline it)

“A noisy neighbor burns everyone’s TPM.”

Shared inference is fine until a whale eats the budget. I’d put per-tenant token budgets at the gateway first, fair scheduling second, dedicated capacity for the tenants who actually pay for isolation. I wouldn’t shard the control plane on day one.

“EU residency — prompts never leave EU.”

Pin stores and cache keys to the region. Scope tool credentials so an EU tenant’s Stripe key never leaves. Honest limit: the public spine has a thin routing contract and a deny, not a full multi-region SaaS. I’d design that on your stack. I won’t sell the demo as if it already is.

“Per-tenant LoRA / custom prompts.”

Adapter registry keyed by tenant. Never share adapters across tenants. Prompts are data. They inherit the same tenant boundary as chunks.

“How do you bill this?”

Meter at the gateway, not in the model client. Budgets and deny-on-overrun are policy. If billing is reconstructed from GPU logs after the fact, you’ll fight finance forever.

“How do you test?”

Negative evals NT-1 through NT-5 in CI, plus a cross-tenant red team. Happy-path RAG quality is not an isolation test.

“Shared vs dedicated model capacity?”

Default shared, with per-tenant budgets at the gateway. Dedicated or PTU-style capacity when a whale is a noisy neighbor or a contract requires it. I wouldn’t shard the control plane just to look enterprise. Same identity spine either way.

“We already have one shared vector index.”

Then the first ship is a mandatory pre-rank filter plus a negative test that must fail if Tenant B leaks — not a six-month re-index program. Per-tenant collections come when the contract or the leak class demands it. I wouldn’t freeze delivery to boil the index.

“Isn’t this just Postgres RLS?”

RLS is a data-plane control. Agents still leak through caches, traces, prompt injection, and tool args that don’t carry tenant. I’d keep RLS. I wouldn’t skip the gateway or the NT suite.

“How does SSO / enterprise identity plug in?”

JWT (or the equivalent assertion) has to carry tenant_id and principal into retrieve and into the gateway. Demo Principal is labeled Demo. I won’t sell an in-memory registry as Okta. Week one on their stack is identity, not a new vector DB.

“What about prompt injection that says ‘ignore tenant’?”

Treat it as an AuthZ bug, not a prompt-engineering contest. The model never gets to widen scope. Tool args and retrieval filters come from the token, not from the completion.

Rehearsal bar

• Short drill without notes
• Deep 09 outline without reading
• Glue script once
• One timed self-mock on multi-tenant

Related

• STAR + designs
• Private evidence bank — don’t invent Lucid tenant counts