Multi-tenant AI architecture

Built to survive tenant one hundred

The first ten tenants forgive a lot. The next ninety do not. Per-organisation tool registry, tenant isolation enforced at the data layer, credentials that live in a vault rather than an environment file, and AI spend metered from the first call.

105+

endpoints behind 13 routers on the platform we are building

5

roles in the access model, enforced at the data layer

0

cross-tenant leakage, proven by a script rather than promised

A facade of balconies, each one separate
Five ways multi-tenant AI fails

The problems that bite at tenant twenty

Every one of these is a configuration decision in week one and a rewrite in month twelve. That asymmetry is the entire argument for getting the architecture right before the tenants arrive.

A tenant leak is one forgotten clause away

Single-database multi-tenant code without row-level security leaks one tenant's data into another's response the first time an engineer forgets to scope a query.

One leak triggers a nonconformity, a disclosure, and a conversation with every enterprise tenant you have.

Per-tenant credentials in environment files

Tenant-owned API credentials end up in environment variables or an unscoped secrets manager. Rotation requires a deploy, and nothing limits the blast radius.

A single compromise exposes every tenant's credentials at once rather than one tenant's.

Every new tenant requires engineering

Tools, models and prompts are hardcoded per customer behind conditionals. Onboarding the twentieth tenant means a deploy; the hundredth means a war room.

Sales close deals you cannot ship, and the roadmap becomes a queue of customer-specific flags.

AI spend is unbounded and unattributed

A runaway agent loop burns model spend before anyone notices, and usage is not attributed per tenant, so it can be neither billed nor capped.

Margin moves without anyone being able to explain it, and pricing becomes guesswork.

Logs and metrics are not tenant-filterable

When one tenant reports a bug, on-call greps plaintext logs that were never indexed by tenant, against dashboards that aggregate everyone together.

Time to resolve a customer incident climbs from minutes to days, and the tenant feels invisible while it does.

The request pipeline

How one request becomes a safe tenant response

Six checks, all of them before the model is called. The ordering is the design: identity decides entitlement, entitlement decides tools, and the cost cap decides whether the call happens at all.

01

Tenant identity

A validated token carrying immutable organisation, plan and region claims, signed at issue time so they cannot be altered downstream.

02

Tool registry lookup

A cached allow-list resolves which tools, models and data sources this tenant is entitled to, invalidated when the plan changes.

03

Agent dispatch

The supervisor graph only ever sees the tenant's resolved tools. State is namespaced, and checkpoints are written under row-level security.

04

Rate limit and cost cap

An atomic check of the sliding-window quota and remaining spend against the plan. Soft caps slow the tenant; hard caps stop them, with a clear message either way.

05

Tool execution

The outbound credential is fetched at call time from the vault under the tenant's scoped path, used once, and dropped from memory.

06

Response and audit trail

A signed, append-only record of prompt, tool calls and response, keyed to the organisation — the artefact an enterprise tenant's security review actually asks for.

The technical trust story

Isolation proven, not promised

A forgotten scope should return zero rows, never somebody else's data.

That is a property you can test rather than a promise you have to accept: our Quote Rabbit demo ships with a script that proves row-level isolation by trying to breach it, and the same pattern goes into what we build for you. When an enterprise tenant's security review asks how isolation is enforced, the answer should be a policy in the database and a test that fails loudly, not a paragraph in a document.

The build

Six capabilities, none of them optional

Each one answers a failure mode above. Once you have enterprise tenants, every one of them will be asked about.

Zero-trust tenant isolation

Row-level security policies keyed on the organisation, vector-store metadata filters enforced in a wrapper the application cannot bypass, and agent state namespaced per tenant. A forgotten scope returns zero rows, never someone else's data.

Per-org tool registry

Each tenant has a row-level allow-list of tools, models, prompts and data sources. The supervisor cannot dispatch to a tool the tenant is not entitled to, because it never sees it.

Per-tenant credential vault

Tenant credentials live in a managed key service under a path scoped to that organisation, fetched at call time, used once and dropped. Rotation is a vault operation, not a deploy.

Fast tenant routing

Identity, registry lookup, rate-limit check and cost-cap check are engineered to stay inside a tight latency budget: cached key sets, and atomic limiter and ledger operations in the cache layer.

Metered from the first call

Every model and tool call emits a usage event tagged with the organisation, the model and the token count, tracked in real time against that plan's cap. Billing and capping read the same ledger.

Per-tenant observability

Traces tagged with organisation, user and run identifiers, with per-tenant dashboards and alert routes, plus a signed append-only audit log the tenant can be shown.

Who builds this with us

The shape of the problem repeats

Five situations, one architecture. The choices made on day one decide whether you scale to a hundred tenants or rewrite at twenty.

Vertical AI startups

Serving dozens of professional-services firms, each bringing its own document corpus, its own single sign-on, and its own audit trail.

Professional-services platforms

Shipping AI copilots to enterprise tenants who require tenant-side identity federation, dedicated keys and exportable audit logs.

B2B SaaS adding AI

Scaling AI features across an existing tenant base, with per-plan entitlements and metering so the AI line item appears on the invoice.

White-label AI

Reselling an assistant to your own customers, each needing their own branding, credential vault and nested organisation model.

Enterprise tenants on shared SaaS

A large customer requires their data in their own cloud account behind their own network, while still consuming the shared control plane.

Implementation

From an empty repository to a multi-tenant platform

Same codebase from one tenant to one hundred — which is the point, and the reason the first four weeks look slower than they should.

Week 1

Architecture and identity

The tenant model is finalised, identity is wired with signed claims, and row-level security policies are drafted for every tenant-scoped table.

Weeks 2–4

MVP with one tenant

Per-org tool registry, a supervisor with isolation-backed state, a rate limiter and a single audit-log table — end to end, for one tenant.

Weeks 5–8

Production guarantees

The credential vault, metered billing wired to the cost ledger, per-tenant observability, and the signed audit trail.

Week 9+

Scale-out

Onboarding a new tenant becomes a configuration change. Enterprise patterns — identity federation, bring-your-own key, a dedicated stack — are wired in as they are needed.

Bring your tenant model

Bring a tenant model, a target plan structure and a date, and we will tell you the shortest honest path — including when your current architecture is fine and the answer is a smaller change than you feared.
Our second practice

This is our AI engineering practice

It is real work and it is where our four products came from. But what Cognilium leads with is narrower: optimization apps that run in tandem with Microsoft Dynamics 365, computing the decisions the ERP records but does not derive — the optimal price, the optimal pick path, the optimal stock level. See the optimization apps · How we build inside the ERP.