Data Classes, Processing Lanes and Tokenisation
Classify every model request by its data, route it only to a processing lane the class permits, keep derived artefacts inside their source class, and treat retention, roles and tokenisation as recorded, testable decisions.
- Version
- 0.1.0
- Owner
- DEIXEI
- Last reviewed
- Reading time
- 33 min
For Cto · Ai Architect · Platform Engineer · Security Engineer · Data Leader · Dpo · Risk And Assurance · Internal Audit
Sources and details
- Section
- Build the platform
- Source works
- Book 1 · Book 3 · Supporting Tools · Research Series
- Source chapters
- 10, 12, 13, 16
- Source appendices
- 01:D
- Tool specifications
- audit_evidence_schema@0.1.0
- Refresh trigger
- vendor release
A language model has to see data to work on it, and it often sees that data on infrastructure the firm does not operate. That turns two governance questions into design-time and run-time questions: how confidential is what is in this request, and where, and by whom, may it be processed. This page is the implementation guide for answering both on every call. It applies Working Paper 09 and the tokenisation discipline of Book 1 Chapter 13 and Book 3 Chapter 10.
Three things make AI different from ordinary application classification (Paper 09 §1): the request, not the application, carries the data; processing often happens outside the firm; and AI systems manufacture copies (chunks, embeddings, caches, traces, memory, evaluation sets). Classification and location therefore have to be decided at design time and enforced per call, at the model gateway.
Standing of the content. The classes, flags, lanes, matrix and inheritance rules are the author’s design proposal in Paper 09, not a survey result and not any firm’s policy; your own information-classification policy takes precedence. The seven-question tokenisation model is an author recommendation from Book 1. Bank A is a composite institution used in the books. Vendor statements are dated and marked for verification. Statements about what GDPR, the Swiss FADP, the EU AI Act or DORA require, and any regulatory dates, live on the regulatory map; this page gives design consequences only and is not legal advice.
Decisions, owners and artefacts
The owner assignments below are this page’s recommendation, assembled from the roles the sources name (Paper 09 places jurisdiction approval with the firm and its DPO and the C4-on-Lane-B exception with the risk committee; Book 3 Ch 10 PR-1 has the matrix co-signed by the CDO, DPO, second line and procurement; Book 1 Ch 13 §7 has the privacy officer countersign tokenisation papers). Align them with your own RACI.
| Decision | Decided by | Recorded in | Re-reviewed when |
|---|---|---|---|
| Map C1–C4 and the five flags onto the firm’s existing classification scale | CDO with the DPO and CISO | Information-classification policy | Classification refresh; new regulation |
| Approved jurisdictions for inference and storage | Firm, with the DPO, recorded once | Policy, read by the gateway | Supervisory or client commitment changes |
| Lane of each model deployment | Platform/model governance, from vendor terms | Model catalogue entry per deployment | Model version change; vendor term change; sub-processor change |
| Permission matrix cell values | Risk and data owners; the C4-on-Lane-B exception by the risk committee | Gateway policy plus signed matrix | Quarterly and on every class, route or vendor change |
| Approved masking, redaction, pseudonymisation and aggregation services | Data protection with platform | Service register; versioned, tested | Each release of the service |
| Tokenisation decision per invocation type | Product team, countersigned by the privacy officer | One-page decision paper per invocation type | Route, model, data or task change |
| Controller/processor and provider/deployer roles per arrangement | DPO and counsel | Processing record and contract file | New vendor, mandate or route |
Classify the information: four classes and five flags
Four confidentiality classes
The class answers a single question: how much damage results if the information reaches the wrong recipient (Paper 09 §3.1). Map the codes onto the scale the firm already runs; do not introduce a parallel one.
| Class | Meaning | Typical financial-services content |
|---|---|---|
| C1 Public | Approved for publication | Published product terms, factsheets, disclosures, public market data |
| C2 Internal | For employees; limited harm if disclosed | Intranet, IT procedures, training material |
| C3 Confidential | Disclosure causes material harm | Non-public positions, pricing and underwriting models, individual customer and policy records, lending files, board papers |
| C4 Strictly confidential | Highest harm; need-to-know | Unannounced transactions, recovery and resolution planning, key material, supervisory correspondence, bulk customer data sets, medical evidence in claims files |
Five category flags
Flags mark information that is regulated because of what it is about. A flag can tighten the handling of its class; it can never loosen it (Paper 09 §3.2). Paper 09 proposes the single-letter codes below for the classification code, written as class plus flags, for example C3+P or C4+S (§10); the suffix notation is a suggestion.
| Flag | Code | Marks | Design consequence |
|---|---|---|---|
| Personal data | P | Information about an identifiable natural person | Lawful ground and stated purpose; a processor contract with every party that sees the data; retention rules for prompts, traces and memory |
| Sensitive personal data | S | Special-category data, criminal-offence data and, for Swiss entities, the wider Swiss sensitive-data list | Handle at least as C4; impact assessment expected; no vendor-API lane without explicit approval |
| Client-identifying data | K | A client relationship under a statutory or contractual confidentiality duty, including corporate clients | Pseudonymise or tokenise before any lane outside firm-controlled infrastructure; log every access |
| Inside information | I | Precise, non-public, price-sensitive information about issuers or instruments | Information barriers enforced in retrieval and memory; no shared index or memory across a barrier |
| Regulated records | R | Information with a statutory retention period | Traces and outputs archived under the retention schedule, not the logging default |
Two operating rules follow. First, the sensitive flag is set by a classifier at the gateway, not by the designer, because free-text systems meet special-category data unplanned (payment narratives, fraud files, voice biometrics, claims). Second, keep class and flags separate: the class decides where data may go; the flags decide what else must be true when it gets there (contract, barrier, retention schedule, impact assessment). A published customer testimonial is C1+P; a deal team’s working list can be C4+I with no personal data at all (§3.3).
Reconcile the scales you will meet
The corpus uses more than one scale. Do not blend them; map each to the firm’s policy once and record the mapping.
| Scale | Where it appears | Values | How this page treats it |
|---|---|---|---|
| C1–C4 plus five flags | Paper 09; classification code of Paper 01 | Four classes, flags P/S/K/I/R | The model this page follows. |
| Bank A five-tier refresh (composite) | Book 3 Ch 10 §8 | Public; Internal; Confidential split into non-customer and customer-financial; Restricted; Strictly Confidential | A worked example of a firm’s own scale with stricter route rules (below). |
Dictionary data_classification | Book 1 Ch 16 §5.1, field 4 | public / internal / confidential / restricted, plus sensitivity tags | Map C1–C4 to the four values; carry flags as sensitivity tags. |
Audit Evidence Schema dataEntry.classification | AES v0.1.0 | public, internal, confidential, restricted, regulated | Single scale with no flags. The page’s recommendation: map C1–C4 to the first four and decide explicitly whether regulated represents the R flag; record that decision in the classification policy so evidence and routing agree. |
Composite example — Bank A’s refresh (Book 3 Ch 10 §8, PR-1). The bank kept five tiers but split Confidential, because the old definition swept too much together. Confidential — non-customer (strategy, internal financials) was permitted on every Claude route. Confidential — customer-financial was permitted only with in-region inference or gateway tokenisation. Restricted (customer PII, transaction records, credit-decision inputs) was permitted only with tokenisation, with a cleartext exception through in-region Bedrock, customer-managed keys and second-line sign-off per workflow. Strictly Confidential was removed from the Claude estate at source rather than routed. On this page’s reading, every Claude route in that matrix (Direct API and Claude Platform on AWS as Lane A; Foundry, Bedrock and Vertex AI as Lane B) is Lane A or Lane B, so the categorical exclusion is a firm choice stricter than Paper 09’s default (which allows C4 on Lane C). The book reports illustrative traffic shares after go-live, explicitly flagged by the author as representative rather than measured.
Assign a processing lane to every model deployment
A processing lane is the route a request takes to a model, defined by two facts: who operates inference and who can see the data (Paper 09 §4.1).
| Lane | Description | Operates inference | Can see prompts and outputs |
|---|---|---|---|
| A — Vendor API | Model developer’s or aggregator’s API under an enterprise contract, no training on customer data, limited retention | Model vendor | Model vendor, during processing and any contractual retention |
| B — Private cloud tenancy | Vendor model through the firm’s own cloud tenancy, approved region, private endpoints, customer-managed keys where available | Cloud provider | Cloud provider; typically not the model developer |
| C — Controlled infrastructure | Open-weight or licensed model on infrastructure the firm controls or that is dedicated to it | The firm, or a provider acting solely on its instructions | No model vendor |
Assign the lane per deployment, never per vendor or console (§4.2). The same model family sits in different lanes by route. As of 10 November 2026, per Paper 09 §4.2: Anthropic’s documentation describes Anthropic as processor on its own API and certain marketplace offerings, and the cloud provider as processor on Amazon Bedrock and the Google-managed offering; Microsoft states that for models it designates as sold by Azure prompts and completions are not available to the model providers, a statement limited to that designation. Lane B can still hold where the weights run in the cloud provider’s service account rather than in your tenancy (Paper 09 cites Amazon Bedrock), because both defining facts hold. [FACT-CHECK: re-verify the Anthropic, Microsoft and AWS processor statements against current vendor documentation before relying on them.]
Procedure. For each deployment in the catalogue:
- Answer the two defining questions from the vendor’s current terms and data-processing addendum, not from marketing pages.
- Record inference location and storage location separately, and whether the vendor may change either. Test both against the firm’s approved-jurisdictions list.
- Record the retention configuration that the lane depends on, and the test that proves it is still set (see zero data retention).
- Record the model version. Re-run steps 1–3 whenever the version changes, because retention terms can change with the model.
- Do not store the lane on the application. Paper 09 makes the lane derived: the gateway computes permitted lanes from class and flags; the catalogue says which deployments sit in which lane (§10).
Where inference runs — vendor detail as of 10 November 2026, per Paper 09 §6.2. [FACT-CHECK: all four rows are current product claims; re-verify against official documentation before use.]
| Platform | Location behaviour reported | Lane consequence |
|---|---|---|
| Microsoft Azure (Foundry) | Data at rest stays in the designated geography for all deployment types; Global may process prompts in any region; Data Zone within a Microsoft-defined zone that Microsoft may extend; Standard and Regional Provisioned within the customer-specified geography. Two Microsoft pages describe the EU zone edge differently. | Only geography-bound deployment types keep inference in one jurisdiction. |
| Amazon Bedrock | Geographic cross-Region inference profiles stay within a geography; global profiles may route worldwide. Where a model requires retention, retained content is stored in the Region that processed it. | Prefer geographic profiles for residency; record where retention lands. |
| Google Cloud | Regional or multi-regional endpoints keep the durable cache in location; the global endpoint may serve from any region. | Avoid the global endpoint for residency-bound data. |
| Anthropic direct API | Inference geography global (default) or us; workspace storage us only. Book 3 Ch 10 §5.2 reports the same as of 3 August 2026: no eu inference geography. | For an EU or Swiss firm, the direct API is a cross-border lane; a cloud regional endpoint is a different lane with different answers. |
Apply the class × lane permission matrix
The matrix joins class to lane and is enforced per request. It is Paper 09’s default (§5.1); align cell values with the firm’s policy and have them signed.
| Use of data | C1 Public | C2 Internal | C3 Confidential | C4 Strictly confidential |
|---|---|---|---|---|
| Allowed lanes | A, B, C | A, B, C | B, C. Lane A only with an approved contract and after approved redaction | C. Lane B only with explicit approval and customer-managed keys |
| Prompts and context | Yes | Yes | Yes, need-to-know | Yes, named users only |
| Retrieval indexes and embeddings | Shared indexes | Shared, with entitlement filters | Permission-aware, separated by domain | Dedicated index per use case |
| Agent long-term memory | Allowed | Allowed, with retention limit | Session-scoped by default | Not allowed |
| Agent actions (ceiling) | Autonomous within envelope | Autonomous within envelope | Act with approval | Draft only; a human executes |
| Generated code in a sandbox | Allowed | Allowed | Allowed; read-only mounts, no network egress | Only in a Lane C sandbox |
| Fine-tuning and evaluation sets | Allowed | Allowed | Data owner’s approval; synthetic preferred | Synthetic or masked only |
| Trace and log retention | Standard | Standard | Restricted access; content redacted | Metadata only; content in a secured store |
| Development and testing | Real data | Real data | Masked or synthetic | Synthetic |
How to read it (§5.1 notes):
- Agent actions is a ceiling, not an entitlement. A system still earns each action scope on recorded behaviour, starting at read-only or draft (bounded agent). C4 caps scope at draft however good the record.
- Flags tighten the column. S moves the request into the C4 column whatever its class. I adds a barrier check to whichever column applies. K adds pseudonymisation before Lane A or B.
- The C3-on-Lane-A exception is narrow. It applies only after an approved redaction or pseudonymisation service has run; what leaves is the redacted request. It exists because some tasks need a model only available as a vendor API, and a controlled exception beats an uncontrolled one.
- The C4-on-Lane-B exception is a risk-committee decision, not a platform-team one. Customer-managed keys protect data at rest; they do not hide a prompt from whoever runs inference.
- Risk tier is independent of class. A credit-scoring model on pseudonymised data can be high-risk; a summariser of C4 board papers need not be. Classify risk separately on risk tiers.
- Archetype defaults are the matrix without exceptions. Paper 02’s archetype cards state default lanes; a team needing either exception is asking for something the card does not grant and should expect to justify it (§10).
Classify each request at the highest class of all its inputs
The gateway, not the application, makes the per-request decision (Paper 09 §5.2):
- Collect every input: the user’s text, retrieved passages, tool results and memory entries.
- Classify at the highest class among them, plus the union of flags. There is no “mostly C2” prompt; one C4 passage pulled into context changes the lane for everything else in it.
- Ask whether an approved redaction or pseudonymisation service may run for this task. If yes, run it before the lane decision; the service, not the agent, sets the resulting class and records what it removed.
- Select a lane permitted for the resulting class and flags, then a deployment in that lane that meets the approved-jurisdictions test.
- If no lane is permitted, block and log. Show the user that the call was blocked and why; an invisible block teaches people to route around the gateway.
- Classify the gateway itself at the highest class it handles. It sees raw prompts and responses; its logs follow the trace-retention row for that class.
Evidence per request (minimum). The resulting class and flags, inputs that set the class, the masking service and version that ran and what it removed, permitted lanes, the deployment selected or the block reason, and the correlation ID that joins this decision to the audit evidence record. The AES dataEntry object requires operation, dataset_id and classification for each dataset an action touched.
Align the model registry with the dictionary. Book 1 Ch 16 §8 names a fifth data anti-pattern: the model usage registry’s data_classes_touched and the dictionary’s data_classification drift apart, and an agent invokes a model with classes it is not approved for. Run the cross-reference query at registration time.
Keep derived artefacts inside their source class
The matrix is easy to defeat without four inheritance rules (Paper 09 §5.3), which the author calls the part of the scheme to defend hardest:
- Derived artefacts inherit the highest class of their sources. An embedding of a C4 document is C4; an adapter fine-tuned on C3 data is C3 and may be served only where C3 may go.
- Outputs start at the class of their inputs. Only a named human or an approved classifier may lower an output’s class; by default, summarising a confidential file yields a confidential summary.
- One context window, one class. If classes mix in a single context window, the whole request takes the highest of them.
- Down-classification only through approved redaction, pseudonymisation or aggregation services, never inside the agent’s prompt. “Remove any personal data before answering” is not a control: the model has already received the data on whatever lane it runs.
| Derived artefact | Inherits from | Where the rule is enforced | Owner check |
|---|---|---|---|
| Chunks, embeddings, indexes | Source documents | Grounding data API stamps the store’s class on every result | Index build manifest lists source classes |
| Caches and prompt logs | The request class | Gateway and logging pipeline | Retention follows the trace row of the matrix |
| Traces and evaluation sets | The request or dataset class | Observability and evaluation pipelines | C3+ content redacted; C4 metadata only |
| Agent memory | Highest class written to it | Memory store policy | C3 session-scoped; C4 prohibited |
| Fine-tuned adapters | Training data class | Model catalogue serving rule | Adapter’s lane limited to its class |
| Generated outputs | Inputs (rule 2) | Output labelling, e.g. sensitivity labels on responses | Lowered only by an approved classifier or a named human |
| Rehydration maps | The original values (page synthesis of rule 1) | Held inside firm-controlled infrastructure | See rehydration-map placement |
| Coding-agent transcripts on endpoints | Whatever entered the session | Endpoint baseline (below) | Workstation named as a processing location |
Endpoint-held history is in scope. Book 3 Ch 10 §5.3 reports, as of 3 August 2026, that Claude Code keeps plaintext session transcripts under ~/.claude/projects/ for 30 days by default, tunable through cleanupPeriodDays, with no documented way to disable local persistence. A firm with zero data retention on the server can still hold weeks of prompts and outputs in cleartext across thousands of laptops. Treat the fleet as a records-retention and legal-hold scope: name the workstation as a processing location in the record of processing, bring the transcript directory into endpoint DLP scope, set a shorter cleanup period (Bank A, composite, uses 7) and pin sign-in to the covered organisation. Book 3’s landing-zone integration checklist (Ch 12 PR-1, item 17) sets the acceptance evidence: local conversation history is named in the records-retention schedule, the legal-hold and device-wipe procedures explicitly cover Claude transcripts, and the DPO signs the endpoint retention policy. When a legal hold is issued, its scope therefore has to include the transcript directories on every device in the affected population, not only server-side stores. The full baseline is on the coding-agent workstation page. [FACT-CHECK: transcript location, default period and whether persistence can now be disabled, against current Claude Code documentation.]
Erasure depends on rule 1. A firm that cannot list the derived artefacts of a client’s data cannot delete them at the end of a mandate. Maintain the source-to-derived map on the data and retrieval page.
Pseudonymisation lowers class, not flag. Pseudonymised data remains personal data for whoever holds the key. A pseudonymisation service may lower the request’s class for routing, but the firm keeps the P flag; whether the recipient’s position changes is for counsel (§5.3).
Assign controller, processor and AI Act roles per arrangement
Roles decide which contracts each lane needs. Paper 09 states them as an architect’s working hypotheses for the DPO and counsel to confirm arrangement by arrangement (§7). The legal text and article references are on the regulatory map.
Three organising propositions (author’s reading; the first two need counsel’s confirmation):
- An agent is never a party. It has no legal personality; its processing is the firm’s, and accountability sits with its named owner.
- Every vendor that sees personal data needs a role; the default hypothesis is processor or sub-processor. A vendor that uses the data for its own purpose becomes a controller for that processing, which is why “no training on our data” is a role-defining term, not a confidentiality preference.
- Roles shift one level when the firm acts on a client’s instructions. In outsourced administration or white-label services the firm may be processor and its AI vendors sub-processors that the client must accept.
| Party | Data-protection hypothesis | EU AI Act role | Third-party lens |
|---|---|---|---|
| Firm, own purposes | Controller | Deployer of systems it uses; provider of systems it builds and puts into service under its name | Remains accountable for outsourced functions |
| Firm, on a client’s instructions | Processor for that client (subject to counsel) | Deployer | Client contract and sub-processor acceptance govern AI use |
| Intragroup service company | Typically processor | Depends on who develops and uses | Intragroup outsourcing |
| Cloud provider (Lane B) | Processor | Depends on offering | ICT third-party arrangement with audit, location and exit terms |
| Model vendor API (Lane A) | Processor or sub-processor; controller for any own-account purpose | Provider of a general-purpose AI model | Third-party and concentration risk |
| Open-weight developer (Lane C) | No role on these facts; receives no data | Provider of a general-purpose AI model | Supply-chain and licence risk |
| SaaS or tool server receiving data | Processor | Possibly provider of an AI system | Assessed per service |
| External data vendor | Independent controller or licensor | — | Licence and quality terms |
| Agent | None | Part of the AI system | Named owner accountable |
Design consequences when the firm is controller (Paper 09 §7.3, right-hand column): the declared purpose is a field in the AI inventory and is enforced by tool scopes and retrieval filters; inventory entries generate or link to the processing record; no lane exists without an approved processor contract; vendor sub-processor lists are monitored and a change triggers review; every lane approval records destination and transfer mechanism; an impact-assessment template exists per solution type and is completed per use case; a human decision point sits wherever a design approaches a solely automated decision with significant effects; the incident process covers prompt, trace and memory stores and asks vendors the same question; retention applies to traces, caches, indexes and memory, not only source systems. Where the firm is under both EU and Swiss regimes, build the incident process to the stricter clock; Paper 09 also flags that the Swiss processor conditions can decide the lane for client-identifying data before any other analysis.
Platform rules when the firm is processor for a client (§7.4):
- Process the client’s data only on its documented instructions; no reuse for the firm’s purposes.
- Never use one client’s data to fine-tune a shared model, populate shared memory or enrich a shared index.
- Isolate each client in retrieval indexes, memory and traces, with tenant-scoped storage and keys (store per tenant; see grounding-store isolation).
- Pass the same obligations on to every AI vendor; a new model or tool vendor becomes a sub-processor only once the client has authorised it.
- Support the client with data-subject requests, incidents and audits, including evidence from AI traces.
- Delete or return the client’s data at the end of the service, including embeddings, caches and memory.
The grounding data API is the natural place to implement rules 2, 3 and 6 as a shared platform service.
Hold vendors to minimum terms before assigning a lane
Minimum terms (Paper 09 §8.1; the author offers them as a design proposal, not as a substitute for a contract checklist):
| Term | Why it decides the lane |
|---|---|
| No training or product improvement on the firm’s data | Keeps the vendor a processor and stops leakage through the model |
| Stated retention period or zero data retention, with every exception written down | Controls copies outside the firm; exceptions are where surprises are |
| Processing location for inference and storage, separately, and whether the vendor may change it | Transfer analysis and approved jurisdictions |
| Sub-processor list with notice of changes | Sub-processor authorisation |
| Whether vendor staff may review content, when, and where they are located | Human review of content is itself a disclosure |
| Security attestations and audit and information rights | Due diligence on the processor; access for supervisors |
| Breach notification within a fixed period | The firm’s own notification clock depends on it |
| Model deprecation notice period | Time to evaluate a replacement and to exit |
| Exit and deletion assistance | Keeps the arrangement reversible |
Paper 09 reads every Lane A or B deployment as an ICT third-party arrangement belonging in the DORA register, and notes that these terms overlap with DORA’s key contractual provisions without a clause-by-clause mapping; that mapping is legal work. Sign the matrix (Bank A, composite: CDO, DPO, second-line reviewer and procurement lead) and review it quarterly and on every class change, route addition or vendor sub-processor update (Book 3 Ch 10 PR-1).
Treat zero data retention as recorded configuration
Paper 09’s conclusion from four vendors’ documentation is that zero data retention is a property of a configuration, not of a vendor: whether it holds depends, on each platform, on a mode being set, a feature being disabled or an approval being granted (§8.2). Record, per deployment, the configuration that makes the claim true and a test that proves it is still set.
Vendor documentation snapshot, as of 10 November 2026 per Paper 09 §8.2. Documentation is not the contract; the contract and the data-processing addendum govern. [FACT-CHECK: each cell is a current vendor claim; verify against official documentation and your signed terms.]
| Topic | Microsoft (models sold by Azure) | Amazon Bedrock | Google Cloud | Anthropic (own API) |
|---|---|---|---|---|
| Training | Not used to train foundation models without permission or instruction | Pages opened state model providers have no access to prompts, completions or logs; no training statement in those words | No training or fine-tuning without prior permission or instruction | No training on commercial inputs or outputs by default; feedback and opt-in are exceptions |
| Retention and review | Abuse monitoring may sample content for automated and authorised human review; approved customers can have modified abuse monitoring | Retention mode per account and Region; none writes no request or response data to durable storage; some models require up to 30-day retention with possible AWS review | Input/output caching up to 24 hours on by default for supported models; abuse-monitoring logging possible; documented switches reach zero retention | Deletion within 30 days by default; zero retention on request; flagged content kept up to two years; some models require 30-day retention and are excluded from zero retention |
Claude-specific detail, as of 3 August 2026 per Book 3 Ch 10 §5. [FACT-CHECK: re-verify every item against Anthropic’s current “API and data retention” and Claude Code “Zero data retention” documentation.]
- Eligible surfaces. On the direct API, zero retention covers the Messages API, the Token Counting API and Claude Code with commercial keys. It does not cover Console, Workbench, Managed Agents, the Files API, the Batch API, Code Execution, Agent Skills, the MCP connector or Claude for Excel. A Messages API workflow that uses the MCP connector for one tool call drops out of zero retention for that call; the owner reads the eligibility table for every surface the agent touches.
- Enablement is per organisation and separate from the Enterprise plan, follows an eligibility review, cannot be switched on from admin settings, and is audit-logged. A group with three organisations has three enablements to evidence.
- Coverage follows the authenticated organisation. A developer signed into a personal account or another organisation’s key is outside it. Pin clients with the managed settings
forceLoginMethodandforceLoginOrgUUID; treat that pair as a baseline control. - It costs functionality, enforced server-side. Web and desktop cloud sessions, Artifacts, feedback submission and Remote Control are disabled; some models are unavailable because their class requires retention (Book 3 names one). Put the retention posture and its model constraints in the agent envelope.
- “Zero” has carve-outs: retention to comply with law or combat misuse, and up to two years for policy-flagged sessions. Name both in the impact assessment and the records schedule.
- On cloud routes Anthropic’s zero retention does not carry over. On Bedrock, Vertex AI and Foundry the cloud provider is processor and its terms govern: configure its retention mode and keys, opt out of improvement programmes and document the result. Book 3 reports that on Claude Platform on AWS Anthropic operates the platform outside the AWS security boundary, with zero retention on request.
- Two Anthropic pages differ. Paper 09 notes the privacy-centre article states deletion within 30 days while the developer documentation states content is not retained by default (with 30-day exceptions); Book 3 states 30-day standard retention. The page draws no conclusion: the contract governs.
Retention is not a transfer control. Book 3 Ch 10 §5.2 notes that every direct-API call from the EU is an EU-to-US transfer requiring its own instruments (Standard Contractual Clauses, Module 2, with a documented transfer impact assessment, and the EU–US Data Privacy Framework where the vendor is certified, verified annually). Zero retention reduces copies; it does not legitimise the transfer. Instrument detail and status belong on the regulatory map.
Zero-retention configuration record (page recommendation, synthesised from Paper 09 §8.2 and Book 3 Ch 10): deployment and route; organisation or account identifier; the settings, modes or approvals that make retention zero; enablement evidence; the surfaces in and out of scope; carve-outs; endpoint posture; the test that verifies it; last verified date; model versions excluded.
Review model upgrades and lanes together. Retention terms can change with the model version, so a quality-driven upgrade can change the firm’s retention position without anyone deciding to (§8.2). Make lane re-assessment a mandatory step of every model change in model governance and change governance.
Decide tokenisation per invocation, not as a corporate default
Where tokenisation sits relative to the matrix
The sources come at tokenisation from two angles; this page layers them rather than blending them.
- Paper 09 (floor). The matrix and flags set mandatory transforms: K requires pseudonymisation or tokenisation before Lanes A and B; C3 on Lane A requires approved redaction first.
- Book 1 Ch 13 (decision within the floor). The tokenisation gate decides per invocation which fields go plain, which are tokenised, which are redacted and where the rehydration map is held. It argues against both always tokenise and never tokenise.
Difference to note. Book 1’s worked example (Bank A relationship-management copilot, composite) sends client names in plain form to an in-tenant Swiss deployment, which is Lane B, because relationship awareness is the substance of the task. Under Paper 09, client-identifying data on Lane B needs pseudonymisation. This page follows Paper 09 as the floor. Plain-form K data on Lane B is a recorded exception approved under the firm’s policy (with the privacy officer and, for client secrecy, counsel), not the default outcome of the seven questions. Where the matrix sets no mandatory transform, the seven questions decide.
Know the transforms apart
| Transform | Reversible? | Still personal data? | Use |
|---|---|---|---|
| Tokenisation | Yes, by a party holding the de-tokenisation map or key | Yes (it is a form of pseudonymisation) | Model does not need raw values; response restored at the gateway |
| Masking | No, in production | Depends on what remains | Value never needed; development and test data |
| Labelling | Not a transform; attaches a sensitivity label | Unchanged | Propagate class through generated content |
| Redaction | No; the entity is removed | Depends on what remains | Class or entity not permitted for the task or route |
| Pseudonymisation (GDPR sense) | Yes, with separately held additional information | Yes | Lowers routing class (rule 4), never removes the P flag |
| Anonymisation | No | No; leaves the data-protection perimeter | Only when re-identification is not reasonably possible |
Tokenisation is not anonymisation (Book 1 Ch 13 §4.5; Appendix D).
Run the seven questions
Book 1 recommends answering seven questions in order on each model invocation type, enforced at runtime and audited by the gateway (Ch 13 §5.3):
| # | Question | Effect on the decision |
|---|---|---|
| 1 | Is the model in-tenant or external? | External biases towards tokenisation; in-tenant towards plain form |
| 2 | Is the data special-category or criminal-offence data? | Yes plus external: tokenise, or do not send |
| 3 | Does the task need entity resolution (customer 360, supplier reconciliation, look-alike detection)? | Argues against tokenisation; if both apply, deterministic tokens are needed and leak through their determinism |
| 4 | Does the task need relationship awareness (family graphs, corporate hierarchies, ownership chains)? | Argues strongly against tokenising names; a relational graph beside tokens is possible but expensive |
| 5 | Does a solely automated significant decision apply? | The firm must explain the logic, so it must record what the model saw, plain or tokenised |
| 6 | Is the model endpoint inside the EU? (Page note: for a non-EU firm, read this against its approved jurisdictions.) | No adds the transfer obligation; no plus special-category data: tokenise, or do not send |
| 7 | Is logging compliant with retention rules? | Plain-form personal data in a log takes the retention regime of the original data; no forces tokenisation in the log |
The three branches Book 1 resolves: external + special-category + non-EU → do not send; redirect to an in-tenant model. External + no special-category + summarisation + EU endpoint + compliant logging → tokenise direct identifiers, send the rest plain. In-tenant + relationship task + EU endpoint + compliant logging → send plain, log structurally with sensitivity labels (subject to the matrix floor above).
Why both defaults fail (Ch 13 §4.5). Tokenising everything removes meaning the model was trained on (TOKEN_42 means nothing), destroys entity resolution and hides relationships; projects degrade quietly. Tokenising nothing discloses personal data to an external model, raises residency and transfer questions for non-EU endpoints, and leaks across contexts when plain-form prompts sit in long-retained logs read by analysts outside the original purpose. In Book 1’s composite counterfactual, always-tokenise dropped a copilot’s validation task completion from 84% to 61%.
Operating the decision (Ch 13 §7, recommendation): run the seven questions across the top five invocation types by volume (volume is where exposure compounds) from the model usage registry; record one page per type; have the privacy officer countersign each. A smaller firm runs them on typical invocations. The per-invocation audit record (Ch 13 §6) carries agent identity, wrangler, MCP server called, gateway policy decision, the tokenisation decision with the seven answers keyed by invocation type, sensitivity labels applied and retention class, joined by correlation ID.
Vendor tooling, as of May–June 2026 per Book 1 Ch 13 §4.5–4.6. Azure: API Management as AI gateway, Foundry PII detection (reported GA April 2026), Purview for cross-environment telemetry, and a Purview SDK in the Agent Framework announced for late June 2026 to push labels into the agent runtime. AWS: Bedrock Guardrails and the ApplyGuardrail API for PII redaction. GCP: Cloud DLP with Vertex AI safety filters and Apigee. Book 3 Ch 10 §6 adds Amazon Macie for S3 discovery and notes that inline DLP on AWS typically needs Lambda functions behind API Gateway. [FACT-CHECK: GA status and capabilities of Foundry PII detection, the Purview Agent Framework SDK, Bedrock Guardrails and Cloud DLP.] A Foundry PII detector can run in audit-only mode to show what it would have tokenised without transforming, which is useful when the decision is plain form.
Place and protect the rehydration map
The rehydration map is the table that turns tokens back into values. It is a copy of the original values and, by inheritance rule 1, carries their class; it must never travel with the request to the lane.
| Placement choice | When | Controls |
|---|---|---|
| Per-call, gateway-held (Book 3 Bank A pattern) | Tokens needed only to restore one response | Named-values cache backed by a key vault, keyed by correlation ID; TTL equal to the longest expected workflow; cache flushed on every gateway release; a tokenisation-restore-failed event on every reverse-lookup miss, reviewed daily by the SOC |
| Per-session | Multi-turn work referring to the same entities | First-class store, encrypted at rest, retention-managed, scoped to the session and deleted at session end where the use case allows (Book 1 archived draft) |
| Per-customer durable | Repeat processing that must reuse stable tokens | Book 1’s archived draft applies the same first-class-artefact discipline whether the map is per-session or per-customer. Page recommendation on top: key rotation, a documented re-identification risk assessment, and the determinism limit from the failure table below |
Book 3 Ch 10 PR-3 gives an API Management policy fragment as a starting point, not production code; Bank A’s real policy is reported at roughly 180 lines with regex coverage, cache TTL, key rotation and deploy-time regression tests.
Tokenisation failure modes
| Failure | Recognise it by | Recover by |
|---|---|---|
| Tokenisation as compliance default | Business complaints that the copilot was better in the pilot than in production | Retrofit the seven questions to agents in production; per-task justification is the artefact the supervisor reads |
| No re-identification path | Outputs containing unrestored tokens; map held in process memory or a cache that drifts from the CRM | Make the map a first-class, retention-managed, encrypted artefact (above) |
| Tokenisation that drifts | Wrong values restored (wrong customer, wrong currency); restore-failure events | Weekly regression tests in the gateway release; flush on deploy; pattern review at each classification refresh |
| Leak through determinism | An analyst re-identifies a sample of tokens against an external reference set | Session-scoped salt, rotated; determinism only within an invocation, not across the corpus |
| Detector as the only defence | A spot check finds a name or IBAN in a model log | Four layers: labels and DLP at source; gateway detector; prompt construction that omits unneeded fields; periodic log sampling against the classification regime |
| Gateway taxonomy not aligned with classification | Two taxonomies in two tools that cannot answer each other | Derive gateway entity definitions from the classification; flag AI-relevant categories in the classification; quarterly cross-review until the dictionary joins them |
| One-sided DLP | Inbound policy never calls the classifier | Classify on the inbound leg; a response-side filter cannot recover data already sent |
Sources: Book 1 Ch 13 §8 (first row); Book 1 archived draft §8 (second, fourth, fifth, sixth); Book 3 Ch 10 §4.2 and §9 (third, seventh).
Decide prompt-content streaming to the SIEM with the DPO first
Streaming prompt content written by named employees into a security monitoring system the second line queries is also processing capable of monitoring employee behaviour. Book 3 Ch 10 §4.3 notes that this engages works-council co-determination in Germany and equivalent regimes elsewhere, and warns that vendor guidance may default to including prompt content in telemetry streams. Bank A’s answer (composite), agreed with its works council before go-live: stream metadata and classification verdicts in full, prompt content only for regulated workflow classes, under a documented retention period with second-line access logged. Take the question to the impact assessment before the SIEM team builds the stream. Book 3 also notes that compliance-export coverage is per product surface, not per vendor; track it quarterly. [FACT-CHECK: current Anthropic Compliance API coverage per Claude product surface.]
Failure and recovery beyond tokenisation
| Failure | Recognise it by | Recover by |
|---|---|---|
| Lane assigned by console or vendor | Catalogue entries without inference/storage location or processor statement | Re-assess per deployment from current terms |
| Zero retention assumed where it does not apply | Gateway logs show surfaces outside the eligibility table; fewer enablements than organisations; sign-ins outside the covered organisation | Retention posture as an explicit envelope field; fleet-wide login pinning; quarterly attestation |
| Retention answered at the server only | Processing record omits the workstation | Endpoint baseline; impact-assessment paragraph naming carve-outs |
| Classification refreshed without re-checking envelopes | retrieval-blocked-stale-envelope events; no re-attestation after refresh | Make classification approval a control-plane trigger: every envelope touching the affected class re-attests (Bank A runs it within four weeks) |
| Model upgrade changes retention silently | Model version differs from the version the lane was assessed on | Joint upgrade and lane review |
| Mixed-class context | A C2-routed request whose retrieval returned C3/C4 passages | Classify after retrieval, before routing; separate indexes by class |
Acceptance checks
- The firm’s scale is mapped to C1–C4 and P/S/K/I/R, and the mapping to the AES classification enum and the dictionary enum is written down.
- Every model deployment in the catalogue records lane, processor statement, inference and storage locations, retention configuration with a passing test, model version and last-assessed date.
- The gateway classifies after retrieval at the highest class of all inputs, sets the sensitive flag by classifier, and blocks visibly with a reason.
- Negative tests prove: a C4 passage in context forces Lane C (or the approved Lane B exception); C3 does not reach Lane A without the approved redaction service; K data does not reach Lane A or B unpseudonymised unless a recorded exception exists.
- Derived artefacts (indexes, caches, traces, memory, adapters, evaluation sets, rehydration maps, endpoint transcripts) carry their inherited class and follow the matrix rows.
- No down-classification happens in a prompt; every approved masking service is versioned, tested and logs what it removed.
- Roles are recorded per arrangement by the DPO; no lane exists without approved terms; sub-processor changes trigger review.
- Each high-volume invocation type has a countersigned seven-question paper, and the per-call audit record carries the tokenisation decision.
- Rehydration maps are encrypted, retention-managed, scoped and monitored for restore failures.
- Model upgrades trigger lane and retention re-assessment.
Source status. Classes, flags, lanes, matrix, inheritance rules, minimum terms and role allocations are design proposals and working hypotheses from Working Paper 09, which records machine-assisted verification on 10 November 2026 and no legal review. The seven questions and tokenisation anti-patterns are author recommendations from Book 1 (two anti-patterns from an archived draft chapter consolidated into Chapter 13). Bank A material is composite. Vendor details are dated snapshots marked for fact-checking. Nothing here is legal advice.
Source material
- DEIXEI Research Series, Working Paper 09, Data Classes, Processing Lanes and Controller/Processor Roles for AI (§§3–8, 10)
- Book 1 (Building Organizations for the Age of AI), Chapter 13, Gateway, MCP, and the API Boundary (§4.5 tokenisation gate, §5.3 seven questions, §6 worked example, §7, §8 anti-patterns)
- Book 1, archived draft chapter 'The AI Gateway and the Tokenisation Dilemma' (manuscript/_archive_consolidated, consolidated into Chapter 13), §8 anti-patterns
- Book 1, Appendix D, Glossary (tokenisation, masking, labelling)
- Book 1, Chapter 16, Data as the Enterprise Multiplier (§8, alignment of dictionary classification with the model usage registry)
- Book 3 (Deploying Claude Safely), Chapter 10, Data Classification, Retention, and DLP (§§4–5, §8, §9, PR-1 to PR-3)
- Book 3, Chapter 12, The Landing Zone Tied Together (PR-1 landing-zone integration checklist, item 17, endpoint estate)
- Supporting Tools, Audit Evidence Schema v0.1.0 (dataEntry classification enum)
Related research
Related guidance
-
Design and Operate the Model GatewayBuild the model gateway as the governed edge for every model call: per-client identity, per-request highest-class routing, tokenisation, pinned models, a conformant wire contract, a correlated evidence record, a tested concentration answer, and day-two cost, latency and reliability operations across two estates and three meters. -
Govern APIs, Data Products and Retrieval Agents Can TrustTurn interfaces and data into owned, versioned capabilities that applications and agents consume under identity, entitlement and evidence controls, and give agents access to meaning, limits and authority, not only bytes. -
Design the Cloud Production Line and Hosting BoundariesClassify, segment, constrain and observe every workspace at the plane where actions execute, choose hosting routes per request, and keep vendor maturity, contracts and exit equivalence dated and explicit. -
Govern MCP Connectors Through a BrokerAdmit, tier, pin and enforce every MCP server through one registry and two enforcement points, keep tokens audience-bound, govern A2A delegation through its own registry and gateway, and expose client-facing agent channels only through the same policy layer. -
Coding-Agent Workstation Baseline (Claude Code)Put Claude Code on developer workstations and build agents as privileged automation: one device-delivered managed policy per surface, lock flags that stop rules merging upward, command hooks for enforcement and HTTP hooks only for evidence, the OS sandbox where it runs, telemetry by policy, and the workstation recorded as a processing location. Dated to November 2026. -
Choose the Shape of an AI ProductA least-agency decision sequence for choosing a product design: agency classes and their trade-offs, the eight questions, evidence for every step up, the consequence matrix that caps action scope, the cut-set review for agents, the other selection dimensions, and the design record. -
Classify Use Cases into Risk TiersThe T0–T4 use-case taxonomy as a working decision reference: the prohibited-use gate, four intake questions, data and action boundaries per tier, the twelve-area control matrix, retention and payload rules, runtime enforcement, approval gates, regulatory routing and re-tiering. -
Route Regulatory Questions to Obligations and EvidenceA dated, article-level reference that routes AI design questions to the EU AI Act, DORA and its RTS, GDPR and sectoral rules, and turns each obligation into a control, an owner and evidence: change records under RTS 2024/1774 Art 17, the 4 h / 72 h / one-month incident clocks, the CIF hinge and its five artefacts, the three-count Register of Information reconciliation, Art 14(4) oversight controls, the Art 26(5) discharge, the FRIA gate, Art 49 and Art 50, SCHUFA-grade review and supplier controls. -
Model Governance: Usage Registry and Upgrade LifecycleGovern models at the grain regulators ask about: a catalogue for what exists, an 18-field usage registry for what is used, risk classified per usage, a registry the policy decision point reads by usage_id, pinned model IDs, and a four-phase upgrade lifecycle from Day −14 to Day +90 with numeric rollback thresholds. -
The Audit Spine and the Evidence RecordHow to capture, join, protect, retain and prove per-action evidence: the AES v0.1.0 record contract, three capture points and one correlation ID, tiered immutable storage, retention and legal hold, replay rehearsal, the route-approval evidence gate, and the Claude-specific ways evidence goes missing. -
Bounded-Agent PatternA class 3 agent lets a model choose the next step, but only inside a registered, validated and enforced envelope of Mandate, Means, Memory and Monitoring. -
Respond to Agent IncidentsSort an agent failure into one of four incident shapes, suspend it in three verified layers, keep humans on every classification and disclosure decision, route one playbook to every authority that needs a cover sheet, and close each incident with a named control change.