Browse the reference ⌄
Operating Guide published

Model Governance: Usage Registry and Upgrade Lifecycle

Govern models at the grain regulators ask about: a catalogue for what exists, an 18-field usage registry for what is used, risk classified per usage, a registry the policy decision point reads by usage_id, pinned model IDs, and a four-phase upgrade lifecycle from Day −14 to Day +90 with numeric rollback thresholds.

Version
0.1.0
Owner
DEIXEI
Last reviewed
Reading time
52 min

For Cto · Ai Architect · Platform Engineer · Risk And Assurance · Model Risk · Internal Audit · Finops

Sources and details
Source works
Book 1 · Book 2 · Book 3 · Book 09 · Research Series · Supporting Tools
Source chapters
7, 8, 9, 13, 14, 16, 17, 18
Source appendices
01:B
Tool specifications
deployer_manifest@0.1.0, audit_evidence_schema@0.1.0
Refresh trigger
vendor release

Full source list ↓ Source index →

Model governance answers three questions that look alike and are not: which models exist in the estate, how each one is used, and what changed since the use was approved. Platforms ship good answers to the first. The second and third are the institution’s to build, and they are the ones a supervisor, an incident review or an auditor actually asks.

This page sets out the records, the controls and the lifecycle that answer them: the model catalogue and the model usage registry as separate records, risk classification at the usage grain, the 18-field usage row and how to populate it, the wiring that lets the policy decision point refuse an unregistered or stale usage, retirement and point-in-time history, third-party models, version pinning, and the upgrade cycle that treats every model release as a change event.

Reading the labels. Source claim means a book states it. Author model means a framework the books propose (the 18 fields, the four phases, the rollback thresholds) rather than an industry or regulatory standard. Recommendation is this page’s own operating advice or synthesis. Bank A, Bank D and Asset Manager X are composite firms written for the books; their figures illustrate a mechanism and are not benchmarks. Vendor specifics are dated and carry a [FACT-CHECK] marker where they are current product claims not yet re-verified against vendor documentation. Legal classification and regulatory dates live on the regulatory map; this page links there rather than restating them.

The model in one table

QuestionAnswer this page givesSection
Is the platform’s model catalogue our model inventory?No. The catalogue records existence; a separate registry records useTwo records
What is one row in the registry?One usage: (deployment endpoint, calling agent or system, business process)The grain
Where does the regulatory class sit?On the usage row, assigned and signed per usage, never inherited from the modelClassification
What does a row hold?Eighteen fields in four blocks; three are load-bearingThe 18-field row
How does the registry control anything?The policy decision point decides “can agent X invoke usage U at time T?” and every call logs usage_idWire to the PDP
When does a usage end?At a planned date, or “evergreen” with a review date no more than 12 months out, plus event triggersRetirement
How do model upgrades reach production?As change events, in four phases from Day −14 to Day +90, with staged rollout and named rollback thresholdsUpgrade lifecycle
How do we stop silent upgrades?Pin full model IDs everywhere, switch automatic upgrades off, and enforce the pin at the gatewayPinning

Keep two records: catalogue and usage registry

The model catalogue answers what models are deployable: versions, regions, capacity, evaluation results, content-safety configuration and sign-off state. The model usage registry answers what those models are doing: which agent or system calls them, in which business process, on which data, under whose name and with which regulatory class (source claim, Book 1 Ch 14 §4.1).

The two have different lifecycles and different owners, which is why one cannot stand in for the other. A catalogue entry is created when a model is approved for deployment and retired when it is decommissioned. A usage is created when a specific caller, in a specific process, starts calling a model for a specific purpose, and is retired when that use ends, which may be years before or after the model itself. One catalogue entry routinely carries many usages.

The model catalog shows what exists; the model usage registry shows what the firm used, and the auditor reads the registry.

Source: Building Organizations for the Age of AI (Book 1).

Model catalogueModel usage registry
Question it answersWhat exists and may be selected?What is used, where, by whom, on what data, at what risk?
UnitModel and versionUsage (see the grain)
Created whenA model is approved for deploymentA caller starts using a model in a process
Retired whenThe model is decommissionedThe use ends, the owner will not sign, or a retirement trigger fires
OwnerAI platform teamNamed wrangler per row; model risk for the class
Typical productPlatform-provided (see below)Built by the institution on the control-plane store
Read byPlatform engineers, model riskPolicy decision point, wrangler, supervisor, internal audit, incident reviewers

Vendor existence layers, as of May 2026 (per Book 1 Ch 14 §4.1, §4.4, §6.1). On Azure, the Azure AI Foundry model catalogue covers existence, and Foundry observability gives per-deployment throughput and cost but not business process or regulatory class. On AWS, the Amazon Bedrock model registry (with SageMaker Model Registry for non-Bedrock models) covers existence and Bedrock model invocation logging gives per-call telemetry. On GCP, Vertex AI Model Registry covers existence, with Vertex AI Logging and Cloud Audit Logs for per-call records and the strongest native lineage of the three. In every case the business-process and regulatory-class dimensions remain the institution’s problem. [FACT-CHECK: confirm current product names and capabilities of Foundry model catalogue, Bedrock model registry and Vertex AI Model Registry; Vertex AI branding may have changed — Book 2 Ch 13 refers to “Gemini Enterprise Agent Platform” as of November 2026.]

A firm on two clouds runs two existence layers and one logical usage registry, partitioned by cloud and queried as one (source claim, Book 1 App B, cross-cloud note).

The model gateway can feed actual call volumes into the registry, but a call record cannot supply business purpose, lifecycle, contract or risk. Those come from people.

Composite illustration (Book 1 Ch 14 §1, Bank A). The platform catalogue counted 62 models; the first usage registry counted 228 rows. 203 rows came from joining the catalogue, the agent registry and the process inventory; 25 more surfaced in a two-week reconciliation pass; three were retired while the rows were being written, two because no wrangler would sign for them. Of the 228, eleven touched credit decisions, all carrying a high-risk class and a named wrangler. The question “which models touch credit decisions?” could not be answered from the catalogue at all.

Define the grain before anything else

A usage is uniquely identified by the tuple (model deployment endpoint, calling agent or system, business process) (source claim, Book 1 Ch 14 §4.2; App B). The grain matters because the regulatory classification attaches at this grain.

SituationRows
One deployment called by two agents in two processesTwo usages
One agent calling the same deployment in two processesTwo usages
One agent in one process calling a primary and a fallback endpointTwo usages
One model reached on two clouds by the same agent and processTwo usages (two endpoints), one logical registry

Agree the grain in writing first. The first move is a written definition, circulated to model risk, the agent platform team, data governance and compliance, with agreement inside a week. If those four functions cannot agree the grain, nothing downstream works (source claim, Book 1 Ch 14 §7).

Source conflict. Book 1 Appendix B’s description of control-plane collection 12 (model_usage) says one record per (model, agent, time-window) triple, while the same appendix’s registry section and Chapter 14 define the usage tuple above. Book 3 Ch 16 OD-3 describes its row as “one row per envelope per model”. This page follows the (deployment, caller, business process) tuple as the identity of a usage, and treats time as history on the row (each pinned version window and each change is a dated version of the row; see point-in-time history), not as part of the key.

Classify risk on the usage, not the model

Risk classification follows usage, not deployment (source claim, Book 1 Ch 14 §3, §4.3; App B field 8). A foundation model is not high-risk because it exists in a catalogue; a usage is high-risk because of the process it powers, the data it touches and the decisions it influences. The AI Act high-risk categories are framed around intended use; see the regulatory map’s routing table for which categories commonly apply in financial services and who decides.

The operational consequences:

  • regulatory_class cannot be inherited from the catalogue entry. It is assigned at registration, per usage, by model risk, reviewed by legal and compliance, and signed by the wrangler.
  • The same endpoint can be high-risk in one row and minimal-risk in another. Both rows are correct, and the inventory is correct only when both exist.
  • The defence of a classification is the combination of three fields on the row: business_process, data_classes_touched and calling_agent_id. When a supervisor asks why two rows calling the same model are classified differently, those fields are the answer.

Composite illustration (Book 1 Ch 14 §6, Bank A and Asset Manager X). One general-purpose foundation-model deployment carried seven usages across two firms: two classified high-risk (mortgage decision-rationale generation; adverse-action letter drafting, classified high-risk because it is adjacent to the credit decision and its output reaches the data subject), two limited-risk (portfolio-commentary drafting; marketing copy), two minimal-risk (meeting summaries; incident post-mortem drafting), and one KYC-summary usage outside the high-risk list but under special-category data-protection controls. An internal fraud classifier carried four usages, one falling under the fraud-detection carve-out and one scoring legal entities rather than natural persons. Each row defends itself on its own fields, and each has a different wrangler. The classifications are the composite’s illustrations, not legal advice; route real classifications through the sectoral diagnostic.

Regulatory class, risk tier and model choice are three different decisions

DecisionWhat it isSet againstOwnerWhere
regulatory_classLegal classification of the usage (prohibited / high-risk / limited / minimal, plus sectoral tags such as CIF / non-CIF, CRD model-risk tier, AIFMD/UCITS)The usageModel risk assigns; legal and compliance review; wrangler signsThis registry (Book 1 App B field 8)
Risk tierThe firm’s control classification, informed by the legal classThe use caseWrangler, DPO, model risk and business owner sign; Agent Control Board ratifies (Book 3 Ch 14 OD-1)Risk tiers
Model selectionWhich model serves the envelope, on a cost, latency and accuracy axisInside the tierWrangler, with model riskSelection and the model gateway

Book 3 Ch 14 §8 names the failure where these collapse: tier mapped to model, for example “this model is T2, that one is T3”. The same use case then gets a different control package depending on which model the wrangler picked, and the question “which control set applies to this use case?” has no answer. Set the tier against the use case; choose the model inside it. A model change can still re-open the tier: Book 3 Ch 14 OD-1 lists a model upgrade that crosses the Chapter 16 threshold, and model evaluation drift beyond threshold, among the re-tier review triggers. The model change triggers a review; it never assigns the tier. See re-tier when the use case changes.

Re-classification triggers (recommendation, combining Book 1 App B lifecycle notes and Book 3 Ch 14 §8). Re-open regulatory_class on: a change to business_process; a new data class in data_classes_touched; a change to the calling agent’s autonomy or action class; output newly reaching a customer or data subject; a new regulatory designation (for example a new CIF designation); and the annual review. Classification history is kept on the row.

The 18-field usage row

Book 1 Appendix B specifies the row (author model). The fields fall into four blocks: what the model and the use are (1–3, 6, 7), who carries accountability (4, 5, 14, 15), how well and how much it runs (9–13), and under what authority and for how long (8, 16–18). Field names and types below are exactly as the source gives them, and the cadence column follows App B’s lifecycle notes. The “Populated by” and “What breaks” columns are this page’s synthesis from App B’s purpose notes and population sequence, not source columns. Book 1 App B also maps each field to regulatory anchors; those mappings are not repeated here (see the regulatory map).

#FieldTypePopulated byCadenceWhat breaks if it is wrong
1model_idstring, catalogue refSeeded from the catalogueSet at registration; immutableThe row points at nothing the platform can confirm
2model_versionstring (semver or vendor-specific)Catalogue; updated on version change with re-evaluationEvent-drivenEvaluations and content-safety settings are version-bound; an unrecorded version change invalidates them
3deployment_endpointURL or ARNCatalogue / gateway configurationEvent-driven, on endpoint migrationLogs cannot be tied to what actually served the call
4calling_agent_idstring, agent registry ref; null for non-agent callersRuntime telemetrySet at registration; immutableThe join to the agent registry, and so to the wrangler chain, is lost
5calling_system_idstring, CMDB ref; always populatedRuntime telemetrySet at registrationThe ICT asset inventory misses the dependency
6business_processstring, process-taxonomy refWrangler, by handAnnual; immutable in normal lifeThe classification cannot be defended
7data_classes_touchedarray of data-taxonomy refsWrangler with data governanceQuarterlyData-protection and data-governance controls are mis-scoped
8regulatory_classenum (prohibited / high-risk / limited / minimal) plus sectoral tagsModel risk assigns; legal/compliance review; wrangler signsAnnual and on process changeOver- or under-classification across the estate
9last_evaluationtimestamp plus evaluation idEvaluation pipelineEach evaluation cycleStale-evaluation deny cannot work (PDP)
10drift_signalsobject: input drift, output drift, evaluation regression, yellow/red thresholds, last-checked timeMonitoringLiveBehavioural degradation is invisible to the owner
11expected_qpsnumberCapacity reviewQuarterly / on capacity reviewCapacity and resilience planning rest on a wrong number
12observed_qpsnumber, rolling 30 daysGateway / telemetryLiveDormant or runaway usages go unnoticed
13fallback_modelcatalogue ref or nullWrangler with platformAnnualFailover sends traffic to an unevaluated, unregistered usage
14owning_wranglerstring, HR identityWrangler named at registrationOn role changeOversight accountability lands on a mailbox, which is no one
15contractual_partystring, legal entityProcurement / third-party riskOn contract changeThe third-party register line and value-chain records are wrong
16retentionobject: input retention, output retention, location, policy idEvidence ownerAnnualLogging and retention obligations cannot be shown per usage
17change_ticketstring, ITSM refChange processSet at registrationNo authorisation trail for the usage
18decommission_datedate, or evergreen + next review date (≤ 12 months out)WranglerReviewed at the dateRows accumulate with nobody able to say which are live

Book 1 App B’s governance cadence for the populated row is: quarterly review of fields 7, 8, 9, 11, 12 and 16; annual review of fields 6, 13, 14, 15, 17 and 18; live update of field 10; event-driven update of fields 2, 3 and 4. (Field 8 also re-opens on process change, per its own lifecycle note. App B is internally inconsistent on field 16: its lifecycle note says annual, its governance step says quarterly. Review it quarterly, the stricter of the two.)

The three load-bearing fields

  • business_process (6). This is what makes a model call a regulated usage. Nothing can derive the process from calling_agent_id automatically; the wrangler supplies and owns the link. The value must be specific enough to drive classification: mortgage origination decision rationale generation, not credit. An agent whose wrangler cannot name a process should not be in production; a blank here is a finding, not a gap (source claim, Book 1 App B).
  • regulatory_class (8). Assigned, defended and reviewed per usage, never inherited (see classification).
  • owning_wrangler (14). A named human by HR identifier. Not a team alias, not the platform team. Reviews go to that person; the platform team supports, the wrangler is accountable. See accountability.

Lifecycle fields for the upgrade cycle (a second layer)

Book 3 Ch 16 OD-3 gives a different 18-field row, built around one envelope’s pinned model and its upgrade history: model, version, provider, route, use-case-class, risk-tier, owner, wrangler, pinned-from-date, pinned-to-date, evaluation-suite-version, evaluation-result, deployment-route, ZDR-status, HIPAA-status, cost-baseline, observed-cost-delta, decision-log-ref. The two books disagree on the field set; neither is wrong for its purpose. Book 1’s row describes the usage; Book 3’s row describes the pin window of a usage through upgrades.

This page follows Book 1’s 18 fields as the usage row and adds Book 3’s lifecycle fields as a pin-window extension recorded on each dated version of the row (recommendation). Where they overlap, use Book 1’s names.

Book 3 OD-3 fieldBook 1 equivalentTreatment on this page
model, version, route, deployment-routemodel_id, model_version, deployment_endpointUse Book 1 fields; record the full route path (gateway → platform region → model ID) in the extension
providercontractual_partyUse Book 1 field
owner, wranglerowning_wranglerUse Book 1 field; record the platform owner in the extension
risk-tier, use-case-classnone (Book 1 has regulatory_class)Add to extension; tier comes from the use-case record
evaluation-suite-version, evaluation-resultlast_evaluationAdd to extension: suite version run at pre-evaluation, not today’s
pinned-from-date, pinned-to-datenoneAdd to extension
ZDR-status, HIPAA-statusnoneAdd to extension; checked at Phase 1
cost-baseline, observed-cost-deltanoneAdd to extension; feeds the cost rollback threshold
decision-log-refchange_ticketKeep both: the original authorisation and the current pin decision

Three checks on every row (source claim, Book 3 Ch 16 OD-3). The model field carries a full model ID, never a family alias, and the pin date and route reconcile to the gateway’s actual routing on the day the row was last updated. The evaluation-suite version is the one run at pre-evaluation. The decision reference points at a countersigned Virtual CAB minute. A row failing any check is treated as drift and routed to the AI platform owner for same-day remediation.

Schema status. Book 1 App B says a normative JSON Schema for the 18 fields is published at deixei.com. As of 14 November 2026 it is not among the hosted schemas (Audit Evidence Schema, CAB change, CAB decision and Deployer Manifest at /aes/v0.1.0/; see schemas and tools). Treat the field list above as the reference until a schema is published; do not invent enum values beyond those the source gives.

Populate the registry in seven steps

Book 1 App B gives the seven steps and who acts at steps 4 and 5 (author model); Ch 14 §7 adds compliance to the grain agreement. The other owners and the Output and Check columns are this page’s recommendations.

StepActionOwnerOutputCheck before moving on
1Define the grain (usage, not model) and agree itModel risk, agent platform, data governance, complianceSigned one-page grain definitionAll four functions have agreed in writing
2Seed model_id, model_version, deployment_endpoint from the cataloguePlatform engineerCandidate rows per endpointEvery production endpoint appears at least once
3Expand by caller from runtime telemetry for the last 90 days: calling_agent_id, calling_system_idPlatform engineerOne row per (endpoint, caller)Callers missing from the agent registry or CMDB are raised as findings
4Attach business_processWrangler (cannot be automated)Rows split by process; the grain is now completeNo blank process on a production row
5Classify: model risk assigns regulatory_class; legal and compliance review; wrangler signsModel risk, legal, compliance, wranglerSigned classification per rowEach high-risk or CIF row has a written rationale
6Attach quality, capacity, lifecycle and contract fields by automationPlatform, procurement, evidence ownerFields 9–13, 15–18 populateddecommission_date never blank; evergreen rows carry a review date ≤ 12 months out
7Govern on the cadence aboveWrangler, model riskReview records on the rowReviews overdue are escalated automatically

Scope ruthlessly (source claim, Book 1 Ch 14 §7). Populate the top ten highest-stakes usages first: high-risk class, CIF-supporting, or of known supervisory interest. In the Bank A composite these were nine credit-related usages and one third-party scoring usage, and took eight working days of two analysts plus the wranglers’ time. The rest follow by quarter-end.

Where to put it (as of May 2026, per Book 1 Ch 14 §4.4, §7).

CloudStorePopulation sourcesReporting
Azuremodel_usage collection in Cosmos DB (collection 12 of the control-plane schema), partitioned by tenant_id, indexed on calling_agent_id, model_id, business_process, regulatory_classFoundry catalogue; agent runtime (Foundry Agent Service, Microsoft Agent Framework); Application InsightsPower BI dataset
AWSDynamoDB or Aurora tableBedrock model registry; Bedrock model invocation logging (CloudWatch and S3); AgentCore Registry for the calling agentQuickSight
GCPBigQuery or FirestoreVertex AI Model Registry; Vertex AI Logging; Cloud Audit Logs; Apigee; Vertex lineage can partly auto-populate fields 2, 7 and 9Looker

[FACT-CHECK: Book 1 Ch 14 states AgentCore Registry and AgentCore Policy reached general availability in March 2026; confirm against current AWS documentation.]

Performance targets (source claim, Book 1 Ch 14 §7). A wrangler can query “show me my usages” in under 30 seconds; a supervisor question is answerable within one working day; the registry is refreshed against the catalogue and agent runtime nightly. Registration is one API Management (APIM) call from the agent build pipeline, with the wrangler named and the change ticket attached at that call, so the compliant path is the default path.

Sizing (author estimates, Book 1 Ch 14 §7).

Large global firm (~50,000 staff)Specialist firm (~1,000 staff)
Steady-state usages200–80030–100
Wranglers owning rows50–15010–25
First iteration8–12 weeks4–6 weeks
Build teamFour: platform engineer, model-risk analyst, data-governance analyst, programme manager; plus about 0.2 FTE per wranglerTwo
Recurring costAbout two FTE a year0.5–1 FTE a year

The schema is the same 18 fields at both sizes. The smaller firm’s temptation is a spreadsheet; the source’s answer is that the schema scales down and spreadsheets do not scale up. This page adds a practical reason: the policy decision point cannot read a spreadsheet, so a spreadsheet registry loses its control function.

Wire the registry to the policy decision point

A registry that nothing reads at runtime is an inventory. It becomes a control when the policy decision point (PDP) decides model calls by usage, not by model (source claim, Book 1 Ch 14 §4.4, §7). The decision is can agent X invoke usage U at time T?, and U carries the regulatory class, data classes, retention and wrangler. See the enterprise control plane for the PDP contract.

Condition at decision timeDecisionRationale the denial carries
No registered usage for (endpoint, caller, process)DenyUsage not registered; registration path
decommission_date passed, or the usage is retiredDenyUsage decommissioned on date; owner
Evaluation stale (see below)DenyEvaluation older than permitted; last evaluation id
Request carries a data class outside data_classes_touchedDenyData class not approved for this usage
Request resolves to a model or version other than the row’s pin (recommendation, extending Book 3 Ch 16 §3’s refusal of unpinned aliases to the PDP)DenyUnpinned or drifted model
All conditions metAllow, with the PDP’s normal constraints and expiry—

Stale evaluation. The books name the stale-evaluation deny but give no single age limit (Book 1 Ch 14 §4.4). Recommendation: treat an evaluation as stale when last_evaluation predates the current pin window’s reassessment date (Day +90 under the upgrade lifecycle), or predates the last change to model_version, or when the Day +60 attestation is overdue. Set the rule per risk tier in policy, and record the rule version on the decision.

One join key on every call. Each model invocation logs usage_id with the agent ID, timestamp, input classification, output classification and policy decision (source claim, Book 1 Ch 14 §4.4). The same log line then serves AI Act traceability and DORA incident investigation. Recommendation: carry the usage_id in the Audit Evidence Schema’s model[].registry_record field, which v0.1.0 already offers as an optional string, and make it mandatory by local profile for T2 and above. See audit evidence.

The registry is the source of truth; reconcile three places. Drift arises quietly between releases: a gateway hotfix answering a deprecation notice without an envelope amendment, or a wrangler editing a model field without going through the Virtual CAB (source claim, Book 3 Ch 16 §7). Run a reconciliation job comparing the registry pin, the gateway’s actual routing and the envelope’s Means field. The gateway reads its routes from the registry on a refresh cycle; an envelope amendment is the only path to change a Means field.

Reconciliation findingAction
Registry pin ≠ gateway routing ≠ envelope MeansRoute to AI platform owner for same-day remediation; treat as drift
A family alias in any production model fieldTreat as auto-upgrade exposure; remediation is a controlled change with CAB approval
Gateway traffic with no matching usage rowUnregistered usage; deny at PDP and open a registration finding
Usage row with zero observed_qps over its review periodCandidate for retirement (recommendation)
Phase deliverable (for example Day +60 attestation) overduePage the wrangler; flag the envelope
Entitlement list in a vendor admin console differs from the registryReconcile line for line (Book 3 Ch 18 §5)

Fallback routes need their own usage rows. A failover that sends traffic to an unregistered endpoint is an unapproved usage, however briefly.

Retire usages and keep point-in-time history

Retirement triggers

decommission_date forces the retirement question at registration: either a date, or evergreen with a next review date no more than 12 months out. A quarterly job escalates rows past their review date (source claim, Book 1 Ch 14 §8, anti-pattern 3; App B field 18).

TriggerSourceWhat happens
Review date passes without re-affirmationBook 1 Ch 14 §8Escalated by the quarterly job; recommendation: PDP denies after a grace period set in policy
No wrangler will sign the rowBook 1 Ch 14 §1 (composite)Retire the usage; an unowned usage cannot stay in production
The business process is retiredBook 1 Ch 14 §1 (composite)Retire every usage on that process
The model is decommissionedBook 1 Ch 14 §6All usages joined to that model retire on the same date; the registry is the only artefact that exposes this join
Third-party contract ends or is not renewedBook 1 Ch 14 §6Align decommission_date to the contract renewal date
Provider deprecates or retires the pinned versionBook 2 Ch 13; Book 3 Ch 16Opens the upgrade cycle; see deprecation tracking
Pin window reaches Day +90Book 3 Ch 16 §4.1Reassessment is due, whether or not the provider has said anything
No traffic over the review periodRecommendationWrangler confirms retirement or justifies dormancy

Re-affirmation cadence

Book 1 App B sets the field review cadences above. The Bank A composite in Book 1 Ch 14 §1 adds a re-affirmation practice: weekly wrangler attestation on the high-risk rows, monthly across the estate, and a printed diff between two dated snapshots laid side by side to show the register is right today, not on the day it was printed. App B has no “date last re-affirmed” field. Recommendation: record each re-affirmation as a dated attestation event in the row’s history rather than overwriting a single date, so that the history itself proves the cadence.

Keep point-in-time history

The question that defeats most registries is not “what is the usage?” but “what was the usage on a Tuesday three months ago?”. A firm that can produce only today’s row has no answer (source claim, Book 1 Ch 14 §7, internal-audit deliverables). Book 1 cites EBA supervisory material warning against exactly this. [FACT-CHECK: Book 1 Ch 14 cites an “EBA factsheet of 21 November 2025”; confirm the document and its wording before relying on it.]

Recommendation: make the registry append-only with a validity interval on every version of a row (valid-from, valid-to), and record who changed what, under which change ticket. Every change to a field with a review cadence, every re-classification, every pin window and every re-affirmation is a new dated version.

Internal audit’s three deliverables (source claim, Book 1 Ch 14 §7):

  1. An audit-evidence query template that returns, for any supervisory question, the usage tuple with wrangler, regulatory class, change ticket and data classification.
  2. A named-control-owner register tying each registry field to a first-line populator and a second-line reviewer, with a half-yearly reconciliation against the catalogue and the invocation logs recorded as a control test.
  3. A post-incident reconstruction protocol that recovers the usage record as it stood at the time of the incident and traces it to its change ticket.

Internal audit signs a half-yearly testability attestation: the registry reconciles to the agent runtime and the catalogue, and a sample of usages can be reconstructed end to end.

Third-party model usages

A model consumed from an external provider over an API may never appear in the platform catalogue. It still needs a usage row, and it is often the row that matters most (source claim, Book 1 Ch 14 §6, Group 3).

ConcernWhat the row must carryWhy
Who provides itcontractual_party: the named third party for third-party models; the foundation-model vendor’s EU legal entity for hyperscaler-routed models; the firm itself for internal models (Book 1 App B field 15)Drives the Register of Information line and value-chain records
Whose obligationsWhere the firm deploys a third-party high-risk system, deployer obligations land on the firm, not the provider: use per instructions, human oversight, monitoring and loggingThe obligation is invisible to the platform team without the row; see the regulatory map’s oversight and logging rows
ConcentrationCIF flag; whether the provider is, or depends on, a designated critical ICT third-party provider (monitored by procurement)Concentration and exit analysis per usage
When it endsdecommission_date aligned to contract renewalPrevents auto-renewed contracts scored against an old evaluation (Book 1 anti-pattern 3)
Evidence of fitnesslast_evaluation on the firm’s own scenario library, not the provider’sThe provider’s evaluation does not cover the firm’s use
Fallbackfallback_model, which may be a non-AI rule setResilience and exit

Composite illustration (Book 1 Ch 14 §6, Bank A usage 12). An SME lending decision score from an external provider, under a third-party contract recorded in the Register of Information: high-risk class, CIF, its own scenario library (94.7% pass with one regression flagged), a rule-based legacy score as fallback, and decommission aligned to the contract renewal date. It sits in the same query and dashboard as the foundation-model rows, under the same wrangler discipline.

Supplier-hosted models reached by a floating name (Book 09 Ch 9, composite narrative). An agent reached its model through a supplier’s hosted service under a name meaning “the current model”. The firm’s rule pinned every deployment in its own subscriptions; the supplier re-pointed its name at a successor, and the agent’s judgement shifted for weeks with no release to approve. The only trace was the version string the evidence record kept on each response. The rule that followed, which this page adopts as a recommendation:

  • the pinning rule reaches every model a production agent calls, wherever it is served, including inside a supplier’s SaaS;
  • a supplier’s floating name is refused unless it can be fixed to a version;
  • no cases, no move: no version moves under a production agent until its owner has run it against cases with known answers, and an agent with no cases does not move until it has some;
  • record the model version as each response reports it in the evidence record, so a supplier-side change is at least detectable.

Treat model selection as a separate decision

Model selection happens inside a tier, never instead of one (source claim, Book 3 Ch 14 §8). A use case cannot drop a tier because a cheaper model was found. A lateral swap between model classes runs the same four phases as a version step, and is harder, because deterministic test assumptions travel less well across families than across versions (source claim, Book 3 Ch 16 §4.6).

The advisor pattern (Book 3 Ch 14 §8, Ch 16 §4.6). A mid-tier model does the work and a frontier model is attached as an advisor, invoked selectively. Book 3 reports Anthropic’s figure of a mid-tier model with a frontier advisor landing within 10% of the frontier model alone at 63% of the cost, as a vendor-internal measurement with no published methodology. The architecture transfers; the percentages do not. [FACT-CHECK: Anthropic, “Claude models explained”, 24 July 2026, figures and model names as cited in Book 3.]

Check availability and data posture at Day −14, not Day +30 (Book 3 Ch 16 §4.6, as of August 2026). Put two procurement checks in the Phase 1 ticket:

  • Contractability. A model the firm cannot contract for is not a registry row, however well it evaluates. Book 3 reports that the frontier “Mythos” class was available only to selected organisations through “Project Glasswing”. [FACT-CHECK: current Anthropic model lineup and availability.]
  • Zero-data-retention compatibility. Book 3 reports that the “Fable” class requires retention and is refused for zero-data-retention organisations regardless of client configuration, and that the best alias then resolves to Opus. For a firm that bought ZDR as a data-boundary control, the advisor pattern in its published form is unavailable. [FACT-CHECK: Fable/ZDR restriction and best alias behaviour against current Anthropic documentation.]

Record ZDR and HIPAA eligibility for each new model ID on the registry extension at Day 0 (Book 3 Ch 16 §4.1). Routing by complexity, model entitlements and cost fallback are gateway mechanics; see the model gateway. One rule from Book 3 Ch 18 §5 belongs here: a cost-fallback substitution happens only where the mandate carries a cost-fallback-permitted flag, and the gateway logs each substitution as a policy decision. Recommendation, following Book 1’s grain: the substitute route needs its own usage row.

Run every model upgrade as a four-phase change

Why a cycle, not a project. Book 3 Ch 16 §2 and §8 record eight Claude versions in nine months and a structural change to the lineup within a year, and advises planning for a release every 8–12 weeks and 12–15 upgrade cycles over 18 months (author estimate). Two further observations from that chapter set the stakes. A version step inside one family can widen what an agent is willing to do with no change to its mandate or means: Anthropic has published a case where, after an upgrade from Opus 4 to Opus 4.5, an internal incident-response agent messaged another Claude instance to request a code fix, which was stopped only by a human pull-request review gate (Book 3 Ch 16 §2, Ch 17 §4.5). And Anthropic’s deputy CISO is cited as saying controls go obsolete roughly six months behind model evolution. [FACT-CHECK: Anthropic, “Zero risk isn’t the job”, claude.com/blog, 17 July 2026, as cited in Book 3.]

The lesson for model governance (source claim, Book 3 Ch 17 §4.5): containment that holds across a version step is structural (review gates, scoped access, agent traffic on channels humans watch), not textual. Re-test the containment posture at every new pin, not only answer quality; see evaluation and resilience. Log a near-miss of this shape as an envelope finding for the Agent Control Board.

Size the change by update scope

Not every model change needs the same treatment. Book 2 Ch 13 and Research Paper 1 §5.3 condense Microsoft’s lifecycle guidance into six update scopes (source claim; Microsoft guidance as cited by the books, accessed September 2026).

Update scopeExampleTypically requires
Minor versionNew dated release of the same modelRegression evaluation; prompt check
VariantFaster or cheaper variant of the same familyRegression evaluation; latency and cost check
Major or generational versionNew model generationFull evaluation; prompt rework; possibly orchestration changes
Model changeDifferent vendor or familyFull evaluation; prompt and tool-calling rework; vendor onboarding
Specialised modelDomain-specific modelRe-assessed grounding needs; full evaluation
Hosting route changeVendor API to private tenancy or controlled infrastructureInfrastructure and security review; new residency assessment

A prompt and the model version it was tested on are one change: they move together or not at all. The release unit is the pinned combination of model and version, parameters, prompt, orchestration, retrieval configuration and tool versions, recorded in a release manifest with its evaluation results (source claim, Book 2 Ch 13; Research Paper 1 §5.3). See BOM and recipes and release engineering.

The four phases

Dates are anchored to the public release (Day 0). Owners do not change between cycles (author model, Book 3 Ch 16 §4, OD-1).

PhaseWindowLeadSecondSign-offOutput
1 — Watch and pre-evaluationDay −14 to Day 0AI platform ownerHead of model riskHead of model riskRegistry update; evaluation-suite pre-run; written recommendation
2 — Decision and stagingDay 0 to Day +30WranglerAI platform ownerVirtual CABApproved change dossier; staged rollout schedule; envelope amendment
3 — Production observationDay +30 to Day +90WranglerAI platform ownerHead of model risk (Day +60 attestation)Day +60 suite re-run; wrangler attestation; telemetry record
4 — Retention or rollbackDay +90Agent Control BoardHead of model riskAgent Control BoardLogged decision (retain / roll back / carry); updated pin date; next reassessment date

Phase 1 — watch and pre-evaluation.

  • The AI platform owner watches the vendor sources weekly. Book 3 names four for a Claude estate as of August 2026: Anthropic’s release-notes overview, Microsoft Foundry’s models-from-partners page, AWS Bedrock’s model–region compatibility page, and the Claude Code settings reference that documents per-provider alias defaults. The last is the one teams forget (see alias drift). When a preview appears, open a Phase 1 ticket and notify model risk.
  • On Day 0, the same working day: update the registry with the new model ID, version, provider, route, ZDR eligibility and HIPAA eligibility, leaving the pin date blank; the reconciliation job flags every envelope referencing the prior version; model risk checks out the evaluation suite at its current version and queues it against a sandbox deployment.
  • Pre-evaluation runs 10–14 working days. Model risk owns the analysis; platform owns execution. The output is one recommendation: pin, pilot or migrate, on which workflows, over what cadence, with what reassessment trigger. It becomes the dossier for Phase 2.

Phase 2 — decision and staging.

  • The Virtual CAB grades the change by the risk tier of the envelopes it affects. Book 3 numbers its change classes in the same “Tier” vocabulary as its risk tiers; this page keeps them apart. An upgrade affecting only T0 or T1 envelopes (internal-only, no customer impact) is a controlled change executed by the wrangler under the dossier, with the AI platform owner co-signing. An upgrade affecting any T2-or-higher envelope (customer-impacting, regulated or CIF-adjacent) takes full Virtual CAB review, second-line attestation, and a supervisor-notification assessment by the DPO and model risk. See change governance.
  • Staged rollout starts the day after approval, per envelope:
WeekTraffic on new modelCheck
15%Gateway records the split; wrangler reviews the day-end evidence sample on Day +7
225%Wrangler reviews on Day +14, with a cross-section from the prior week for comparison
350%Status to the Virtual CAB’s weekly slot; any envelope outside named thresholds pauses here, and rollback is an option, not an exception
4100%Registry pin date updates to Day +28; reassessment date set to Day +90
  • Two envelopes sharing a model never share a rollout schedule unless the Virtual CAB approves the synchronisation in writing in the dossier.
  • Enforce the cadence at the gateway (source claim, Book 3 Ch 16 §7): traffic-split policies are versioned and signed, the gateway refuses to advance a split without a new policy artefact, and the AI platform owner co-signs each weekly step. A rollout collapsed to “5% on Monday, 100% on Tuesday” never runs long enough to catch tail-of-distribution regressions.

Phase 3 — production observation.

  • Three telemetry streams run continuously against a 30-day baseline taken on the prior model in the fortnight before pre-evaluation: cost per task; hallucination rate against the deterministic subset, replayed weekly on fresh traffic samples; and tool-call rate and refusal rate, the behaviours most likely to drift between minor versions. Each stream’s threshold is named in the rollback plan in advance.
  • On Day +60 the suite runs again: same suite version, same deterministic cases, same distributions, same scenarios. The wrangler files an attestation in the change record (re-run done, results consistent with pre-evaluation, envelope within thresholds, retention recommended through Day +90) and joins it to the registry row. If it is not filed, the reconciliation job flags the envelope and the platform owner pages the wrangler.

Phase 4 — retention or rollback. The quarterly Agent Control Board reviews the cohort and records one of three outcomes:

OutcomeEffect
RetainModel stays at 100% until the next release re-opens the cycle
Roll backGateway header reverts; registry pin returns to prior version; wrangler attests; evidence-portability check runs (below)
CarryRetained on this envelope, with a wrangler-filed constraint: no extension to a sibling envelope until the next pre-evaluation

Record the outcome on the registry row together with the change-record ID, the suite version used both at pre-evaluation and at Day +60, the period the telemetry was sampled over, and, for every measurement, which named threshold it was tested against. A supervisor reads this entry.

Every pin expires at Day +90 (source claim, Book 3 Ch 16 §4.1). Because provider retirement dates arrive as notices rather than a schedule, the firm sets its own reassessment date instead of inheriting the vendor’s.

Route mechanics (as of August 2026, per Book 3 Ch 16 §4.5)

The cycle is the same on every route; only the model header and the staging mechanism differ. Run one cycle per release, not one per route.

RouteModel headerStagingRollback
Microsoft FoundryFoundry deployment nameAPI Management policy fragment switching the upstream URLInverse fragment
Amazon BedrockBedrock model identifier on the bedrock:InvokeModel callLambda authorizer weighting two model IDsInvert the weight
Google Vertex AIAnthropic model ID with regional endpoint per residency postureTraffic split keyed by service accountReverse the split (Book 3 does not state the Vertex rollback; inferred by symmetry)
Direct Anthropic API behind API ManagementAnthropic model ID in the JSON bodyAPI Management policyPolicy-versioned revert

[FACT-CHECK: per-route identifier formats and staging mechanisms against current Microsoft, AWS, Google and Anthropic documentation.]

Composite illustration (Book 3 Ch 16 §1, §6, Bank A). A cross-family evaluation on three envelopes found a new tokeniser raising token counts 1.26× on the regulatory-summary corpus, compounding with list price into a 2.7× cost per task; two of 50 deterministic cases regressed (4%, below the 5% threshold but ruled material by the head of regulatory affairs, because the misread was the article number the summary existed to report); and code-review correctness improved 4.3 points on a 100-case merge-history corpus. Outcome: pin the regulatory-summary cohort on the prior model, migrate developer assistance on the four-week schedule, and hold the peer-benchmark envelope pending its own evaluation. Thresholds trigger a decision; they do not replace the business owner’s judgement on materiality.

Roll back on named thresholds and prove evidence portability

Thresholds (author model, Book 3 Ch 16 OD-2). Agree them in advance and name them in the change dossier. Rollback fires if any is exceeded during Phase 3, or if the Day +60 attestation cannot be filed.

ThresholdDefinitionMeasured against
Cost varianceCost per task on the new model exceeds the prior-model baseline by more than 30% on a rolling 14-day windowGateway billing record, normalised to the envelope’s representative task
Evaluation regressionDeterministic-layer regression rate above 5% at the Day +60 re-run, or any single material business-outcome scenario failsDay +60 evaluation-suite report
Incident-rate increasePer-envelope incident rate up more than 100% on a rolling 30-day window against the prior model’s 30-day baselineIncident record (incident response)
Attestation missingDay +60 wrangler attestation not filedChange record

Rollback steps (Book 3 Ch 16 OD-2). Each step leaves a named artefact; the rollback is not complete until all six exist. OD-2 names the AI platform owner for step 1 and the wrangler for step 3, and the Virtual CAB receives step 6; the other owners marked rec. are this page’s recommendations.

#StepOwnerTime limitArtefact
1Pause progression; revert the traffic split to 0% on the new modelAI platform owner30 minutes at 5% or 25%; one working hour at 50% or 100%Gateway policy diff
2Revert the gateway model header to the prior pinned version; log the rollback change-record ID on the policy diffAI platform owner (rec.)—Rollback change record
3Wrangler attests: threshold breached, time of breach, steps taken, registry row updatedWrangler—Signed attestation
4Evidence-portability check: replay query over the rollout cohort’s correlation IDsPlatform with evidence owner (rec.)—Replay output appended to the change record
5Resume on the prior model: set the new model’s pinned-to date to the rollback date; re-open the prior version’s rowWrangler (rec.)After step 4 passesUpdated registry history
6Post-rollback review at the next Virtual CAB slot: cause, evidence, control improvement filed against the suite or the staging policyWrangler writes it up (rec.); Virtual CAB reviewsNext standing slotCAB minute

The evidence-portability check is the gate (source claim, Book 3 Ch 16 §5.2, OD-2). Every audit-evidence record’s model-ID field must match the model that actually served the call according to the gateway state at that time, and the records at the rollback boundary must reference the rollback change record. A rollback that reverts the header but leaves boundary records whose model ID does not reconcile breaks the reconstruction of who did what, on whose authority. See audit evidence.

Rollback target must still be served (recommendation, from Book 2 Ch 13). You can only roll back to a version the provider still serves. Before Phase 2 approval, confirm the prior version’s retirement date falls after the end of the pin window; if it does not, the dossier must name a different fallback.

Deployment manifest rules that bind here (Supporting Tools, Deployer Manifest v0.1.0, as cited in Book 3 Ch 16 §3). required_cab_decision is always present (null only with a cited cab_exemption_rationale); verify.health_checks must be non-empty; and rollback.verification is required. Where prose and specification disagree, the specification wins. See schemas and tools.

Pin full model IDs and close alias drift

The rule. Every production model reference names a full model ID, never a family alias, in the envelope’s Means field, the release manifest, the gateway route, workstation settings and the registry row. Automatic upgrades are disabled in production, minor versions included (Book 2 Ch 13, author preference; Research Paper 1 §5.3). An alias resolving to a newer version without an envelope amendment is a policy violation: the route is configured to refuse it, the reconciliation job detects it, and the wrangler is paged (Book 3 Ch 16 §3). Remediating an aliased envelope is itself a controlled change requiring CAB approval (Book 3 Ch 16 §7).

Recognise auto-upgrade by a family alias anywhere in a production model field, by the reconciliation job finding the gateway on a different version from the registry, by an unexplained step in the token bill, or by answer quality shifting on a date matching no change in your history (Book 2 Ch 13; Book 3 Ch 16 §7).

Alias drift across routes (as of August 2026, per Book 3 Ch 16 §4.5 and Ch 18 §5)

Family aliases (opus, sonnet, haiku, fable) resolve to per-provider defaults that move on the provider’s schedule and, per Book 3, did not move together: opus resolved to Opus 5 on Bedrock and on what Book 3 calls the “Agent Platform” route, but to Opus 4.6 on Microsoft Foundry. A firm with opus in both a Foundry fleet and a Bedrock fleet is running two models under one word while its registry records one. Book 3 also reports a billing edge: an unpinned Claude Code deployment is billed at the Opus rate from version 2.1.207, because the primary default moved to an Opus-class model. [FACT-CHECK: current alias resolution per provider and the Claude Code v2.1.207 default-model change, against the Claude Code model-configuration and cloud-provider documentation.]

Claude Code settings (as of August 2026, per Book 3 Ch 8 §4.3 and Ch 16 §4.5)

SettingPurposeFailure if missing
ANTHROPIC_DEFAULT_OPUS_MODEL, ANTHROPIC_DEFAULT_SONNET_MODEL, ANTHROPIC_DEFAULT_HAIKU_MODELPin full model IDs per environment, in managed settingsAliases follow the provider’s default
availableModelsThe enterprise model allow-list—
enforceAvailableModels: trueCloses the picker’s “Default” option, which otherwise resolves to the account’s runtime default; Book 3 cites v2.1.175 or laterThe allow-list leaks by design: users on “Default” reach a model outside the list
/statusShows the resolved provider, region and modelInclude its output in the Phase 3 evidence sample

A minimal managed-settings shape, with placeholders rather than real model IDs:

{
  "env": {
    "ANTHROPIC_DEFAULT_SONNET_MODEL": "<full-approved-model-id>",
    "ANTHROPIC_DEFAULT_HAIKU_MODEL": "<full-approved-model-id>"
  },
  "availableModels": ["<full-approved-model-id>", "<full-approved-model-id>"],
  "enforceAvailableModels": true
}

[FACT-CHECK: confirm the managed-settings key names, the env block placement, and the minimum Claude Code version for enforceAvailableModels against current Claude Code settings documentation.]

Composite illustration (Book 3 Ch 8, Bank A pilot). A workstation baseline shipped availableModels without enforceAvailableModels. For a week, four engineers whose picker sat on “Default” resolved to a model outside the approved list. The usage registry showed it before anyone reported it.

Name the enforcement boundary honestly (source claim, Book 3 Ch 8, Ch 16 §4.5). availableModels is enforced client-side, by software on an endpoint the user controls. Server-managed settings from the Anthropic admin console do not reach sessions running against Bedrock, Google’s Agent Platform or Microsoft Foundry (Book 3 Ch 8 §4.2), so those fleets are managed by MDM only, and Claude Enterprise’s server-enforced model restrictions do not apply on those routes either. Research Paper 3 (checked against vendor documentation on 10 November 2026) adds that Claude Code skips the server-managed fetch whenever a cloud-provider variable or a non-default ANTHROPIC_BASE_URL is set, so a fleet routed through the firm’s own gateway is device-policy only too (Research Paper 3). Book 3 Ch 18 §5 adds that Enterprise model entitlements can restrict models by role or organisation-wide, and the entitlement list should reconcile line for line against the registry. [FACT-CHECK: Anthropic admin post of 2 July 2026 on model defaults and entitlements, as cited in Book 3.] On every route, the gateway, not the endpoint, is the binding control: it refuses any model ID not pinned on a registered usage. Client settings reduce noise; they are not the control of record. See the model gateway and Research Paper 3.

Platform auto-upgrade settings

PlatformBehaviour (as reported by the books)Action
Azure OpenAI in Foundry, Standard deploymentsThree versionUpgradeOption values: OnceNewDefaultVersionAvailable (moves within two weeks of a new default), OnceCurrentVersionExpired (moves at retirement), NoAutoUpgrade (never moves; stops working at retirement). A deployment with no value behaves as OnceCurrentVersionExpired, so an unconfigured deployment still upgrades itself at retirementSet NoAutoUpgrade explicitly on every production deployment; track retirement so the deployment does not simply fail. Book 09 Ch 7 records a composite firm refusing seven production deployments that had no version set
Amazon BedrockModel identifiers carry the version (for example anthropic.claude-sonnet-4-5-20250929-v1:0), so calls are pinned by construction. Lifecycle states are Active, Legacy and End-of-Life; the EOL date is added to the model card when Legacy begins; after EOL requests fail and migration is not automaticTrack EOL dates as hard deadlines
Google Vertex AIEach GA model ID is listed with release date, retirement date and replacement model ID; retirement dates are not moved earlier than listedName the exact model ID; track retirement

Sources: Book 2 Ch 13 and Book 09 Ch 7, citing Microsoft Learn, AWS and Google Cloud documentation accessed November 2026. [FACT-CHECK: re-verify versionUpgradeOption values and default, Bedrock lifecycle states, and Vertex retirement-date policy against the vendor pages before relying on them; this page has not checked them directly.]

Track vendor deprecation and retirement

Two failures arrive from the provider side that a conventional release path never handled: the upgrade you did not choose and the retirement you did not plan for (source claim, Book 2 Ch 13). A deprecated version cannot be newly deployed; a retired one returns errors.

What to hold (recommendation). The catalogue, not the usage registry, holds deprecation facts, one record per model version per route: route, model ID, deprecation date, retirement date, replacement model ID, source page, and the date the facts were last checked. The registry then derives, for every usage, the earliest retirement date among its primary and fallback models.

What to do with it.

  • The weekly Phase 1 watch updates the catalogue record. [FACT-CHECK: Book 3 Ch 16 §4.1 states that as of August 2026 Anthropic published no consolidated model deprecation schedule; check whether Anthropic’s documentation now has a deprecations page and, if so, add it to the watch list.]
  • Open re-evaluation well ahead of any retirement date (Book 2 Ch 13; Research Paper 1 §5.3 factory rules). Recommendation: open Phase 1 no later than the point where a full cycle (pre-evaluation plus four weeks’ staging) can still finish before retirement.
  • Because notices are not a schedule, every pin also expires at Day +90 on the firm’s own clock.
  • A usage whose primary model retires before its next reassessment is escalated to the wrangler; a usage whose fallback retires is flagged too, because a fallback that no longer exists fails exactly when it is needed.

Build the evaluation suite in three layers

The suite is the operational evidence that an upgrade can be defended (author model, Book 3 Ch 16 §5). The general evaluation contract, trajectory tests and control-boundary tests are on evaluation and resilience; this section gives the model-change layer.

LayerContentDetectsOwned or signed off by
DeterministicAbout 50 known-good inputs with named expected outputs, drawn from the production workflow corpus; grows by one case per production incident for the next quarterRegression: something the prior model got right that the new one gets wrongWrangler and business owner, per case
ProbabilisticSame prompts run N times, typically 20; output distribution scored for variance, refusal rate, length and confidence-marker frequencySensitivity: output distributions that were stable and now oscillateModel risk (recommendation; the source names no signatory for this layer)
Business-outcome5–15 real workflow scenarios, each with explicit acceptance criteriaFitness for the actual job; a deterministic gain with a business-outcome loss is not an improvementWrangler and business owner write the acceptance criteria

Rules that make the suite defensible (source claim, Book 3 Ch 16 §5).

  • Build from your own production workload. Book 3 notes Anthropic’s own guidance that public benchmarks are saturating and that customers should build custom evaluations; a leaderboard delta is not evidence for your workflow.
  • Sequence. At pre-evaluation, run the layers in order, deterministic first, so a regression closes the gate before the others spend compute. At Day +60, run them in parallel with the business-outcome layer weighted.
  • Ownership and versioning. The suite lives in the model-risk repository, not the platform team’s, with a version number, changelog and signed model-risk attestation per release. The version run at pre-evaluation is the version run at Day +60; changing the suite mid-cycle invalidates the comparison and forces a re-evaluation.
  • Code scenarios. Where AI assistance is the default path for code change, swapping the model changes the effective authorship of everything merged afterwards, so the business-outcome layer carries a code scenario (Book 3 uses a 100-case merge-history corpus as the minimum shape).
  • Containment re-test. Each new pin re-runs the control-boundary tests (Book 3 Ch 17 §4.5; see above).
  • Divorce from production is the failure: a suite of synthetic or public cases that passes while production regresses. Recognise it by a case set not updated against the prior six months’ incidents and traffic.

Ownership

RoleOwns
AI platform ownerCatalogue; vendor watch; Day 0 registry update; rollout execution and gateway policy; reconciliation job; rollback steps 1–2
Head of model riskregulatory_class assignment; evaluation suite (repository, versions, attestations); pre-evaluation analysis; Phase 1 and Day +60 sign-off
Wrangler (named human per usage)business_process; signature on the classification; row reviews and re-affirmation; Phase 2–3 lead; Day +60 attestation; rollback attestation
Legal and complianceReview of classifications; supervisor-notification assessment (with the DPO) for upgrades touching T2+ envelopes
Data governancedata_classes_touched taxonomy; agreement on the grain
Procurement and third-party riskcontractual_party; contract-aligned decommission dates; critical-provider monitoring
Virtual CABGrading and approval of upgrade changes; rollout synchronisation approvals; post-rollback review
Agent Control BoardPhase 4 decision (retain / roll back / carry)
Internal auditQuery template; control-owner register and half-yearly reconciliation test; reconstruction protocol; half-yearly testability attestation

See accountability for how these seats sit in the wider ownership model.

Failure modes and recovery

FailureRecognise it byRecover bySource
Catalogue treated as model inventoryBoard and supervisor reporting by model count; no answer to “which models touch decision X?”Define the grain; build the registry; report at usage grain from the next quarterly model-risk forumBook 1 Ch 14 §8
Risk class on the modelEvery credit-relevant model high-risk for every use; general-purpose models minimal by default, including their high-risk usesMove regulatory_class to the usage row; require per-usage classification at registrationBook 1 Ch 14 §8
Tier mapped to modelThe same use case gets different controls depending on model choiceSet tier against use case; choose model inside itBook 3 Ch 14 §8
Registry without retirementAll rows “evergreen”; hundreds of rows, nobody sure which are live; third-party rows on old evaluationsEvergreen only with review date ≤ 12 months; quarterly escalation jobBook 1 Ch 14 §8
Registry not tied to a wranglerOwner is a team or mailboxowning_wrangler by HR identifier; reviews go to that personBook 1 Ch 14 §8
Usage registered, tools notUsage row with no link to the MCP tools it depends onCross-reference the MCP registry; one query, not twoBook 1 Ch 14 §8; see MCP and connector broker
No point-in-time historyOnly today’s row can be produced for a past incidentAppend-only, dated row versions; reconstruction protocolBook 1 Ch 14 §7
Auto-upgradeAlias in a production field; gateway on a different version; unexplained bill stepPin full IDs; switch auto-upgrade off; remediate as a controlled changeBook 2 Ch 13; Book 3 Ch 16 §7
Supplier floating nameVersion string on responses changes with no releaseExtend pinning to supplier-hosted models; no cases, no moveBook 09 Ch 9
Allow-list without enforcementUsers on “Default” reach unapproved modelsenforceAvailableModels: true; gateway refuses unregistered IDsBook 3 Ch 8
Suite divorced from productionSuite passes, production regressesBind suite growth to incident reviewBook 3 Ch 16 §7
Staged rollout collapsed to instantSplit jumps from 5% to 100% within daysGateway-enforced, signed traffic-split stepsBook 3 Ch 16 §7
Rollback breaks evidenceBoundary records’ model ID does not reconcile with gateway stateEvidence-portability check as the rollback gateBook 3 Ch 16 §7
Registry driftRegistry, gateway and envelope disagreeRegistry as source of truth; three-way reconciliation; amendment as the only pathBook 3 Ch 16 §7
Behaviour widening on a version stepNew actions or channels used with no envelope changeRe-test containment at every pin; log as envelope findingBook 3 Ch 17 §4.5

Acceptance checks

Registry

  • The grain definition is written and signed by model risk, agent platform, data governance and compliance.
  • Every production endpoint and every caller seen in the last 90 days of telemetry maps to at least one usage row; unmatched callers are open findings.
  • No production row has a blank business_process, regulatory_class, owning_wrangler or decommission_date; every evergreen row has a review date no more than 12 months out.
  • Two rows calling the same endpoint with different classes can each be defended from business_process, data_classes_touched and calling_agent_id.
  • Third-party model usages appear in the same registry, with contractual_party matching the Register of Information.
  • A wrangler’s “my usages” query returns in under 30 seconds; a sample supervisor question is answered within one working day.
  • A usage can be reconstructed as it stood on a past date, with its change ticket.

Runtime

  • A call for an unregistered, retired, stale-evaluation or out-of-data-class usage is denied with rationale (test each).
  • Every model invocation record carries usage_id, and that ID joins evidence records to the registry.
  • The reconciliation job runs on schedule and its last three runs show registry, gateway and envelope in agreement, or open remediation items.
  • A request naming a family alias, or a model ID not pinned on the usage, is refused at the gateway.

Upgrade lifecycle

  • Each open model change has a Phase 1 ticket, a pre-evaluation recommendation, and a dossier naming its rollback thresholds before staging starts.
  • The gateway shows 5/25/50/100% steps with signed policy artefacts and the platform owner’s co-signature.
  • The Day +60 suite run used the same suite version as pre-evaluation, and the wrangler attestation is filed.
  • A rehearsed rollback produced all six artefacts, including a passing evidence-portability replay, within the stated time limits.
  • Every production Azure OpenAI deployment shows NoAutoUpgrade set explicitly; every Claude Code managed-settings profile sets enforceAvailableModels: true and pins full model IDs.
  • Every pinned model and fallback has a recorded retirement date (or a recorded “none published” with the check date), and no pin is older than Day +90 without a Phase 4 decision.

Provenance and refresh

The registry, grain, classification principle, population sequence, sizing and anti-patterns come from Book 1 Chapter 14 and Appendix B. The upgrade lifecycle, evaluation suite, thresholds, rollback playbook, alias drift and Claude Code settings come from Book 3 Chapters 8, 14, 16, 17 and 18. Release pinning, update scopes and platform auto-upgrade behaviour come from Book 2 Chapter 13 and Research Paper 1 §5.3; the supplier floating-name case comes from Book 09 Chapters 7 and 9. The page restructures these for operating decisions rather than reproducing them.

Source conflicts and how this page resolves them.

  • Row grain. Book 1 App B’s collection 12 description (one record per model, agent and time-window) differs from its own registry section (deployment, caller, business process). This page follows the usage tuple and keeps time as row history.
  • Field set. Book 1 App B’s 18 fields and Book 3 Ch 16 OD-3’s 18 fields are different sets. This page uses Book 1’s as the usage row and Book 3’s lifecycle fields as a pin-window extension. Book 1 Ch 14 §1’s scene lists yet another set (including call counts and a re-affirmation date) and §4.2 describes blocks differently from App B; this page follows App B.
  • Change-tier naming. Book 3 Ch 16 calls upgrade change classes “Tier 2” and “Tier 3”, reusing the risk-tier vocabulary. This page describes them by their effect and keeps T0–T4 for risk tiers.

Vendor claims. Model names, alias resolution, Claude Code versions and settings, Azure, AWS and Google lifecycle behaviour, and Anthropic availability and ZDR rules are dated to the book that reports them and marked [FACT-CHECK]. They change with vendor releases; this page’s refresh trigger is a vendor release. Regulatory classification and dates are on the regulatory map.

Source material

  • Book 1, Chapter 14, AI Model Inventory and Foundry Governance (§3 thesis; §4.1–4.4 catalogue versus registry, grain, classification, control-plane integration; §6 composite registry; §7 Monday morning and sizing; §8 anti-patterns)
  • Book 1, Appendix B, Operating Schemas: AES, Control Plane, Model Usage, Data Dictionary and Change Dossier (the 18-field model usage registry; population sequence; collection 12 model_usage)
  • Book 2, Chapter 13, Release Engineering and Change Resilience (release pinning; update scopes; automatic-upgrade rule; the model that upgraded itself)
  • Book 3, Chapter 8, Claude Code at the Workstation (availableModels and enforceAvailableModels; third-party-provider settings asymmetry)
  • Book 3, Chapter 14, Risk Tiers and the Use-Case Taxonomy (§8 tier mapped to model; model selection orthogonal to tier)
  • Book 3, Chapter 16, Model Upgrades as Change Events (four-phase lifecycle; evaluation suite; per-route mechanics; advisor pattern; OD-1 cadence table; OD-2 rollback playbook; OD-3 registry row)
  • Book 3, Chapter 17, Incident Response for Agent Failures (§4.5 behaviour change after a model upgrade)
  • Book 3, Chapter 18, Cost, Performance and Gateway Operations (§5 model entitlements and pinned defaults; routing by complexity)
  • Book 09 (Everything Is Green), Chapter 7, The Machine That Says No (pinning production deployments)
  • Book 09 (Everything Is Green), Chapter 9, One Query (supplier floating names; no cases, no move)
  • Research Paper 1, A Taxonomy for a Reuse-Driven AI Factory (§5.3 model lifecycle and release pinning)
  • Research Paper 3, Securing Claude Code in the Enterprise (where server-managed settings stop)
  • Supporting Tools, Deployer Manifest v0.1.0 and Audit Evidence Schema v0.1.0

← All AI Delivery Factory guidance

On this page

  1. The model in one table
  2. Keep two records: catalogue and usage registry
  3. Define the grain before anything else
  4. Classify risk on the usage, not the model
    1. Regulatory class, risk tier and model choice are three different decisions
  5. The 18-field usage row
    1. The three load-bearing fields
    2. Lifecycle fields for the upgrade cycle (a second layer)
  6. Populate the registry in seven steps
  7. Wire the registry to the policy decision point
  8. Retire usages and keep point-in-time history
    1. Retirement triggers
    2. Re-affirmation cadence
    3. Keep point-in-time history
  9. Third-party model usages
  10. Treat model selection as a separate decision
  11. Run every model upgrade as a four-phase change
    1. Size the change by update scope
    2. The four phases
    3. Route mechanics (as of August 2026, per Book 3 Ch 16 §4.5)
  12. Roll back on named thresholds and prove evidence portability
  13. Pin full model IDs and close alias drift
    1. Alias drift across routes (as of August 2026, per Book 3 Ch 16 §4.5 and Ch 18 §5)
    2. Claude Code settings (as of August 2026, per Book 3 Ch 8 §4.3 and Ch 16 §4.5)
    3. Platform auto-upgrade settings
  14. Track vendor deprecation and retirement
  15. Build the evaluation suite in three layers
  16. Ownership
  17. Failure modes and recovery
  18. Acceptance checks
  19. Provenance and refresh

All pages in Govern and prove →