Browse the reference ⌄
Implementation Guide published

Govern APIs, Data Products and Retrieval Agents Can Trust

Turn interfaces and data into owned, versioned capabilities that applications and agents consume under identity, entitlement and evidence controls, and give agents access to meaning, limits and authority, not only bytes.

Version
0.2.0
Owner
DEIXEI
Last reviewed
Reading time
27 min

For Cto · Ai Architect · Platform Engineer · Data Leader · Security Engineer · Risk And Assurance · Internal Audit

Sources and details
Source works
Book 1 · Book 2 · Book 3 · Supporting Tools · Research Series · Book 09
Source chapters
3, 4, 10, 12, 13, 16
Source appendices
01:B
Tool specifications
audit_evidence_schema@0.1.0
Refresh trigger
source change

Full source list ↓ Source index →

Applications and agents need dependable ways to use the firm’s capabilities and data. An API is a boundary of accountability: it identifies an owner, a contract, an authorisation model, a lifecycle and the evidence a consumer can rely on. A reachable database table is not a governed data product, and a retrieval index is not a substitute for source entitlements.

Agents raise the bar in a specific way. Access to data is a solved problem in most firms; access to meaning is not. What does this column represent, in what unit, is this the master or a copy, how fresh is it, what is known to be wrong with it, and may an agent use it at all? A tenured employee answers these tacitly. An agent does not ask; it proceeds (Book 1 Ch 16 §4.2). This page sets out the artefacts that make meaning, limits and authority machine-readable, and the controls that stop agents treating plausible output as record.

Standing of the content. The six-layer model, the operational antibodies, the 17-field dictionary, the Authority Process, accuracy thresholds and sizing figures are author proposals and experience-based estimates from Book 1. The minimum data estate and the reuse rules are author recommendations from Book 2. Bank A, Asset Manager X and Insurer I are composite institutions; the Book 09 material is a composite narrative. Classification, lanes and inheritance rules are on data classes and lanes; legal duties are on the regulatory map.

Why agents make the abstraction gap consequential

Every schema is a snapshot of the business at the moment it was built, and the business drifts faster than the schema is revised. Definitions change at the speed of policy, schemas at the speed of releases, master data at the speed of stewardship. The most hazardous form is a schema that looks updated while the meaning has not moved (Book 1 Ch 3 §4.2). Humans compensate silently with tacit knowledge. Agents reason over the abstraction as if it were reality, and three properties make that consequential (Ch 3 §4.3):

PropertyWhat happensControl that answers it
ScaleA systematic misreading becomes a systematic misaction across millions of recordsRecord-level metadata and refusal rules; accuracy thresholds
PlausibilityFluent, consistent, confident output bypasses lay reviewKnown-limitations register read before acting; authoritativeness field
ComposabilityOne agent’s misread becomes the next agent’s premise, laundered through layers of reasoningLineage queryable at runtime; the Authority Process before anything becomes record

An agent reading a stale or drifted store cannot tell from the data that it is stale; nothing in a schema announces its own obsolescence. The remedy is not a better abstraction (the next migration produces its own gap) but making the gap legible to the agent (Ch 3 §3).

Separate six data-governance layers and keep the flow one-way

Book 1 Ch 16 §4.0 identifies confusing adjacent layers (the data product with the retrieval index, agent memory with the authoritative record) as the most common cause of agents inventing facts the firm then has to defend. Give each layer one owner and one governance artefact.

LayerWhat it isOwnerGovernance artefact
Source systemSystem of record where data originatesSystem owner (business area)Source-system change management
Data productCurated, documented, versioned dataset from one or more sourcesData-product ownerData-product SLA and the agent-readable dictionary entry
Semantic contractAgreed meaning of each field, its valid values and what it does not meanData-product ownerThe 17-field dictionary entry
Retrieval indexSearchable form for agents (vector, graph, structured)Platform teamIndex refresh cadence and access policy
Agent memoryWhat the agent retains within and across interactionsAgent wrangler (named owner)The memory side of the bounded-agency envelope
Authoritative recordThe firm’s record of decision that survives the interactionRecords ownerThe Authority Process

Flow rule. Data flows one way, from source through product and index into memory. The only return path from agent output to an authoritative record is the Authority Process. Agents may produce explanations, suggested actions, draft records, proposed classifications, recommended journal entries, drafted contracts and hypotheses; only the firm’s governed process turns them into decisions, executed actions, posted records, applied classifications, booked entries, signed contracts and confirmed facts (Ch 16 §4.4).

The Authority Process (Ch 16 §5.2) has one instance per record class, the same for agent, human, vendor or integration drafts, with named human and machine signatories at each step:

StepPurposeSignatory
1. ValidationCompare the draft with the master (dictionary field 8 = master) and flag mismatchesValidation tool; the producing agent’s wrangler
2. Business-rule checksApply rules for the record class, such as finding references or four-eyesRules engine; rules owner; second reviewer where required
3. Data-quality checksCompleteness, consistency, conformance to the schema (field 14)DQ engine; receiving dataset’s steward
4. ReconciliationAgainst parents, subsidiaries, prior versions and related entitiesReconciliation tool; steward
5. Human approvalA named approver with authority for the record class, rationale mandatoryNamed approver; no agent substitutes here
6. Audit loggingFull trail written immutably against the Audit Evidence SchemaLogging service; audit-evidence owner
7. Versioning and postingPost through the dataset’s write API, which only this process may callWrite API; data-product owner
8. Separation of duties and evidenceApprover is not the wrangler; exception path with standing register, override authority and timed re-reviewControl owner; internal audit

Agents may drive steps 1–4; step 5 is human; steps 6–8 are mostly automated with human owners.

Gate agent access on a minimum data estate

Book 2 Ch 12 §7 answers what does the data estate need before agents scale? with five items and no substitutes. Use them as an admission gate per dataset:

  1. Contracts. Every dataset an agent touches has a versioned data contract and a named owner. No contract, no agent access.
  2. Semantics. Meaning is delivered machine-readably through the agent-readable dictionary into the agent’s standing context, so it does not infer meaning from column names.
  3. Lineage carried as evidence. Where a figure came from is referenced from the audit record’s evidence pointers, queryable rather than documented.
  4. Quality SLOs. Freshness, completeness and accuracy thresholds per data product, tested by the governance loop like any policy (see policy, audit and actuals). In Book 2’s worked example (Ch 12 §6, Asset Manager X, composite) a feed that misses its declared refresh window blocks the agent’s draft exactly as any other policy breach would.
  5. Identity-aware access. Every read resolved against the actor’s entitlement at query time, with one grounding data API in front of every grounding store so a deletion request has one place to land.

Treat APIs as products

An API record includes a named owner, purpose, schema, authentication and authorisation model, versioning and compatibility policy, rate limits, error conventions, monitoring, lifecycle state and known consumers. Use semantic versions only when the team has defined a compatible change and runs a deprecation window. A contract that many agents call cannot break casually: versioning becomes load-bearing, and a careless change is an estate-wide incident at machine speed (Book 2 Ch 12 §4). Agents do not read release notes, so a major-version migration window must be announced in the standing context agents read. Refuse to publish an API without an owner, a version and a consumer record.

Choose between sharing code and sharing a capability

QuestionFavour code reuseFavour a running capability behind an API
Must the logic be identical across consumers?A stable, low-stakes utility may tolerate independently versioned copies.Regulated rules, prices or client-visible figures that must change consistently.
How often will consumers need updates?Low change rate, few known consumers.Frequent changes across many consumers.
What does a divergent copy cost?Low impact, easy to detect.High blast radius or hard-to-reconcile outputs.
Will agents consume it?Local execution is required and the library contract is dependable.Usually easier to discover, authorise, observe and update through a service contract.
Can the firm operate the runtime dependency?Prefer local code where latency, offline use or availability rule out a call.Only when the service team can meet the dependency’s service objectives.

Service reuse trades copy drift for a runtime availability and ownership obligation; code reuse avoids a runtime call but makes every consumer responsible for adopting versions. Record the rationale in the architecture decision and the BOM.

Code reuse creates independent copies to maintain; capability reuse shares one running implementation through an API contract.

The reuse decision changes the dependency graph and the way change is governed. Source: The AI Delivery Factory, Chapter 12, Figure 12.1.

Govern the fork as written policy, where agents read it. Copying is free for a coding agent, so an ungoverned estate drifts to N copies of everything. Book 2 Ch 12 §7 and Book 09 Ch 4 (composite) give the operating form:

  • Rule. Anything that carries a rule or a number that must match everywhere runs once, behind a versioned contract with an owner, and its data carries an owner, a stated freshness and a stated meaning. Pure, stable, low-stakes utilities may stay as copies with a known-copies list.
  • Write the decision into agent context. “There is a performance-calculation service; call it” in the agents’ standing context prevents the tenth copy.
  • Count duplicates. Search the estate for an important function on a schedule; more than two copies of something that carries a rule or number marks a capability that wants to be a service. In the Book 09 composite the weekly count opens a review, not an automatic extraction, so people who will run the service decide.
  • Retire copies at the speed of their own change, drifted copies first, and measure success as a falling duplicate count.
  • Wrap once at the right layer. A system of record gets exactly one MCP server, owned by the team closest to it, listed with owner, version and data classification, reached only through the connector broker. A second wrapper is a contribution to the first, never a fork.

Make data contracts machine-readable

A data contract records what a data product means and what consumers may rely on: field definitions and allowed values, owner, source and lineage, quality thresholds for freshness, completeness and accuracy, sensitivity classification, retention, permitted uses, schema version, definition version and change policy. Publish it with the data product.

Keep a golden source for definitions and promote approved changes through environments. A governed source feeds analytics stores, operational stores, APIs and the agent-readable dictionary while preserving one reviewable change history. Consumer-specific schemas may differ; meaning and ownership must not silently diverge.

Fit the operational antibodies

Book 1 Ch 3 §5 names five artefacts that make the abstraction gap legible. They are principles; the dictionary below is their machine-readable form.

AntibodyImplementationAgent behaviour it enables
Metadata that travels with the dataRecord-level effective-from, effective-to, definition-version, source-system, last-validated-atRefuse to act on a record whose definition version is older than the policy under which it is asked to act
Queryable lineageLineage exposed programmatically through the catalogue (Purview Unified Catalog, AWS Glue Data Catalog with Lake Formation, Google Dataplex Universal Catalog, per Book 1 Ch 3)Answer where did this number come from before treating it as fact
Named ownershipA named steward per domain, entity and authoritative source, not a roleA human escalation point when the gap is suspected; roles rotate, seats go empty
Known-limitations registerPer data product (below)Read before acting; apply compensating behaviour
Versioned business glossary, separate from the schemaGlossary terms with their own steward and versionTranslate a business question into a schema query; audit schema–meaning alignment

Refusal rule (record-level). Before acting on a record, the agent (or the tool that serves it) compares the record’s definition-version with the definition required by the current policy. If older, the agent does not propose the low-risk outcome; it routes to the defined fallback. In Book 1’s composite KYC case, legacy records carried pre-2022 customer-type values with beneficial ownership defaulting to unknown through a catalogue rule the field did not show; the fix was a known-limitations entry, an effective-from-version on every record, and a guardrail refusing low-risk for records older than the current guideline, defaulting to enhanced due diligence (Ch 3 §6). Agent confidence fell; actual safety rose.

Known-limitations register. One short, structured register per major data product or service, in whatever catalogue the firm already runs, containing:

  • what the product cannot be used for;
  • what is known to be wrong;
  • what is being worked on;
  • what compensating controls exist;
  • the named steward’s sign-off, reviewed quarterly (dictionary field 10).

The agent reads it before acting. If the catalogue cannot yet be queried by an agent, the register starts as documentation and becomes known_limitations in the dictionary entry. Book 1 Ch 3 §5 places the antibodies on the Memory side of the bounded-agency envelope: they keep the agent’s picture of the world honest about its own gaps, which no model upgrade can do.

Run the steward gap-statement exercise

For each of the three highest-stakes entities (in a large firm typically client, product or instrument, and counterparty or legal entity; in a 1,000-person specialist firm typically customer or investor, account or mandate, and transaction or policy), write a one-page, two-column gap statement: what the model says and what the business now means (Book 1 Ch 3 §7).

  • The named steward attends in person, not a delegate. About ninety minutes per entity.
  • The statement is descriptive, not a roadmap. If the steward cannot describe the gap, that is the finding.
  • Output: the gap statement plus a first known-limitations register per entity, signed by the steward.
  • Ownership: the data office with CIO sponsorship in a large firm; the CIO personally with the head of operations and one engineer in a small firm. A working group with no named owner is not acceptable.

Publish the 17-field agent-readable data dictionary

The dictionary is the semantic contract layer: one entry per dataset, machine-queryable, kept current by the owning domain, fronting both human APIs and agent MCP servers (Book 1 Ch 16 §5.1; Appendix B). Field names, types and lifecycle below are as the source gives them.

#FieldTypePurposeLifecycle
1dataset_idstring (catalogue ref)Unique dataset identifierSet at registration; immutable
2business_namestringPlain-language business nameAnnual review
3owning_domainstring (domain ref)Accountable data-mesh domainOn organisation change
4data_classificationenum (public / internal / confidential / restricted) plus sensitivity tagsDrives access and tokenisation decisionsAnnual and on schema change
5refresh_frequencystring (real-time / hourly / daily / weekly / monthly / event-driven)How current the data isAnnual
6lineage_upstreamarray of dataset_idsWhat it is derived fromOn pipeline change
7lineage_downstreamarray of dataset_ids and report_idsWhat consumes itOn consumer change
8authoritativenessenum (master / reference / authoritative-copy / derived / operational / draft / unverified)Position in the authority hierarchySet at registration; changed only with governance approval
9semantic_glossary_anchorstring (glossary term ref)Glossary term defining business meaningWhen the glossary changes
10known_limitationsarray of stringsCaveats and edge cases consumers must knowQuarterly
11accuracy_slaobject (target %, measurement method, last-measured value, alert threshold)The owning domain’s accuracy commitmentMeasured continuously; reported monthly
12mcp_endpointURL or nullAgent access pathAt MCP publication
13api_endpointURL or nullHuman-facing APIAt API publication
14schemastring (schema ref) or inlineTechnical schemaVersioned with each change
15versionstring (semver)Contract versionBumped on contract change
16regulatory_classobject (DORA critical-or-important-function flag, AI Act class if applicable, GDPR sensitivity, sectoral overlays)Regulatory frameAnnual and on regulation change
17retention_and_agent_allowed_listobject (retention policy id, allowed-agent-uses array)Retention and the enumerated agent usesAnnual and at incident

Blocks: identity and ownership (1–3), sensitivity and timeliness (4–5), lineage and authority (6–9), operational contract (10–17). Map field 4 to the firm’s C1–C4 classes and carry flags as sensitivity tags, as set out on data classes and lanes.

The two load-bearing fields.

  • Field 8, authoritativeness, tells the agent this is master, do not summarise it from a knowledge-base extract; this is operational, do not treat it as master; this is unverified, do not cite it without a master cross-check. Expect resistance: domains must stop calling things master that are not.
  • Field 17, the agent allowed list, is enumerated and empty by default. A read tool such as get_supplier_by_id may be listed; a write such as write_supplier_record is never granted to an agent, because only the Authority Process writes to authoritative datasets.

Enforcement points.

  • The MCP server and API pull schema and tool descriptions from the entry, so the descriptions agree everywhere.
  • The broker validates that the retrieved entry permits the calling envelope; the gateway validates the route; the wrangler validates the envelope mapping at attestation (Book 3 Ch 10 §7).
  • Every retrieval carries the entry to the agent, and every agent action is logged with the entry’s identity, last-validated date and status, so audit can replay the meaning the agent acted on. The AES dataEntry object records dataset_id, classification, operation and optionally lineage and retention.
  • Require the agent to cite the dictionary entry of every dataset it reads.
  • Cross-check at registration that the model usage registry’s data_classes_touched matches field 4 for every dataset the agent reads.

Staleness controls (Book 3 Ch 10 §7, §9; Bank A composite): quarterly attestation by the named data owner; an 18-month maximum entry lifetime with automatic deprecation; a retrieval-blocked-stale-dictionary event when an agent reaches for an expired entry; a CDO dashboard of ageing entries. A dictionary nobody updates lies, and is worse than none because the agent acts on it confidently.

Two 17-field dictionaries in the sources. Book 1 defines the entry per dataset (fields above). Book 3 Ch 10 PR-4 defines a different 17-field entry per data class: name, description, owner, upstream lineage, sensitivity class; permitted use cases, permitted envelopes, permitted routes; retention obligation, lawful basis, data subjects, recipients, transfers, retention period (server and endpoint); last-validated date, validation cadence and status (Active / Under review / Deprecated / Withdrawn). This page treats them as two layers: the dataset entry is the semantic contract for a data product, and its field 4 points to a class entry that holds the class-wide permitted routes, envelopes and legal posture. Do not merge the field lists. Book 1 Appendix B states that a normative JSON Schema for the dataset dictionary is published at deixei.com; it is not yet among the schemas in the site’s schemas and tools reference.

Set accuracy thresholds below which agents invert value

For human consumers, quality and value move roughly together because people catch most errors. For agents the error rate is industrialised: in Book 1’s illustration, scaling a process from 1,000 to 10,000 transactions a day on 95% accurate data moves the error count from about fifty (most caught by humans) to about five hundred (none caught). Below a decision-relevant threshold, risk outruns value (Ch 16 §4.1).

Data typeReference threshold (author’s experience, not a standard)
Master customer dataAbout 99.5% record-level accuracy on the fields the agent uses
Reference data (currency, ISIN, LEI, country codes)About 99.99%
Asset-manager composite: positions / attributions / house view99.95% / 99.9% / current as of this morning
Unstructured knowledge basesLower in absolute terms, higher in stakes; the agent cannot tell an obsolete chunk from an authoritative one

Apply it: set the threshold per dataset, process and regulatory class in accuracy_sla; measure continuously; block or restrict agent uses in field 17 while measured accuracy is below threshold; improve quality before deployment, which the author treats as a precondition, not a guideline.

Pair the API and the MCP server from one dictionary entry

Data mesh, APIs and MCP are three layers of one regime per data product, not competing patterns (Ch 16 §4.3): the mesh is organisational (domains own and publish data products; the central function owns platform, dictionary and policy); the API is the human-facing technical layer; the MCP server is the agent-facing technical layer, with tool names such as get_supplier_by_id and descriptions pulled from the dictionary. One owner, one OAuth flow, one set of rate limits, one audit trail.

Procedure for the first pairing (Ch 16 §7): pick one of the three highest-stakes datasets; populate its entry; build the MCP server through the gateway and the human API through API management, both reading schema and descriptions from the entry; declare per tool whether it reads or acts so the broker can grant read tools to advisory agents and withhold acting ones (Book 2 Ch 12 §4).

Platform equivalents, as of May 2026 per Book 1 Ch 16 §4.3. [FACT-CHECK: product names, capabilities and maturity are current vendor claims.]

ConcernMicrosoft AzureAWSGoogle Cloud
Registry for data products, glossary and dictionaryPurview Unified Catalog (governance domains, data products, glossary terms)Glue Data Catalog (technical) plus DataZone (business metadata, glossary, subscription approval)Dataplex and Dataplex Universal Catalog (glossary, lineage, tag templates)
Agent-facing MCP pathAPI Management MCP featureBedrock AgentCore GatewayApigee MCP support with Vertex AI Agent Builder
AI layerPurview AI hub; Purview SDK in Agent Framework (Ch 16 reports it released June 2026; Ch 13 reports GA announced for late June 2026)——

The author’s caution: an AI security hub is an observability and protection layer, not the agent-data primitive; the primitive is the dictionary entry per data product.

Make integration patterns explicit

Select the pattern for the interaction rather than letting each urgent feature create a private connection:

  • synchronous API for timely request/response;
  • events or messaging where producers and consumers should not block each other;
  • managed batch or file exchange for volume and legacy interfaces;
  • workflow orchestration for multi-step processes;
  • data-product publication for governed datasets;
  • a governed vendor integration boundary for connections outside the institution.

Give each pattern a recipe, owning team and evidence path. Agent context should point to the approved pattern and interface; a coding agent will otherwise wire a plausible connection to whatever it can see.

Onboard external feeds as data the firm depends on, not as an integration. In the Book 09 composite, a custodian feed onboarded as plumbing failed silently; the recipe was changed so that nothing is built until the sender and the firm agree a feed promise, field by field: what each record shows (for example booked versus closing balance), arrival time each business day, what a well-formed record and file look like (including a trailer record count), and what the sender does when it sends a bad one. A boundary check then refuses non-conforming files at arrival with the reason recorded. Number every file per business day so replacements apply in sequence and duplicates are refused rather than silently undoing a day’s processing. Test with deliberately malformed files before go-live.

Client-facing read-only endpoints. Where the firm already describes its reporting services machine-readably, a read-only endpoint per client, bound to that client’s mandates, resolved by the firm’s identity service and recorded per call, can reuse the reporting service behind the portal (Book 09 Ch 3, composite). In that composite the pilot ran with two institutional clients, the adoption lead compared endpoint calls with the old portal logins every week, and the endpoint was cheap because the reporting services were already described machine-readably, in contrast to an exception tool that had no such description. The chief risk officer’s team chose to govern the endpoint as a client-facing AI system, a deliberately cautious reading of the transparency duties. Whether those duties apply to a given endpoint is a per-case analysis on the regulatory map.

Put retrieval behind a governed data API

Place a grounding data API between product code and the underlying stores. It resolves the caller’s identity, selects the corpus or tenant, applies current entitlements, filters results and records what was returned. Agents, orchestrators and applications do not query vector stores directly with their own copy of the filtering logic. The filter applies whether retrieval happens before a model call or inside an agent’s loop (Paper 09 §6.1, citing Microsoft’s multitenant RAG guidance).

Isolation choiceAppropriate starting contextMain trade-off
Store per tenantClient data held as processor; information barriers; C4 corporaMore deployment and operational overhead
Shared multitenant storeMany internal domains at C2–C3, every query carrying an enforced tenant and entitlement filterA filter defect exposes data across tenants; test the boundary directly
Shared corpusPublic or firm-wide materialRequires confidence that restricted content cannot enter

Resolve user or delegated-user entitlements at query time. Carry source identifiers, dictionary entry and data labels through parsing, chunking, enrichment and indexing. Stamp the store’s class on every result so the gateway can apply the highest-class rule and inheritance rule 1 (Paper 09 §5.3): derived artefacts inherit the highest class of their sources. The access record captures caller, agent or product identity, corpus, permitted query context, documents returned and decision outcome without logging unnecessary sensitive content.

Applications, orchestrators and agents call one grounding data API that routes to isolated stores, enforces tenant and user filters, and logs what was served.

A single governed path centralises routing, entitlement checks and access evidence. Source: The AI Delivery Factory, Chapter 12, Figure 12.2.

Version and evaluate the retrieval pipeline

Treat an index as a build artefact. Version the configuration and source inputs for every stage, then test the index against golden questions owned with subject-matter experts.

PhaseVersioned inputsExample quality signal
PrepareConnectors, parsers, OCR and source filtersExtraction completeness and source coverage
ChunkStructure-aware or semantic strategy and parametersRecall on representative questions
EnrichMetadata and generated summariesFilter accuracy and ranking lift
EmbedModel, version and dimensionsRecall against latency and cost
RetrieveSearch mode, result count and rerankerPrecision, recall and entitlement-filter correctness
PromptGrounded-answer instructions and citation formatGroundedness and citation accuracy
EvaluateDataset version, scoring method and reviewer inputRelevance, completeness and quality by data class

Changing a parser, chunking strategy, embedding model or index configuration creates a new retrieval release. Promote it through evaluation and approval gates and keep the previous version available for rollback where the operating model requires it.

The retrieval pipeline has seven versioned phases; evaluation feeds back into preparation, chunking and retrieval, and source pointers support deletion.

Retrieval quality and deletion handling belong to the pipeline and the index release, not only to the prompt. Source: The AI Delivery Factory, Chapter 12, Figure 12.3.

Maintain a source-to-derived-artefact map identifying chunks, embeddings and caches derived from each document. Define how deletion, correction, changed entitlement or expired licence propagates, how query-time suppression works during the cascade, and what evidence confirms completion. Legal and privacy owners determine the retention and erasure interpretation; engineering makes the approved action executable and observable.

Roll out the dictionary in sized iterations

Book 1 Ch 16 §7 gives a sequence and author estimates (not benchmarks):

  1. Pick the three highest-stakes datasets: the relevant master (customer, supplier, counterparty, holdings or policy), the feed for the largest regulatory submission, and the master for the process most aggressively automated with agents.
  2. The owning domain populates the 17 fields; the central data function reviews; the agent platform team checks the allowed list; the CDO signs.
  3. Wire one MCP server and one API to one dataset from its entry.
  4. Run the Authority Process on one real generated artefact end to end with named signatories; time and log every step; fix gaps before scaling.
Parameter~50,000-employee firm~1,000-employee specialist firm
First iteration8–12 weeks, team of six (data architect, two data-product engineers, agent platform engineer, governance lead, programme manager) plus domains4–6 weeks, team of three
Steady-state entries2,000–8,000 across 30–60 domains100–400 across 5–12 domains
MCP-published entries200–80030–80
Recurring effort3–5 FTE central enablement plus about 0.1 FTE per data product1–2 FTE

The rest of the dictionary follows in two further iterations through the year. Compliance owns the field-to-regulation mapping and a quarterly per-domain attestation; internal audit owns a query template that reconstructs the Authority Process chain from one record reference and a half-yearly sample of posted records (Ch 16 §7).

Failure modes

Anti-patternRecognise it byRecover by
ETL as ground truth (Book 1 Ch 3 §8)Asked what a value means, an engineer answers with a SQL CASE statementExtract the definition, write it in the glossary, get it signed, version it
Schema as the contractA value is renamed (retail to individual) with no schema change; integrations pass, meaning shiftsCarry definition-version beside schema version; consumers refuse definition versions they have not been validated against
Integration as semantic transferA migration ticks “no data loss” but meaning stayed with the old teamSemantic conformity test: for the highest-stakes entities, a sample of business questions must give the same answer on both sides, run by the named steward, not the migration team
Agent as semantic auditorAn LLM-produced schema mapping is used without validationTreat it as generative output; it becomes authoritative only through the Authority Process
Agent reads the data lake without a dictionary (Ch 16 §8)Agent reaches operational exhaust, prior agent outputs and obsolete snapshotsRevoke broad read; populate entries for needed datasets; access only through MCP; require citation of the dictionary entry
MCP servers but no glossarysemantic_glossary_anchor null on most entries; two domains mean different things by customerGlossary first; anchor every entry; quarterly glossary review
Generative output treated as recordDrafts posted because posting is automated; confidence used as a quality gateReintroduce the Authority Process; reconcile the period; forbid agent writes except through it (field 17)
Authority Process bypassed under time pressureExceptions become routineStanding exception register, named authority, timed re-review, board-visible exception counts
Dictionary nobody updates (Book 3 Ch 10 §9)Last-validated dates past cadence; agent acts on a withdrawn classEnforced validation dates, auto-deprecation, attestation in the owner’s RACI, ageing dashboard
Direct data access by convenience (Book 2 Ch 12 §8)An incident finds an unknown consumer of an unexposed tablePut a data product with a contract in front; retire the direct read as debt
The second wrapperTwo MCP servers over one system, or broker logs showing traffic to an unlisted oneMerge into the owner’s server; broker refuses routes to unlisted servers
The index nobody can delete fromErasure requires a full rebuildMove the corpus behind the grounding API with source pointers on every fragment

Acceptance checks

  • No agent reads a dataset without a versioned contract, a named owner and a populated dictionary entry; the allowed list is enumerated and empty by default.
  • Fields 8 and 11 are honestly populated for every master dataset, and measured accuracy is compared with the threshold.
  • Records used by agents carry effective-from, definition-version and last-validated-at, and a test proves the agent refuses a record older than the current definition.
  • Each high-stakes data product has a known-limitations register signed by a named steward, and agents demonstrably read it before acting.
  • The three highest-stakes entities have signed gap statements.
  • The glossary is versioned separately from the schema and every dictionary entry anchors to a glossary term.
  • Migrations include a steward-run semantic conformity test; no unvalidated LLM mapping is in production.
  • The API and MCP server for each published dataset derive schema and descriptions from one entry; tools declare read or act.
  • Only the Authority Process can call write APIs on authoritative datasets; a sample of posted records shows a named step-5 approver who is not the wrangler.
  • Stale entries are blocked at retrieval and raise an event.
  • Known-copies lists exist for permitted code copies; the duplicate count is running and falling.
  • External feeds have a signed feed promise and a boundary check proven with malformed test files.
  • Retrieval uses the caller’s current entitlements, cannot bypass the grounding API, stamps store class on results, and passes cross-tenant negative tests.
  • Index builds are reproducible from pinned inputs and evaluated before promotion; deletion and correction cascades cover derived data and produce evidence.

Source status. The six-layer model, antibodies, dictionary, Authority Process, thresholds and sizing are author proposals and experience-based estimates from Book 1, not independently validated; the author reports that two years of design reviews have not required adding a dictionary field or an Authority Process step. The minimum data estate, reuse rules and retrieval pipeline are author recommendations from Book 2. Bank A, Asset Manager X, Insurer I and the Book 09 firm are composites. Vendor product statements are dated and marked for verification. GDPR and retention interpretations require review by the institution’s legal and privacy owners.

Source material

  • Book 1 (Building Organizations for the Age of AI), Chapter 3, Software as an Abstraction of Reality (§§3–8)
  • Book 1, Chapter 16, Data as the Enterprise Multiplier (§§3–5, §7, §8)
  • Book 1, Appendix B, Operating Schemas: the 17-field agent-readable data dictionary
  • Book 2 (The AI Delivery Factory), Chapter 12, APIs, Data, and Integration (§4, §5, §7, §8)
  • Book 3 (Deploying Claude Safely), Chapter 10, Data Classification, Retention, and DLP (§7, §9, PR-4)
  • Book 09 (Everything Is Green), Chapter 3, The Quote Turned Face Down; Chapter 4, Right About the Code, Wrong About the Product; Chapters 12–13, Who Holds the Dial and The Inspection (composite narrative)
  • DEIXEI Research Series, Working Paper 09, Data Classes, Processing Lanes and Controller/Processor Roles for AI (§5.3, §6.1)
  • Supporting Tools, Audit Evidence Schema v0.1.0 (dataEntry)

← All AI Delivery Factory guidance

On this page

  1. Why agents make the abstraction gap consequential
  2. Separate six data-governance layers and keep the flow one-way
  3. Gate agent access on a minimum data estate
  4. Treat APIs as products
  5. Choose between sharing code and sharing a capability
  6. Make data contracts machine-readable
    1. Fit the operational antibodies
    2. Run the steward gap-statement exercise
  7. Publish the 17-field agent-readable data dictionary
  8. Set accuracy thresholds below which agents invert value
  9. Pair the API and the MCP server from one dictionary entry
  10. Make integration patterns explicit
  11. Put retrieval behind a governed data API
  12. Version and evaluate the retrieval pipeline
  13. Roll out the dictionary in sized iterations
  14. Failure modes
  15. Acceptance checks

All pages in Build the platform →