Browse the reference ⌄
Artefact published

The Audit Spine and the Evidence Record

How to capture, join, protect, retain and prove per-action evidence: the AES v0.1.0 record contract, three capture points and one correlation ID, tiered immutable storage, retention and legal hold, replay rehearsal, the route-approval evidence gate, and the Claude-specific ways evidence goes missing.

Version
0.2.0
Owner
DEIXEI
Last reviewed
Reading time
53 min

For Cto · Ai Architect · Platform Engineer · Security Engineer · Risk And Assurance · Internal Audit · Data Protection

Sources and details
Source works
Book 1 · Book 2 · Book 3 · Supporting Tools · Book 09
Source chapters
1, 3, 6, 9, 10, 11, 14, 18
Source appendices
01:B, 04:E, 04:G
Tool specifications
audit_evidence_schema@0.1.0, cab_decision_record@0.1.0, deployer_manifest@0.1.0
Refresh trigger
vendor release

Full source list ↓ Source index →

An audit spine joins the evidence that delivery and operation emit, so that one material action can be reconstructed from records rather than from a new narrative assembled out of disconnected logs. This page is the operating reference underneath the nine-question schema: what each record must contain, where each answer is captured, how records are joined, protected, retained and deleted, how the firm proves the spine works, and where evidence is lost on the Claude estate in particular.

It is a design reference, not a compliance claim. An event store, however well built, does not by itself discharge audit, privacy, retention or regulatory duties. Legal floors, article-level obligations and dates live on the regulatory map; this page links there rather than restating them.

The audit spine: every emission point speaks one envelope, and every reader queries the same surface.

At a glance: every emission point speaks one envelope, and every reader queries the same surface. The detailed diagram is under Four readers, one surface, named cadences. Source: The AI Delivery Factory, Chapter 9, Figure 9.2.

How to read the labels. Source means the books or the Supporting Tools specification state it. Proposal means a Supporting Tools design document that is marked draft or “proposed” and has not been built or verified. Recommendation is DEIXEI’s synthesis for this site. Composite marks the books’ illustrative institutions (Bank A, Insurer I, Asset Manager X): they are composites of several engagements, not real firms. Vendor statements are dated and carry [FACT-CHECK] where they describe current product behaviour that this page has not verified against vendor documentation.

The decision this page supports

The spine’s governing rule comes from Book 1 Appendix B and Book 3 Chapter 11. If the firm cannot answer the nine questions for an agent’s action, its observability is insufficient. The agent does not deploy, or the surface it runs on does not carry work at that risk tier. Every section below exists to make that rule testable:

DecisionOwner (source)Where on this page
Which record shape every emitter must producePlatform team, bound to the published specThe record contract
Which capture point answers which questionPlatform team; wrangler attests per agentCapture points and joins
How records are made tamper-evident and completePlatform and security engineeringIntegrity
How long each record class is kept, and how it is deletedGroup Risk sets policy; DPO decides erasure conflicts; General Counsel owns holdsStorage, retention and deletion
Who may query the evidence, under what basisSecond line; internal audit; legal for regulator scopeReaders and the query surface
Whether the spine actually worksSecond-line evidence assurance functionProving the spine works
Whether a vendor surface may carry a given tierRoute approval, with the evidence gateRoute-approval evidence gate and Claude surfaces

Two numberings of the nine questions

The books use two orderings of “the nine questions”, and they are not interchangeable. Mixing them silently is the fastest way to build a capture map with holes.

  • The record layer (Book 1 Appendix B and the AES spec). Nine field groups that every record carries: initiator, executing agent, envelope snapshot, action, data, model, approval, time, outcome. This is what the JSON Schema validates.
  • The capture layer (Book 3 Chapter 11 and Appendix E). Nine operational questions a reconstruction must answer, phrased around where the evidence is captured, including two procedural questions about storage and replay.

This page uses R1–R9 for the record groups and C1–C9 for the capture questions. The crosswalk below is a DEIXEI synthesis; neither book prints it.

Capture question (Book 3)Answered mainly by record group(s) (AES)Notes
C1 Which agent performed the action? (Appendix E: who or what initiated it)R1 initiator, R2 executing_agentBook 3’s Chapter 11 and Appendix E phrase C1 differently; Appendix E’s wording matches R1
C2 Under which identity, with which model, prompt and tool chain?R2, R3 envelope_snapshot.means, R6 modelThe tool chain is captured at the agent end, not at the gateway
C3 Which data sources were accessed?R5 dataIncludes memory writes and retrieval sources
C4 What decision, recommendation or change was produced?R4 action, R9 outcomeBook 1 requires verb, target and parameters, not a description
C5 Was a human approval required?R7 approval.required and policy referencerequired=false still cites a policy
C6 Who approved or rejected?R7 approval.approvers[]Empty only when approval was not required
C7 Which policy checks passed or failed?R7 policy_decision_id, policy_version, rationalePer-tool decisions at the agent end add to the gateway verdict
C8 What evidence was stored, and where?R9 outcome.evidence_pointers[], plus storage-tier metadataAES v0.1.0 has no record-level retention-class field (only optional free-text data[].retention and envelope_snapshot.memory.retention); see below
C9 How can the action be reconstructed later?Procedure, not a fieldProven by the replay rehearsal

R8 (time) has no single capture question of its own in Book 3. It underpins every reconstruction, because ordering across capture points is only as good as the clocks.

The record contract: AES v0.1.0

The normative contract is the Audit Evidence Schema v0.1.0, JSON Schema draft 2020-12, with worked examples at /aes/v0.1.0/examples/credit_scoring_action.predicate.json and /aes/v0.1.0/examples/credit_scoring_action.intoto.json. The related CAB Decision Record and Deployer Manifest ride inside it as payloads. Wherever this page, or any book’s prose, disagrees with the JSON, the JSON wins: every book says so explicitly.

Wire layering

The AES defines a predicate only. On the wire (Supporting Tools decision D13.1):

  1. A Sigstore signature wraps
  2. an in-toto v1 Statement (_type: https://in-toto.io/Statement/v1, subject[], predicateType: https://www.deixei.com/aes/v0.1.0), whose
  3. predicate is the AES record.

The spec owns the predicate; it deliberately does not redefine the in-toto envelope or the signature. A conformance check therefore has three parts: the signature verifies, the Statement is well formed with the right predicateType, and the predicate validates against the schema.

Top-level fields

The predicate has additionalProperties: false. Anything not listed here is rejected.

FieldRequiredType and rule
schema_versionYesString matching ^\d+\.\d+\.\d+$
record_idYesNon-empty string; ULID or UUID recommended (the resilience playbook specifies UUIDv7, monotonic)
session_idYesNon-empty string grouping one bounded work unit
kindYesDotted discriminator matching ^[a-z][a-z0-9_]*\.[a-z][a-z0-9_]*$, so exactly one dot (for example agent.action, cab.evaluate, deployer.apply, chk.advise)
initiator … outcomeYesThe nine groups R1–R9
change_idNoString or null; the second join key
payloadNoObject, untyped at AES level; carries the kind-specific extension
parameters_schema_refNoURI of a kind-specific JSON Schema that constrains action.parameters, outcome.output and payload

Field groups, required members and enums

GroupRequired membersEnumerations and constraints
R1 initiatortype, then by branch: human needs user_upn, channel, session_start; agent needs upstream_agent_id; scheduled needs schedule_id, trigger_eventDiscriminated union (oneOf) on type: human, agent, scheduled. Optional: session_ref, on_behalf_of.is_delegation, on_behalf_of.access_package, chain[], scheduler_record_id
R2 executing_agentagent_id, agent_version, deployment_environment, zonezone ∈ development, diagnostic, remediation, privileged-incident. Optional: agent_id_claim, system_prompt_version, system_prompt_sha256 (64 hex), control_plane_registry_record
R3 envelope_snapshotregistry_version, mandate_digest, means.tools[] (each name, version), memory.read_scopes[], memory.write_scopes[], monitoring.workspace, monitoring.rule_set, monitoring.evaluation_cadenceSnapshot of the envelope in force at action time, not at registration
R4 actionverb, target.entity, target.id, parametersOptional target.label, target.context_ref, downstream_effects[]
R5 data[]Each entry: operation, dataset_id, classificationoperation ∈ read, write, read_write; classification ∈ public, internal, confidential, restricted, regulated. Optional lineage, retention
R6 model[]Each entry: model_id, roleOptional model_sha256, registry_record, endpoint, region, deployment_id
R7 approvalrequired, policy_decision_id, policy_version, rationaleConditional rule below
R8 timeinitiation, completion (date-time), duration_ms (≥ 0)Optional end_to_end_latency_ms, clock_source, clock_last_sync, clock_drift_ms
R9 outcomeresult, output, evidence_pointers[] (each type, ref)result ∈ SUCCESS, FAILURE, PARTIAL. Optional side_effects[]

The conditional approval block

The approval group is where most hand-built emitters fail validation.

  • Always required, whatever the answer: required, policy_decision_id, policy_version, rationale. A record that says no approval was needed must still name the policy decision that made it so.
  • If required is true: workflow_id and approvers[] become mandatory. Each approver needs upn, role and timestamp; ticket_ref is optional. The v0.1.0 schema sets no minimum length on approvers[], so an empty array still validates. Recommendation: reject it in the firm’s emitter profile, because an approval with no named approver answers nothing.
  • Negative cases that must be rejected (the spec’s own verification ran these): a record missing any required top-level field; a record with required: true and no workflow_id or approvers.

Acceptance check: keep both negative cases in the producer’s CI suite alongside the two positive examples, so a schema upgrade or an emitter refactor cannot quietly loosen the rule.

Field-level answer rules

Book 1 Appendix B is specific about what counts as an answer. These rules are not all machine-enforceable, so they belong in the wrangler’s weekly review and in the second line’s sample.

GroupNot an answerAn answer
R1 initiatorA blank initiator (Book 1 calls it the most common reason a trail fails internal audit); “autonomous” without naming the upstream agent or the scheduleHuman UPN, channel, session reference and start; or the upstream agent and chain; or the schedule, trigger event and scheduler record
R2 executing agentA shared service-account name that several agents useA per-agent identity resolvable to a registry record, plus version and environment; the version matters because a prompt change makes a different actor for audit purposes
R3 envelopeA link to the current envelopeThe registry version and digests in force at the moment of action
R4 action”Made a decision”; “approved the application”Verb, target and the parameters actually applied: the values written, the thresholds and policy version used, the record identifier
R5 data / R6 modelOmitting the arrayEmpty arrays (data: [], model: []) are valid and mean “touched none”: an explicit answer, not a missing one
R7 approvalSilence, or an empty approval objectThe approver chain, or the policy clause that made approval unnecessary
R8 timeLocal time without offset; second-level precisionUTC, sub-second, with clock source and drift where the runtime can attest them (the schema keeps the clock fields optional)
R9 outcome”Done”Result code, output values, side effects in other systems, and pointers that resolve to the business event and the policy trace

Two related practices from Book 1 Chapter 6 (Bank A, composite) are worth adopting as recommendations: tag every inference trace with the AI-system identifier, because supervisors read logging obligations at that granularity, and carry digests of the retrieval payload and of the explanation output on each trace, alongside the model version and the user-side decision context.

Bindings to obligations and standards

Book 1 places regulatory bindings (DORA, AI Act and OWASP Agentic Top 10 codes such as ASI03 and ASI06) on each evidence_records entry, and the Policy Decision API response carries owasp_mappings[] and regulatory_bindings[] (for example DORA_art9, AI_Act_art12). AES v0.1.0 has no top-level binding field, and additionalProperties: false forbids adding one.

Recommendation: keep bindings queryable without breaking the envelope. Either resolve them through approval.policy_decision_id to the stored policy decision, or carry them in payload under a kind-specific schema named by parameters_schema_ref. Do not invent a top-level field. Which bindings apply to which obligation is a legal mapping held on the regulatory map and the control map.

Freeze the envelope; extend by payload

Book 2 Chapter 9 states the design position directly: the nine groups are frozen, and domain detail travels in the typed payload and in evidence_pointers. Every review eventually proposes a tenth question (lineage is the usual candidate; Book 09 shows a “did this change a control?” variant). Both books refuse it for the same reason. Every reader, query and collector relies on the same nine objects, and a record that grows a question per stakeholder becomes a form nobody completes correctly. Lineage belongs in pointers to data contracts and dataset versions. Whether a value is a control belongs in a firm-maintained list that the record points at, not in a field the change author fills in.

Kinds and payloads

The kind discriminator selects the payload. Book 2 names the main families: agent actions, cab.evaluate (payload: the CAB decision), deploy.* (the Deployer Manifest sits in the deploy.plan payload; each of plan, preflight, apply, verify and rollback emits its own record), and a gateway request kind for model calls. The Supporting Tools add operational kinds such as session.open, session.close, query.run, retention.delete, legalhold.applied, identity.key_rotated and chaos.exercise, and the ops log journal (log.intent, log.decision, log.observation and log.risk, the last with payload.severity; entries are signed and appended, never edited) for reasoning that is not a single command.

Source conflict on kind names. Several kinds used in the books and Supporting Tools documents do not match the v0.1.0 pattern, which allows exactly one dot: Book 2’s bare action, and multi-segment names such as deploy.apply.start, forwarder.batch.sent, retention.cycle.start and legalhold.check.batch. The schema’s own examples also say opslog.decision where the ops log use cases say log.decision. This page follows the schema. Recommendation: use two-segment kinds (carry a sub-phase in payload) until a spec revision changes the pattern, and validate kinds in CI.

Versioning and pinning

  • Books pin a floor and an implicit ceiling: conforming implementations of any version >= 0.1.0, < 1.0.0 satisfy the books.
  • The whole 0.x series is one compatibility band. Within it: new optional fields and new kind values may be added and semantics may be tightened compatibly; existing names, required lists and types do not change.
  • 1.0.0 locks the structural contract. After it, breaking changes need a major bump with a documented deprecation window, and producers and consumers should declare the highest major they understand.
  • Re-issue rule: patch or minor within the floor’s major needs no book change; a new major triggers a reprint or erratum.

Acceptance check: every producer declares schema_version; the edge rejects records outside the pinned band; the pin is recorded in the schemas and tools inventory.

Capture points and joins

Three capture points, and why no single one suffices

Book 3 assigns each capture question to one of three capture points, with the identity layer as the origin of C1. The table condenses Chapter 11 PR-1 and Appendix E §2. Field and event names are the books’ (as of August 2026); the Claude-specific ones are explained below.

QuestionGateway (network edge)Hook or SDK callback (agent end)Approval queue (human oversight)
C1 agentprincipal, principal_typeSession identity at UserPromptSubmit; OTel enduser.id—
C2 identity, model, prompt, toolsprincipal, model, prompt payloadPreToolUse/PostToolUse chain; SDK trace; session transcript—
C3 dataInbound dlp_classificationMCP-matcher hook; mcp_server_connection; memory audit log—
C4 outputOutbound response payloadSide-effect records; tool_resultEdited or approved draft
C5 approval requiredpolicy_decision = allow-with-approval—Routing entry exists for the correlation ID
C6 approver——Approver UPN, timestamp, justification, edits
C7 policy checkspolicy_decision and policy versionPer-tool decisions; tool_decision; permission_mode_changedApproval-policy reference
C8 stored whereretention_tierHook payload retention tag; session-store targetApproval-queue retention tag
C9 reconstructionQuery on the correlation IDSame join keySame join key

The reason for three points is structural. The gateway sees the model and the wire, but not the agent’s loop: a five-tool agent making one outbound response is one gateway record and five agent-end records. The agent end sees tools but cannot vouch for routing, DLP or the policy verdict. Neither sees the human judgement. Book 1 Appendix B adds that the persisted evidence record is built on top of operational action traces; traces are not themselves evidence until structured into the schema.

The evidence pipeline: an agent action becomes a structured, queryable evidence record governed by the Audit Evidence Schema.

Operational traces become evidence only once structured into the schema. Source: Building Organizations for the Age of AI (Book 1).

The correlation-ID contract

The correlation ID is what turns three capture streams into one story. Book 3 Chapter 11 PR-2 makes its propagation a platform contract, owned by the platform team, not a duty of each agent team:

  1. The gateway issues a v4 UUID at ingress for any request that arrives without an X-Correlation-ID header, and sets it on the inbound request and the outbound response.
  2. The gateway’s prompt-shaping band (never the agent teams) adds a short system-prompt instruction asking the model to echo the ID in structured output.
  3. The gateway evidence record writes correlation_id to the hot tier and, in parallel, to the immutable store.
  4. Workstation hooks read the ID from session metadata and include it on every prompt and tool event; the SDK includes it on every callback; it is set as an OpenTelemetry resource attribute so tool-decision and tool-result events join on the same key; the MCP connector broker writes it on every server invocation.
  5. The approval queue writes it on both the routing entry and the decision entry.
  6. The agent’s final response carries it where the format can hold it.
  7. Replay is one query filtered on that ID, ordered by time.
  8. IDs are immutable and records append-only: never edited, deduplicated or repurposed.
  9. A request that reaches any internal layer without the header is logged as a propagation gap and routed to the platform team’s gap queue.

How the correlation ID relates to session_id and change_id. Book 3’s capture records (gateway, hook, SDK, approval) are Claude-programme record types with their own field lists; they are not AES predicates. Book 2 and the AES use session_id (one bounded work unit) and change_id (everything about one change) as the two join keys. The books do not say how the request-edge correlation ID maps onto them. Recommendation: treat the correlation ID as the request-level key inside a session; set session_id to the work unit that contains it (for a single-request interaction they may coincide); carry the correlation ID in payload and as an evidence_pointers entry so the AES record and the capture records join without a translation table; and map both keys to the firm’s incident, case, ITSM and pipeline identifiers where those systems stay authoritative.

Approvals ride the same pipeline

The anti-pattern Book 3 names first is the approval queue held in its own collection, with its own export, retention and search interface, so that at replay time an engineer joins the human-judgement line to the execution line by hand. The rule: approval records emit through the same ingest fabric, into the same hot tier and the same immutable store, with the same correlation ID. When the ID is missing, the oversight evidence is detached and the replay is incomplete.

Appendix E §3.4 gives the minimum approval-queue record:

FieldMeaning
correlation_idMatches the originating gateway call
approval_event_idUnique to this approval event
envelope_id, agent_idThe envelope and agent whose action is queued
principal_requestingThe agent or workload that queued the request
principal_approving, principal_approving_roleUPN and role (wrangler, deputy, escalation tier) of the named human
timestamp_request, timestamp_decisionQueue and decision times
decisionapproved, rejected, escalated, expired
justification_textRequired for approved and escalated
mandate_field_extensionThe diff, if the approver widened the envelope’s Mandate
review_minutes_referenceAgent Control Board or Virtual CAB minutes where they exist

Edits the approver makes are retained as a diff linked by approval_event_id. Note the vocabulary difference: Book 1’s approval_records collection uses approve, refuse, escalate and defer, with rationale mandatory for refusals and escalations. Pick one enumeration in the firm’s own schema and map the other at the edge; do not store both.

One identity everywhere: executing_agent.agent_id_claim

Book 2 Chapter 11 binds agent identity to a specific field. executing_agent.agent_id_claim holds the claim the platform’s identity provider issued at execution time, and the same claim is carried at every emission point, so the registry, the gateway and the evidence lake name one identity rather than three that happen to share a name. Rules:

  • Every non-human identity resolves to a human owner; for agents that owner is the wrangler (accountability).
  • The claim always names the agent. When the agent acts for a person, the person is recorded beside it (in initiator and on_behalf_of), never in its place.
  • A claim that names a role or a service principal shared by several agents is a finding. Recovery is one claimable identity per agent before the estate grows (identity and secrets).
  • Provider sources of the claim, per Book 2 Chapters 9 and 11 (as of August 2026): Entra Agent ID on Azure; Amazon Bedrock AgentCore Identity session-tagged identifiers on AWS; the Gemini Enterprise Agent Platform’s agent identity on GCP. [FACT-CHECK: current names and claim formats of provider agent-identity services]

Note that agent_id_claim is optional in the v0.1.0 schema. Recommendation: make it mandatory in the firm’s emitter profile for every production agent.

The gateway record and policy-decision vocabulary

Appendix E §3.1 extends the gateway record of Book 3 Chapter 6 to nineteen fields in three groups. The page reproduces the names because emitters and queries bind to them:

  • Correlation and principal: correlation_id, timestamp (ISO 8601 with offset, millisecond precision), principal, principal_type (user, workload, agent), use_case_class (class0–class3, from a JWT-validated header), envelope_id.
  • Route and model: deployment_route, model_id, model_version_pin, tokens_in, tokens_out (both counted after tokenisation).
  • Policy, DLP and outcome: dlp_classification, dlp_action (pass, tokenised, blocked), tokenisation_event_count, policy_decision, response_status, latency_ms, mcp_servers_invoked, retention_tier.

retention_tier is load-bearing: the writer to the immutable store reads it and routes the record to the matching locked container (below).

Two policy-decision enumerations. Book 3’s gateway uses allow, allow-with-evidence, allow-with-approval, deny. Book 1’s Policy Decision API uses allow, allow_with_evidence, escalate_for_approval, deny, requires every deny to carry a rationale and a path-to-allow, and returns a short-lived expiry to prevent replay of old authorisations. The semantics line up one to one; the spellings do not. Fix one vocabulary in the model gateway contract and map at the boundary.

Agent-end record types and payload truncation

Appendix E specifies two agent-end records (as of May 2026):

  • Hook record (workstation): correlation_id, timestamp, agent_id (the engineer’s session identity), hook_event (UserPromptSubmit, PreToolUse, PostToolUse, ConfigChange), tool_name, tool_input_truncated_4kb, tool_result_truncated_4kb, tool_result_full_16kb_ref, permission_decision (allow/deny, pre-tool only), blocked_reason, mcp_server_ref, error_code.
  • SDK record (embedded agent): correlation_id, timestamp, agent_id, envelope_id, subagent_id, event_type (start, tool_call, message, completion, suspension, error), tool_chain_position, tool_name, extended_thinking_tokens_used, cost_usd_running_total, budget_remaining_usd, status_at_event (running, suspended, failed, completed).

Truncation varies by tier, and the ingest fabric fans out: the SIEM takes the truncated stream, the immutable store takes the full one.

TierHot-path truncationFull payload to immutable store
Tier 0–14 KBNot retained
Tier 216 KBOn tag
Tier 3Hot path onlyAlways

The failure this prevents is concrete: a default 4 KB PostToolUse truncation is sensible for cost and wrong for a Tier 3 response of 12 KB. Tier definitions are on risk tiers.

Integrity: authenticity, immutability, completeness

Supporting Tools decision D6 defines three properties. Specify and test each separately; an append-only store is not automatically complete, and a signed record is not automatically retained.

PropertyMechanism (source: D6, D7, resilience playbook)Detection
AuthenticityEvery event leaves the leaf signed with a per-agent or per-workload key; the edge verifies before accepting. Signing uses a short-lived claim (default 60 minutes) that references a long-lived verification key; verification is checked against the record’s issued-at time, not the edge’s receive timeAn unsigned event is itself a logged anomaly; a signed event that fails schema validation is rejected with a structured rejection record (Book 2 Ch 9). Invalid signatures go to a dead-letter queue and the source is quarantined
ImmutabilityThe lake is counter-signed and append-only, with a hash chain per session. Deletions for retention or subject-access requests are signed events that supersede rather than erase. Key rotation is itself a signed event (identity.key_rotated) with a default 24-hour overlapChain verification references record_ids, so it still works after content has been crypto-shredded
CompletenessSessions open with session.open and must close with session.close, even when the close is abandoned. closure_kind is clean, abandoned or timeout; an open session past the heartbeat threshold is auto-closed and flaggedAlert on absence: a forwarder that stops sending heartbeats, a verify stage that falls silent while others still emit. The most dangerous signal is the expected event that never arrives

Collusion. D7’s defence is the signed forwarder plus the heartbeat. A compromised or colluding leaf can stop sending, but stopping is a detectable absence; it cannot alter what it already sent. Where the agent runs on a vendor-hosted leaf, the firm’s gateway or proxy emits on its behalf, which makes the gateway a control plane in its own right.

(The Supporting Tools documents disagree slightly on the timeout path: the session state diagram closes a timed-out session as abandoned, the data model and ops run use case as timeout. Pick one and document it.)

Edge validation

The validation rule is enforced at the per-jurisdiction edge collector, not at the lake. Book 2 Chapter 9 is explicit that the first line cannot quietly emit malformed evidence. Acceptance check: send a known-unsigned record and a known-invalid record through a test leaf each quarter, and confirm one anomaly event and one rejection record arrive, with nothing written to the lake.

The leaf outbox and forwarder

The Supporting Tools resilience playbook formalises the leaf as a transactional outbox: the signed record is appended and flushed to local JSONL before the call returns, and a forwarder ships asynchronously. Success means written to the outbox, not acknowledged by the edge.

Rules (playbook §1; these are proposals for the reference implementation, normative for it and outcome-only for vendors):

  1. Outbox write is synchronous; edge acknowledgement is asynchronous.
  2. The outbox is the source of truth until the edge counter-signs. The forwarder must never delete an entry that has not been counter-signed. A full leaf disk is a paged incident, not silent loss.
  3. fsync per record for high-stakes kinds (CAB evaluations, deployment apply start and end); batched for routine actions.
  4. Idempotent receipt: the edge de-duplicates on record_id within its 24-hour replay buffer, so forwarder retries are safe. Out-of-order sequence numbers within a session_id are an anomaly.
  5. Dual stamping: each record carries emitted_at (leaf) and received_at (edge); skew is a derived field, and sessions above the threshold (default 30 s) are flagged, usually for NTP drift, occasionally for compromise. These are lake-row columns, not AES predicate fields.
  6. Throttling fails closed: a leaf over its rate ceiling accumulates on the outbox and retries; events are never dropped silently.
Default (v0.1)Value
Batched fsync interval (routine kinds)100 ms
Retry backoffExponential from 1 s, capped at 60 s, full jitter
Retry maximum duration24 h
Page on outbox backlog> 10 MB or > 4 h
Forwarder batch100 records or 1 MB; every 1 s
Per-leaf event ceiling1,000 events/s; edge returns 429 with retry-after
Recovery point objectiveLeaf 0 (flushed); edge 24 h buffer; centre lake ≤ 60 s

Performance budget. The pilot exit trigger is a host-workload impact above 5% (D16.6); the reference CLI benchmarks leaf overhead in CI and treats a per-call regression above 0.5% as a release blocker. A spine that slows production gets switched off, so measure it.

Append-only, including for corrections

Records are never edited or deduplicated after the fact. A session rewind adds records; it does not amend earlier ones. A correction is a new entry beside the original that names what was wrong and when it was found; Book 09 (Asset Manager X, composite) shows exactly this, with the original left untouched. A CAB decision is replaced by a later decision marked superseded, never altered (change governance).

Storage, retention and deletion

Tiered immutable storage

Book 3 uses three storage tiers per record class, fed by a single ingest fabric:

TierPurposeAzure reference (Book 3, as of mid-2026)
HotDetection rules, alerts and operational queries on the live streamMicrosoft Sentinel; configurable 30–730 days
Immutable long-termWrite-once, retention-locked store; the long-term source of truthAzure Storage Blob with a locked time-based retention policy, one container per retention class
ArchiveRecords past the active window where the firm elects to retainCool or archive tier

Two rules carry the design:

  • Lock at write, per class. The writer reads retention_tier and routes each record to the container for that class; each container’s policy is locked once at creation and can then be extended but never shortened. The corollary is harsh: a record mis-classified at write costs a full retention window that cannot be recovered. Test the routing before go-live with one synthetic record per class.
  • The lake feeds the SIEM; the SIEM is not the system of record (Supporting Tools D4). The SIEM keeps the alerting muscle; the lake keeps the evidentiary discipline. The Supporting Tools data model treats the lake as one event-sourced log of AES records (entities are read projections of it, in the CQRS sense), stored as Parquet partitioned by entity of record, year, month, day and kind, with session_id and change_id exposed as first-class query filters. In Book 3’s Azure stack the immutable Blob is canonical and Sentinel is the hot tier, which is the same split.

Vendor-side stores are extraction points, not retention tiers: nothing counts as retained until it is in the firm’s own immutable store.

Retention schedule: the sources disagree

Every source agrees on the structure: retention is bound to the use case’s risk tier, set at write, and locked. They disagree on the numbers. Book 3 alone gives three different schedules.

TierBook 3 Ch 11 PR-3Book 3 App E §5Book 3 Ch 14 and App G
Tier 0Hot 90 d; immutable 1 y (class1-12m)Hot 90 d; immutable 1 y (class0-12m)13-month gateway log (class0-12m)
Tier 1Hot 365 d; immutable 1 y (class1-12m)Hot 180 d; immutable 2 y (class1-2y)Ch 14: 13 months; App G: class1-2y
Tier 2Hot 365 d; immutable 3 y (class2-3y)Hot 365 d; immutable 3 y3 years (class2-3y)
Tier 3Hot 365 d; immutable 6 y (class3-6y)Hot 730 d; immutable 7 y (class3-7y)6-year minimum on class3-7y, plus sectoral overrides (App G cites a 10-year example)
Tier 4Out of scopeOut of scope; bespokeSeparate approval

Appendix E is not even consistent internally: its gateway retention_tier enum lists class1-12m while its own retention table assigns Tier 1 to class1-2y. The Supporting Tools retention engine uses a third model, keyed by record kind rather than by tier (example classes regulatory_evidence_7y, operational_logs_18m, ephemeral_30d). Book 1 Appendix B gives each control-plane collection a five-year floor, which it attributes to DORA’s logging provisions, and a similar floor for high-risk system logs, which it attributes to the AI Act’s record-keeping duty. [FACT-CHECK: the five-year floor Book 1 Appendix B attributes to DORA Art 9. That article is not obviously a retention-period provision, so verify it against the legal text and Delegated Regulation (EU) 2024/1774 before adopting it.] The books also cite different DORA articles for the same retention obligation (Book 1 App B, Book 3 Ch 11, and Book 3 App E and App G each name a different one). The legal analysis belongs on the regulatory map, not here.

What this page follows:

  • Recommendation: treat the periods as an institutional legal decision, made once by Group Risk with legal and the DPO, recorded in a signed retention policy pack, and never inferred from this table. Route the legal basis through the regulatory map.
  • Recommendation: define one retention-class enumeration in that pack and derive both the gateway retention_tier values and the set of locked containers from it, so the enum and the containers cannot drift as they have in the source.
  • Carry the tier on the record and size Tier 3 for the longest applicable period, including sectoral overrides, because a locked container cannot be shortened but an under-sized one cannot be recovered.

Retention engine: signed, superseding deletion

The Supporting Tools retention engine (a draft API, milestone M47) is a useful model for how deletion must work under immutability.

  • Effective retention is the maximum of the kind’s default class period, any explicit per-record extension, and any active legal hold covering the record. It is computed at cycle time, so a policy change applies from the next cycle.
  • Deletion is a signed event, never silent erasure. A retention.delete record supersedes its targets and carries retention_class, target_record_ids, policy_version, cycle_id, effective_at and encryption_key_discarded_at.
  • Crypto-shredding gives tombstone semantics. Target content is encrypted with a key the engine then discards. The ciphertext and the superseding record stay, so existence and the per-session hash chain remain verifiable; the content is gone.
  • Roles: retention.scheduler.system, retention.policy.maintainer (Group Risk), retention.reader.audit, retention.exception.requester (rare, logged). A per-record extension emits retention.extension and is distinct from a legal hold.
  • Events: retention.cycle.start/end, retention.delete, retention.extension, retention.held_back, retention.policy.applied.
FailureResponse (engine spec)
Hold engine unreachableSkip the cycle for affected records, alert, retry next cycle
Key discard failsRefuse to emit the delete record; P1 alert (a half-deleted record is unrecoverable)
Policy pack expiredRefuse the cycle until the pack is republished

SLOs: every class processed every cycle; 100% legal-hold interlock correctness; one signed delete record per deletion. (The spec says the scheduler runs nightly but sets its completion SLO per weekly cycle; the cadence is a configuration decision.) Open items the spec lists as owed: per-jurisdiction overrides, per-agent overrides and backup cycling: a backup that outlives the retention class silently defeats the deletion, so retention coverage must include backups.

Book 3 Appendix E: a record under legal hold (litigation, regulatory inquiry, supervisor request) carries a hold tag; the hold persists until explicitly released and the underlying retention resumes afterwards. General Counsel owns the hold register; the platform team applies and releases tags only on signed instruction.

The Supporting Tools legal-hold engine (draft API, milestone M48) makes this enforceable:

  • Holds are themselves evidence: legalhold.applied, legalhold.released, legalhold.expanded, and a legalhold.check.batch event per retention-cycle interlock.
  • Only legal-team UPNs may apply or release; Conditional Access requires MFA and a compliant device for both.
  • Scope kinds: session, change, agent (with a date range) and filter (an AES query, with preview-then-confirm). Cascade is on by default for session and change scopes, off for agent and filter scopes.
  • A record under several holds becomes eligible for retention only when all are released.
  • The interlock is the load-bearing property: retention batches its candidates through the hold check and excludes held records, and an attempt to delete a held record is a hard failure with an audit trail.
  • SLOs: apply and release within one business hour (p99); daily reminder from 30 days before a hold’s expected end.
  • Proposals still owed: jurisdiction-scoped holds, and sealed holds whose existence is itself confidential (the AES record exists, its payload is redacted to all but the legal team, and the SOC sees a count only).

Retention versus erasure requests

The tension between evidence retention and data-subject erasure is a legal decision, and the sources resolve it differently:

  • Book 3 (Chapter 11 PR-3 footnote, Appendix E §5): retention obligations on processing records take precedence under the legal-obligation exemption to erasure; the standard pattern is to keep the audit record and honour the erasure against the operational store. The DPO is the single point of decision, signs per record class rather than per request, and records the position in the records of processing and the DORA Register of Information. The tension is never resolved at the engineering layer.
  • Supporting Tools retention engine (failure-mode table): on collision, “apply the shorter retention” and emit both records.

These conflict. This page follows Book 3’s governance model (the DPO decides per record class, in advance) because it keeps the decision out of an automated failure path. Recommendation: implement the engine’s collision handler to raise the case to the DPO’s documented class position rather than defaulting to the shorter period. The legal ground itself is for the regulatory map and the DPO, not this page. Where personal data has leaked into agent memory, memory redaction (see the Claude section) is what makes an erasure request tractable.

Cost never trims evidence

Book 3 Chapter 18 names the recurring proposal: cut hook payload retention or drop Tier 3 full payloads to save ingestion cost. The saving is small and the evidence risk large. Any change to the evidence pipeline whose main justification is SIEM cost is routed to the second-line evidence assurance function; retention is set by tier, not by the cost dashboard. Where cost is a real problem, change what goes to the hot tier, not what reaches the immutable store.

Readers and the query surface

Four readers, one surface, named cadences

Book 2 Chapter 9 names four readers of the same records, each with a cadence: the wrangler weekly, to attest the agent stayed inside its envelope; internal audit quarterly, to test controls by replay; the SOC in real time, for alerts and containment; the supervisor on demand. Book 1 adds model risk and internal-control attestation as further readers of the same schema.

Intake, build, test, change, deployment, observation and retirement emit correlated records to one evidence surface, read by operational, assurance and supervisory roles.

One evidence surface, several readers. Storage, access and retention choices remain the institution’s responsibility. Source: The AI Delivery Factory, Chapter 9, Figure 9.2.

ReaderRecurring questionExample acceptance evidence
WranglerDid my agent stay inside its envelope?Decisions, denials, evaluation deltas and lifecycle events for the agent and version
SOCWhat must be contained now?Correlated action, identity, target, policy decision, timing and downstream effects
Change authorityWhat was authorised, and what actually deployed?CAB decision or exemption, deploy phases, health checks and rollback result for one change_id
Internal auditCan the control be replayed from evidence?A traceable sample from requirement to outcome, including failed and refused paths
SupervisorCan the firm tell us what happened?One reproducible query result plus the records in native form on request

One schema, projected per audience. Book 3 Chapter 3 §5 (Insurer I, composite) sends the same nine-question records to two workspaces the firm owns. A Sentinel workspace under the CISO holds the cyber and operational-resilience evidence. An Azure Monitor workspace under the Data Protection Office holds the records-of-processing evidence and the evidence of meaningful human review. Each audience gets its own field projection, and the incident workflow and the insurer’s own risk reporting each draw on the workspace that suits them. Recommendation: implement projections as redaction profiles over one record stream (the query API’s redaction_profile is the hook). Do not fork the schema, and do not keep a second copy that can drift.

Composite example (illustrative, Book 2 Chapter 9, Bank A). A supervisor asks for every change to the cross-border payment-screening agents over six weeks, with approver, test evidence consulted and deployment route. Because every record carries a workload-class tag and the same envelope, one query over one table returns 43 records (3 CAB decisions, 8 deployer manifests, 32 signed agent actions). The cost that disappears is translation between systems, not engineering.

The Supporting Tools specify an evidence-lake query API (M5; decision D25, status Proposed). Its rules are worth adopting regardless of implementation:

  • Seven roles: wrangler (own team’s records), auditor (whole bank, redacted), soc (real-time, PII redacted unless escalated), supervisor and regulator (entity-of-record scope under documented legal basis), second_line (cross-team within the entity), internal_audit (read-only across the entity, including reasoning[] arrays).
  • scope_assertion is mandatory for supervisor and regulator, naming the legal_basis_ref under which the scope is exercised; other roles default to their own scope claim.
  • Scope is enforced in the query planner, before the lake is touched. An over-scope query is refused with m5.scope_violation and recorded as auth.refused.
  • Queries are themselves audited: every query emits query.run and every stream ends with a query.summary (record count, duration, query-run record ID). Cancellations and cost breaches are events too. The audit spine is recursive.
  • Output shapes: streaming JSONL for interactive use; background Parquet, CSV or gzipped-JSONL exports for bulk, delivered via a time-limited link (4 hours in the spec) and kept for 7 days.
  • Cost ceiling: queries are pre-planned; above the published ceiling (default 30-minute scan budget) they need explicit acknowledgement and an additional record.
  • Per-role concurrency and daily quotas, and SLOs of first byte ≤ 1 s and regulator-shape query ≤ 5 minutes (p99).

Signed, self-verifiable evidence pack

Every regulator-facing report in the Supporting Tools design produces a signed tarball: manifest.json, query.json (the exact query), records.parquet (signature-verified AES records), summary.html, chain_of_custody.intoto (signed by the centre) and verifier_instructions.md. The recipient can verify it offline from the published trust roots without trusting the firm’s word. Target SLAs include ≤ 5 minutes for a pre-canned regulator query and for a legal-hold query.

Answering a supervisor query

Book 09 Chapter 9 (Asset Manager X, composite) adds the human procedure around the query, which the API cannot supply:

  1. Two independent queries. The person whose control approved the changes does not run the only count. A second analyst writes a query from the question alone, without seeing the first; the counts are compared before anything is reported.
  2. A reproducible funnel. Read the question fully, start from the whole population, and filter in the open, keeping each step so the supervisor can repeat it. Then read the residue by hand with the record open.
  3. Declared gaps. A gap logged in advance (with the date it opened and closed) is a chosen departure; a gap found afterwards is a finding.
  4. Send the reading and the means to check it. Answer the question asked, include the funnel, and offer the native records on request rather than sending everything raw.
  5. Write what was counted, not what is believed.

Count your vetoes

Both Book 2 and Book 09 recommend one standing quarterly query none of the readers will think to ask for: every refusal on the delivery path. That covers no-go or conditioned CAB decisions, policy-gate denials, reviewer rejections with reasons, and rollbacks after failed verification. Sort them by what they protected. A declared control that shows no recorded cost across two quarters is either fully internalised or decorative; the reading decides which. See policy, audit, actuals.

Proving the spine works

Quarterly replay rehearsal

Replay (C9) is an assertion; the rehearsal is the proof. Book 3 Chapter 11 PR-4 and Appendix E §6 specify it.

StepWhat happensOwner
1. SelectionAt the start of each quarter, draw correlation IDs at random from the prior quarter’s gateway log, stratified by tierSecond-line evidence assurance (an IT audit team member or an SRE rotated into the assurance role)
2. AssignmentEach ID goes to an engineer not on the originating agent’s team; the assignment and time-box are recordedAssurance lead
3. ReconstructionRun the replay query; pull full payloads from the immutable store where the hot record is truncated; write a one-page chronological narrativeAssigned engineer
4. CrosswalkReconcile gateway tool summary against agent-end records; approval timestamp against allow-with-approval; immutable retention class against use_case_classAssigned engineer
5. GapsEvery missing record, broken correlation, mis-tagged retention, dropped mirror batch or defeating truncation becomes a finding with an owner and close date; the lead checks weeklyAssurance lead
6. Meta-evidenceThe rehearsal itself (IDs, engineers, times, findings) is stored as a structured record under a rehearsal- ID prefix (live incidents use inci-), at the Tier 2 retention classAssurance lead; reviewed by head of IT audit and SOC engineering lead

Stratification (Appendix E): Q1 Tier 0, 1, 3; Q2 Tier 0, 2, 3; Q3 Tier 1, 2, 3; Q4 Tier 0, 1, 2, 3. Tier 3 is exercised every quarter because that is where a supervisor is most likely to ask. Tier 0 is included because silent early purge of low-tier evidence is the commonest operational failure. (Chapter 11 says three IDs per quarter with at least one Tier 3; Appendix E’s fourth quarter draws four.)

Reconstruction targets (Appendix E, reviewed annually): Tier 0–2, 4 hours; Tier 3, 1 hour for the raw record set and 4 hours for the formatted packet; Tier 4 out of scope. The books derive the targets from the major-incident reporting clock; the clock itself is on the regulatory map. Composite experience reported in the books: first rehearsals land at 8–12 hours, within target by the fourth, routine by the eighth. Cadence is a policy decision (Asset Manager X, composite, runs monthly); the existence of the rehearsal is not.

Annual review: the four rehearsals are summarised for the AI Steering Committee and internal audit on three measures: median reconstruction time per tier, gap count and close rate, and the trend. A rising trend is a leading indicator of evidence-pipeline rot.

An example replay query from Appendix E (Azure, KQL; table names are the composite’s):

union AzureDiagnostics, EvidenceHub_CL, ApprovalQueue_CL, HookEvents_CL
| where correlation_id_s == "<correlation-id>"
| order by TimeGenerated asc

Complement the rehearsal with the Supporting Tools’ monthly chaos exercise on the spine itself (kill an edge for ten minutes, revoke a signing key, partition a leaf for an hour), each recorded as a chaos.exercise record.

Route-approval evidence gate

Book 3 Chapter 11 PR-5 turns the coverage question into a gate that runs before any Anthropic surface is approved for a risk tier. Recommendation: run the same gate for every other vendor surface. One page, one meeting:

  1. Name the surface and the highest tier requested.
  2. For each of C1–C9, name the record that answers it and the system that emits it. A blank is a gap, not a rounding error.
  3. For each named record, state its export path into the firm’s own store, the retention applied, and whether that store is immutable.
  4. Confirm the correlation ID reaches every named record.
  5. Confirm export-failure alerting exists (for Claude: CLAUDE_CODE_OTEL_DIAG_STDERR, collector liveness, mirror_error).
  6. If any of C1–C9 is unanswerable, cap the surface one tier below the level requested, and record the cap in the registry with a review date.

Where the answer to step 2 is “the vendor’s console”, the answer is not yet evidence. The gate’s output feeds agent lifecycle registration and the risk tier decision.

Twenty-change oversight sample

Book 09’s closing chapter gives a cheap test of whether human oversight of change actually happened, and Book 2 Chapter 1 states the principle behind it: a green build is evidence of execution, not understanding; an approval is evidence of workflow, not of design fitness. Decide explicitly what the firm accepts as evidence that a change was understood.

  • Who: someone outside the delivery teams.
  • When: monthly. The result goes to the risk committee, and the chief risk officer sees it before the dashboard does.
  • Sample: twenty changes drawn at random from the previous month, including some a machine approved, if any machine approves.
  • Five columns, filled from the record only:
ColumnAcceptable entry
Who approvedNamed person or named machine route
What they checkedA review comment, an answered checklist, or a test they added; “they reviewed it” is blank
Who made the testsA person, a different agent, or the same agent that made the change
Minutes from opening to approvalFrom timestamps
Did it alter a limit, threshold or toleranceAnything that decides what a client or investor sees
  • Two counts: rows where what they checked is blank (oversight that is a name on a button), and rows where tests came from the same source as the change (a change agreeing with itself). The books estimate about two hours for twenty rows.
  • Acceptance check: one query over the evidence record should answer the sample. If it needs screenshots, the record is not yet evidence of oversight.

The morning drill

The same chapter adds an unannounced drill. Write down the question a supervisor will one day send (every change to the most important control since a date four months back, who or what approved each, and on what basis), hand it without warning to the platform lead and the head of technology risk, and time them. If most of the time went on finding things and matching identifiers across systems, the gap is translation. Then take one machine-approved and one human-approved change from the sample and compare their entries side by side: if the automated one is thinner (no score, no inputs, no record of the rule in force), automation has bought speed by spending evidence. Last, count what is still entered by hand.

Claude and Anthropic: where evidence is captured and lost

This section is vendor-specific by design. Unless stated otherwise it reflects Book 3 Chapter 11 as of August 2026. Every item describes current product behaviour that this page has not verified against Anthropic documentation, and Book 3 itself says to re-run the coverage assessment every two quarters. See also securing Claude Code in the enterprise §10 for a worked monitoring pipeline.

Per-surface coverage

Coverage is per product, not per vendor. Book 3 calls this the most misread fact in the Anthropic estate.

SurfaceWhat it emitsCapture questions it can answerWhat it cannot answer
Claude Code / Agent SDK: OpenTelemetrytool_decision, tool_result, mcp_server_connection, permission_mode_changed; metrics; beta tracesC1, C3, C4, C7 at the tool layerC5, C6; C8 and C9 unless the collector writes to the immutable store
Claude Code / Agent SDK: JSONL transcriptFull session history (pre-compaction only if exported at PreCompact)C2, C3, C4 at full fidelityRetention, immutability and tamper-evidence are the firm’s to supply
Claude Enterprise and Claude Platform: Compliance APIConversation content (chats, files, projects); activity events (logins, admin actions, configuration changes)C1, C2, C4 for chat-surface use; admin-action provenanceC5, C6, C7; retention and residency unstated
Managed Agents: Console session events, memory audit logExecution and session traces; memory changes with creating agent and session, rollback, redactionC1, C3 for memory writes; partial C4C8, C9: no confirmed export API, retention or immutability
Connector directory dashboardActive users, tool-call counts, error rates, latency, health scoreNoneAll nine
Claude CoworkNo formal Compliance API or audit-log support as of July 2026NoneAll nine

[FACT-CHECK: each row against current Anthropic documentation; specifically whether the Cowork Compliance API and audit-log gap has closed since July 2026, and whether Managed Agents traces now have an export API and retention commitment.]

Readings that follow:

  • Aggregate telemetry is not evidence. A dashboard showing thousands of tool calls at a low error rate answers none of the nine questions, because each question is about one action. A platform team demonstrating a connector dashboard to the second line has not closed the evidence question.
  • Console-resident traces count as unavailable until an export path and a written retention commitment are verified.
  • A surface that cannot answer the nine does not carry Tier 3 work: that is a route-approval matter, decided at the gate above.
  • Memory is the exception worth exporting. Managed Agents memory stores are files with an audit log of the creating agent and session, version rollback and content redaction. Export that log into the firm’s retention regime and treat memory writes as data-source events (C3). Redaction is what makes erasure tractable when personal data reaches agent memory.

The Compliance API: an extraction pipe, not a store

Book 3 Chapters 3 and 10 (as of July–August 2026): the Compliance API gives programmatic access to Claude Enterprise conversation content and to activity events, and is consumed by many partner DLP, SIEM and eDiscovery integrations (Microsoft Purview and Sentinel among them). Book 3 records that, as of July 2026, it is Enterprise-plan only (Team-plan organisations do not have it), and that the announcement states no retention window, no deletion workflow and no EU residency option. The analytics dashboard and API are not available on the Bedrock, Claude Platform on AWS, Google and Microsoft Foundry routes; on every route, OpenTelemetry export into the firm’s own collector is the stream that survives. Settings-change audit events are reachable only through the account team, with no documented retention or export schema. [FACT-CHECK: Compliance API plan availability, retention, deletion, residency and settings-change coverage as of today]

Source conflict. Book 3 Chapter 11 says the Compliance API carries no stated retention window; Appendix E §2 and the Chapter 14 tier text describe a six-year Compliance API window. This page follows Chapter 11 (the later, corrected text) and treats the API purely as a pipe into the firm’s own immutable store.

Operating rule (Book 3 Chapter 3): a Claude product on the estate without evidence coverage is outside the evidence layer; either it gets covered, or it does not touch a critical or important function. Keep the per-product coverage matrix on the AI Steering Committee’s standing agenda and review it quarterly.

OpenTelemetry: per-action events, silent export failure

Claude Code and the Agent SDK carry built-in OpenTelemetry instrumentation (Book 3 Chapter 11 §5.2, as of August 2026). Wire it in parallel with the firm’s own hooks, not instead of them.

  • Enable with CLAUDE_CODE_ENABLE_TELEMETRY=1 and standard OTLP environment variables pointing at the firm’s collector.
  • Set an end-user identity in OTEL_RESOURCE_ATTRIBUTES (and the correlation ID as a resource attribute) so the four log events form a per-user trail that joins the rest of the spine.
  • Telemetry is structural by default (durations, tool and model names, no content). Content export is opt-in through four escalating variables: OTEL_LOG_USER_PROMPTS, OTEL_LOG_TOOL_DETAILS, OTEL_LOG_TOOL_CONTENT, OTEL_LOG_RAW_API_BODIES. Make the setting per tier a documented decision with the DPO.
  • Export fails silently by default. Set CLAUDE_CODE_OTEL_DIAG_STDERR=1 and alert on collector liveness; otherwise gaps are invisible.

[FACT-CHECK: environment-variable names, event names and default export-failure behaviour against current Claude Code documentation]

Transcripts, compaction and the session store

Claude Code writes a JSONL transcript of each session to local disk, reachable from hook payloads via transcript_path. It is the fullest native record of a session, and it is local, unshipped and unretained; the firm builds the forwarding pipeline. Three mechanisms turn it into an evidence-loss risk (Book 3 Chapter 11 §5.3 and §9, as of August 2026):

MechanismWhat is lostControl
CompactionNear the context limit the working record is replaced by a generated summary; anything reconstructed by interrogating the live session is a précisWire the PreCompact hook on every managed workstation as a non-optional part of the settings baseline, and ship the pre-summary transcript to the immutable store at that point
/rewindAn operator re-prompts from an earlier message, creating divergent histories in one sessionEvidence capture is append-only: a rewind adds records and never amends or removes them
SessionStore mirrorWhen the Agent SDK mirrors transcripts to firm-governed storage (reference adapters for S3, Redis and Postgres), local disk stays authoritative and a failed batch is retried three times then dropped, signalled only by a mirror_error system messageMake mirror_error an alerting rule with a named owner, not a log line; implement retention in the adapter (the product neither deletes evidence nor enforces deletion you forgot)

[FACT-CHECK: PreCompact hook behaviour, transcript_path, /rewind semantics and SessionStore retry-and-drop behaviour against current Claude Code and Agent SDK documentation]

Extended-thinking traces

Book 3 Chapter 9 (as of mid-2026): extended-thinking traces are part of the model response and subject to the same retention. For Tier 3 and above, anything customer-affecting or decision-bearing, the trace is the material a meaningful human review is tested against, so the evidence record must capture it. Book 3 Chapter 14 makes the thinking trace and citations chain part of the Tier 3 evidence package. The SDK callback owns the trace, under a retention policy bound to the class tier; decide with the DPO whether trace content is retained in full or as a digest. [FACT-CHECK: current extended-thinking modes and whether full or summarised traces are returned for current Claude models]

Cloud equivalents for the evidence stack

The schema and the rehearsal cadence do not change by cloud; the plumbing does (Book 3 Chapter 11 §7 and Appendix E §7; Book 2 Chapter 9 §4.5; as of mid-2026).

LayerAzureAWSGCP
Ingest fabricEvent Hub (Book 2 edge: APIM plus Container App with Service Bus)EventBridge (Book 2 edge: API Gateway to EventBridge/SQS to ECS or Lambda)Pub/Sub (to Cloud Run or Dataflow)
Hot / SIEMMicrosoft SentinelCloudWatch Logs; Security Lake or Splunk via OTLPGoogle Security Operations (Chronicle)
Immutable storeBlob with locked time-based retentionS3 Object Lock in Compliance mode (not deletable before expiry, even by root)Cloud Storage with Bucket Lock (bucket-wide, so more buckets for tiered retention)
Replay queryKQLAthena over exported logsBigQuery
Gaps to bridgeCleanest native stack for Azure-first firmsWorkstation hook and OTel events need a Lambda or EventBridge bridge; shortest path for the S3 session-store adapterAd-hoc replay experience lags KQL

[FACT-CHECK: product names and immutability guarantees on each provider, read directly from provider documentation]

Two cautions from Book 2: the immutability primitive is the most variable component across providers, so the retention team reads each primitive itself; and OTLP is the cleanest portability seam. Do not run the spine on three clouds at once: pilot on one, prove it, then extend.

Adopting the spine

Ten-week sequence

Book 2 Chapter 9 §7 gives the sequence for an ECB-supervised Tier-1 bank of around 16,000 engineers. A smaller institution (around 1,000 engineers, single business unit) compresses it to five weeks: edge collector, CI/CD wrap, CAB wrap, deployer wrap, then queries, one per week. Collapse the team count, not the controls.

WeeksStepExit condition
1Pin the spec versions (floor v0.1.0 for all three schemas)Pins recorded
2–3Stand up the edge collector for one business unitThe first lake event is an ops.doctor record from a leaf in that unit, validated end to end; nothing else is unblocked until it is
4–5Wrap existing CI/CD with ops run so pipeline commands emit signed records; pilot one product team, then the unitPilot records validate
6–7Wrap the existing CAB with ops cab evaluate, producing cab_decision records from the ITSM changeThe human Board reads the schema’s dossier
8–9Wrap the existing deployer with ops deploy, enforcing the manifest rulesManifest rules enforced in the pipeline
10Write the supervisor and auditor queries against the lake and show them to the second lineThe second line reads them without translation; if not, repeat weeks 6–9

ops is the Supporting Tools reference CLI (ops run, ops log, ops cab, ops deploy, ops chk, ops doctor). The Supporting Tools’ minimum credible v1 (D8) is leaf JSONL, a signed forwarder, one edge per pilot unit, immutability-enabled storage and an OTLP SIEM feed, with no bespoke query layer.

Wrap, do not replace

Adoption is a strangler fig: existing tools keep their jobs and the spine wraps them at the seams. The ITSM keeps owning change records (the CAB change record is rendered from it or emitted alongside). CI/CD keeps owning pipelines. The SIEM keeps alerting, fed from the lake over OTLP-shaped transport carrying the AES payload. The model registry, identity provider and data catalogue keep their roles; the spine reads them rather than duplicating them.

The catalogue law (Book 2): a registry is either a by-product of the production line or a graveyard. Any register fed by documentation rots at the rate the estate changes; only registers whose entries are emitted by the act they describe survive. Book 09 illustrates it: a hand-kept agent register still named a model the agent no longer used, while the emitted record was right because no one maintained it.

Deployer Manifest rules the spine enforces

Book 2 Chapter 9 §4.3; full treatment on release engineering:

  • metadata.agent_wrangler_upn is required: every deployment names its accountable human.
  • required_cab_decision is always present; null only with a cab_exemption_rationale citing the policy that allows the exemption.
  • verify.health_checks must be non-empty; rollback.verification is required. Health checks are a union of http, command and metric kinds.
  • Each of plan, preflight, apply, verify and rollback emits its own AES record (subject to the kind-name conflict noted above).

Anti-patterns and recovery

Anti-patternRecognise itRecovery (source)
Bespoke schema behind a translation layerTranslation drifts from the original after a quarterEmit natively; the migration is one-off, the translation is forever (Book 2)
Skipped join keysSingle-record queries work, cross-cutting ones do notPropagate session_id, change_id and the correlation ID at every emission point; fix at the leaf, not the lake (Books 2, 3)
Approval queue detached from execution evidenceReplay joins oversight to action by handSame ingest fabric, same stores, same ID (Book 3)
CAB as rubber stampcab_decision.reasoning[] empty or boilerplateWrangler reads three reasoning[] entries a month; second line reads ten a quarter (Book 2)
Trivial verificationA health check that only pings a load balancerReview health checks at the Virtual CAB as prompts are reviewed at model risk (Book 2)
Two-team boundaryOne unit declines; cross-unit queries failExecutive decision: declare the spine firm-wide, give holdouts 90 days, minute it (Book 2)
Retention misaligned with tierTier 3 record purged on a Tier 1 policyTier-aware routing at write into locked per-class containers (Book 3)
Replay never testedFirst attempt happens at the supervisor’s requestQuarterly rehearsal with a calendar invite, random ID and measured time (Book 3)
Evidence lost between capture and storeTruncation, compaction, dropped mirror batchesTiered truncation, PreCompact export, owned mirror_error alert (Book 3)
Aggregate telemetry mistaken for evidenceA dashboard demo closes the evidence questionThe route-approval evidence gate (Book 3)

Acceptance checks

A practitioner can sign off the spine when all of these hold, each demonstrated with evidence rather than asserted:

  • Every producer emits AES records that validate against the pinned version; the CI suite includes the positive examples and both negative cases (missing required field; required: true without workflow_id/approvers).
  • An unsigned test record produces an anomaly event; an invalid signed record produces a rejection record; neither reaches the lake.
  • Every production agent’s records carry a per-agent agent_id_claim that resolves to a registry record with a named wrangler.
  • A request injected without X-Correlation-ID below the gateway appears in the platform gap queue.
  • Approval records for a sampled allow-with-approval action are found by the same replay query as the execution records.
  • One synthetic record per retention class lands in the correctly locked container; the container’s retention cannot be shortened.
  • A retention cycle emits signed retention.delete records, the hash chain still verifies after crypto-shredding, and a held record is refused with an audit trail.
  • Backups are inside retention coverage.
  • A supervisor-role query without scope_assertion is refused and recorded; every query emits query.run.
  • The latest quarterly rehearsal met the tier targets, its findings have owners and dates, and the rehearsal record is stored under the rehearsal- prefix.
  • Every approved vendor surface has a completed route-approval evidence gate on file, with any tier cap recorded in the registry.
  • Collector-liveness, OTel export-failure and mirror_error alerts exist, have owners and have fired in a test.
  • This month’s twenty-change sample was drawn by someone outside the teams and answered from one query.

Failure and recovery quick reference

SymptomLikely causeFirst action
Replay returns records from one capture point onlyCorrelation ID not propagatedCheck the gap queue; fix propagation at the emitting layer
Open sessions accumulatingLeaf crashed or forwarder stoppedHeartbeat alert; inspect leaf outbox backlog; never delete un-counter-signed entries
Sudden fall in event volume from one agent or surfaceSilent export failure or collusionTreat absence as an incident; check collector liveness and signer status
Tier 3 full payload missing in replayHot-path truncation without immutable full writeConfirm tier tag and fan-out; backfill is usually impossible, so raise a finding
Records expiring earlyMis-classified at writeRaise a finding; correct routing; the lost window cannot be recovered
Retention cycle refusedPolicy pack expired or hold engine unreachableRepublish the pack; restore the interlock; do not bypass
Supervisor query needs manual joinsSeparate stores, skipped keys or a translation layerConverge on one envelope and one table; repeat the adoption steps for the holdout

Source status and conflicts

This page synthesises Book 1 Appendix B, Book 2 Chapters 1, 9 and 11, Book 3 Chapters 3, 9, 10, 11, 14 and 18 with Appendices E and G, Book 09 Chapter 9 and its closing, and the Supporting Tools specification, decision log and draft APIs. The query API (D25) and the retention and legal-hold engines are drafts or proposals, not built services. Conflicts the page surfaces rather than blends:

  • Two numberings of the nine questions (record layer versus capture layer); crosswalk above.
  • Three Book 3 retention schedules plus a kind-based model in the Supporting Tools; the five-year DORA attribution in Book 1 is [FACT-CHECK]; the page defers periods to the institution’s legal decision.
  • Compliance API retention: “none stated” (Book 3 Ch 11) versus six years (App E, Ch 14); the page follows Chapter 11.
  • Erasure collisions: DPO decides per record class (Book 3) versus “apply the shorter retention” (retention engine); the page follows Book 3.
  • kind names in several sources do not match the v0.1.0 pattern; the page follows the schema.
  • Policy-decision and approval-decision enumerations differ between Book 1 and Book 3; pick one and map.
  • Session timeout closure recorded as abandoned or timeout in different Supporting Tools documents.

Vendor behaviour in the Claude section is dated to August 2026 and must be re-verified; the per-surface coverage table should be re-run every two quarters, and this page refreshed on each relevant vendor release.

Source material

  • Book 1, Appendix B, Operating Schemas: AES, Control Plane, Model Usage, Data Dictionary and Change Dossier
  • Book 1, Chapter 6, DORA in Depth (logging at AI-system granularity)
  • Book 2, Chapter 1, The Day the Review Queue Broke (what counts as evidence)
  • Book 2, Chapter 9, The Audit Spine
  • Book 2, Chapter 11, Identity, Secrets and Configuration (executing_agent.agent_id_claim)
  • Book 3, Chapter 3, The Control Plane Is Yours (evidence layer, per-product coverage and per-audience projections)
  • Book 3, Chapter 9, The Anthropic API and Agent SDK (extended-thinking traces)
  • Book 3, Chapter 10, Data Classification, Retention and DLP (Compliance API coverage)
  • Book 3, Chapter 11, Evidence Capture and the Audit Schema (Practitioner Reference PR-1 to PR-5)
  • Book 3, Chapter 14, Risk Tiers and the Use-Case Taxonomy (evidence controls per tier)
  • Book 3, Chapter 18, Cost, Performance and Gateway Operations (cost never trims evidence)
  • Book 3, Appendix E, Audit Evidence Schema for Claude Actions
  • Book 3, Appendix G, Risk-Tier Control Matrix (evidence retention class)
  • Book 09 (Everything Is Green), Chapter 9, One Query
  • Book 09 (Everything Is Green), Closing, An Afternoon with a Pencil
  • Supporting Tools, spec/audit_evidence_schema (README, versioning.md, audit_evidence_schema.v0.1.0.json)
  • Supporting Tools, planning/decisions_log.md (D4, D5, D6, D7, D13, D25)
  • Supporting Tools, docs/architecture/06_resilience_patterns_playbook.md and 04_data_flows_and_sequences.md
  • Supporting Tools, docs/api/engine_retention.md, engine_legalhold.md and m5_query_api.md
  • Supporting Tools, docs/reports/02_regulatory_queries.md (evidence pack)
  • Supporting Tools, docs/use_cases/02_ops_log.md

← All AI Delivery Factory guidance

On this page

  1. The decision this page supports
  2. Two numberings of the nine questions
  3. The record contract: AES v0.1.0
    1. Wire layering
    2. Top-level fields
    3. Field groups, required members and enums
    4. The conditional approval block
    5. Field-level answer rules
    6. Bindings to obligations and standards
    7. Freeze the envelope; extend by payload
    8. Kinds and payloads
    9. Versioning and pinning
  4. Capture points and joins
    1. Three capture points, and why no single one suffices
    2. The correlation-ID contract
    3. Approvals ride the same pipeline
    4. One identity everywhere: executing_agent.agent_id_claim
    5. The gateway record and policy-decision vocabulary
    6. Agent-end record types and payload truncation
  5. Integrity: authenticity, immutability, completeness
    1. Edge validation
    2. The leaf outbox and forwarder
    3. Append-only, including for corrections
  6. Storage, retention and deletion
    1. Tiered immutable storage
    2. Retention schedule: the sources disagree
    3. Retention engine: signed, superseding deletion
    4. Legal hold and its interlock with retention
    5. Retention versus erasure requests
    6. Cost never trims evidence
  7. Readers and the query surface
    1. Four readers, one surface, named cadences
    2. Query API: role scoping, legal basis, cost ceiling
    3. Signed, self-verifiable evidence pack
    4. Answering a supervisor query
    5. Count your vetoes
  8. Proving the spine works
    1. Quarterly replay rehearsal
    2. Route-approval evidence gate
    3. Twenty-change oversight sample
    4. The morning drill
  9. Claude and Anthropic: where evidence is captured and lost
    1. Per-surface coverage
    2. The Compliance API: an extraction pipe, not a store
    3. OpenTelemetry: per-action events, silent export failure
    4. Transcripts, compaction and the session store
    5. Extended-thinking traces
    6. Cloud equivalents for the evidence stack
  10. Adopting the spine
    1. Ten-week sequence
    2. Wrap, do not replace
    3. Deployer Manifest rules the spine enforces
    4. Anti-patterns and recovery
  11. Acceptance checks
  12. Failure and recovery quick reference
  13. Source status and conflicts

All pages in Govern and prove →