Run a Discovery-Paced Portfolio
How to decide which problems become products when construction is cheap: the selection rule and its evidence, the nine-field bet card, the kill clause, a retirement recipe run before promotion, and the promotion boundary, with owners, failure modes and acceptance checks.
- Version
- 0.1.0
- Owner
- DEIXEI
- Last reviewed
- Reading time
- 19 min
For Cto · Product Leader · Ai Architect · Platform Engineer · Risk And Assurance · Internal Audit
Sources and details
- Section
- Design
- Source works
- Book 2
- Source chapters
- 5, 6, 21
- Source appendices
- 02:A
- Refresh trigger
- source change
When a governed internal application costs weeks of factory time rather than a year of programme budget, the factory stops being paced by how fast it can build. It is paced by how fast the firm can learn what is worth building. Book 2 (Ch 21 §3) calls the supply of that learning validated intent: a problem that has been observed, measured and stated precisely enough to steer agents. An agent lane that never waits for validated intent is not efficient; it is shipping untested intent at scale, with evidence records proving that the unused thing was built under full control.
Use this page at the front of the design stage, before choosing the product shape. The output of using it is, for each candidate, one of three recorded decisions:
- Admit — a bet card with fields 1 to 5 complete (Book 2’s admission point) and, as this page recommends, field 8 named, admitted to the agent lane at the prototype tier.
- Promote — a bet card with all nine fields complete, a kill clause signed and a retirement recipe that has been run, crossing onto the standard application foundation.
- Decline, defer, buy or route out — with the reason written down, and, for a deferral, the trigger that reopens it.
Status of the model. The four rules, the bet card and the two metrics are the author’s operating proposal in Book 2 Chapter 21, not an industry standard or regulatory requirement. Asset Manager X and its staff are the book’s composite institution and people. The page’s own recommendations are labelled as such.
Why discovery now sets the pace
Book 2 (Ch 21 §4.1) traces the scarce skill moving upstream with each software-factory wave: construction in the classic factories, domain analysis in product lines, product judgement inside teams in platform engineering, and now specification and selection. Construction still matters; it has become something people supervise.
The costs of feeding the factory unvalidated intent land in four places (Ch 21 §2):
| Cost | What it consists of | Why it matters in a regulated firm |
|---|---|---|
| Direct | Build, run and token cost, plus the permanent operational tail of anything that reaches production | Shipped-but-unused is the factory’s cost trap (running the factory) |
| Reviewer attention | Every item in the agent lane consumes human review at the same rate, valuable or not | Review capacity is the one input that has not become elastic |
| Supervisory | Each production system keeps its register entry, owner, evidence trail, resilience test and, where relevant, its critical-or-important-function assessment, whether or not anyone uses it | Forty half-used applications are one large, unbudgeted, permanent commitment |
| People | Product managers trained to ration construction, designers asked whether they still have a job | Unanswered, the market answers by hiring them away |
Two role changes follow and belong in the operating model, not only in job titles:
- The product manager moves from rationing to selection and specification. Ranking thirty requests when twenty can be built is a formality. The job becomes deciding which problems deserve to exist as products at all, and writing a testable statement of “good” that an agent cannot misread (agent-speed SDLC).
- The designer becomes the curator of the constraint set. The design system becomes a versioned, machine-readable practice injected into every agent thread that builds a user-facing surface. It must be written to be obeyed, not consulted: “card padding is 16px; deviations require a recorded exception” is checkable, “use whitespace generously” is not (Ch 21 §4.3). In a supervised firm the constraint set also carries control conventions: how confidence and citations are shown on generated output, how approvals are captured, and what a user sees when an agent acted on their behalf.
The four rules
Book 2 Ch 21 §5 states the discipline as four rules and one artefact.
| Rule | What it requires | Who owns it | Evidence that it is being applied |
|---|---|---|---|
| 1. Rank by moat depth, not demand volume | Score each candidate on how much of the firm’s own proprietary data and process it draws on. Commodity data → probably buy (build, configure, consume); data only you hold → where the factory’s advantage compounds | Product lead proposes; portfolio review decides | Field 4 filled on every card; exceptions named as exceptions |
| 2. Size every bet to a learning horizon, not a scope | State the question the bet answers and the date it is answered by | Product lead | Field 5 is a question with a date, not a deliverable |
| 3. Write the kill clause before construction starts | A date and a measurable condition, agreed by the business sponsor while enthusiasm is high | Business sponsor signs | Field 7 signed and dated before the first construction commit |
| 4. No promotion without a retirement recipe that has been run | Run, not written: executed at least once against the prototype | Product lead with the platform team | Field 9 holds a recipe identifier and a run date earlier than the promotion date |
Rule 2 in practice (Book 2’s example): a six-week question about whether operations will accept an agent drafting the lease abstract is a bet; “Q1 delivery of the lease abstraction module” is a project dressed as one.
Select from evidence, not from volume
A selection rule needs an ordering of problems that does not depend on who is loudest. Book 2 (Ch 21 §4.2, with the intake mechanism of Ch 24) builds it from evidence the firm already produces.
- Chart the defects continuously. Every function produces a record of its own defects: reconciliation breaks, exception queues, manual touches, rework, late reports, complaints, NAV corrections. Run a standing agent job that pulls the operational data weekly and produces a Pareto chart ranked by volume and by cost, with the tail visible.
- Work the top two or three bars back to causes across six categories: people, process, policy, data, system, measurement. A cause-and-effect diagram and five whys are enough.
- Attach evidence to every “because”. Each link in the chain carries a pointer to its evidence. Agents may draft the chain (minutes rather than days), but a drafted chain is plausible by construction, so a named human accepts it, pointer by pointer, before it routes anywhere.
- Route the causes. Causes owned outside the factory go to their owners. Only what is left becomes a candidate product. Saying “this one is not ours, and here is the evidence” protects reviewer attention.
- Record the baseline. The bar chosen for attack is measured, dated and recorded before construction. It is the same number read at intake, in production and on the kill date (see one baseline, read three times).
Write the selection rule down (Ch 21 §7). One page: what makes a problem worth becoming a product here, ordered by proprietary-data depth, with exceptions named; and what evidence a problem must arrive with — the defect data showing the bar and its size, and a cause chain whose remaining causes are the factory’s. A selection rule without an evidence requirement is a preference.
Composite example (Asset Manager X, Book 2 Ch 21 §6). The largest bar for the client-reporting family was late quarterly factsheets. A reporting-exceptions console had already been built for exactly that pain, in six weeks, and nobody outside the squad had opened it. An agent drafted the cause chain in about eleven minutes; two people spent about an hour accepting it. The causes were a benchmark feed landing after an old vendor cut-off and a sign-off sequenced after the feed: a data cause and a process cause, no system cause. The fix went back to the business and never entered the factory.
The bet card
The bet card is a one-page operating artefact (Book 2 classes it Level 2), held next to the product’s bill of material. It is the only admissible input to the portfolio review.
| # | Field | What it must hold | Completion rule |
|---|---|---|---|
| 1 | Problem | The user problem in one sentence, in the user’s words, dated | Must state a problem, not a system. “We need a breaks dashboard” fails |
| 2 | Who it serves | A named population and its size | ”The business” fails |
| 3 | Validation evidence | What has been observed (interviews, an instrumented experiment, telemetry from an adjacent product) and the measured baseline the bet is meant to move, dated | Empty means the bet is not yet validated intent and may not enter the agent lane |
| 4 | Moat rank | The candidate’s position on the proprietary-data ordering (Rule 1) | A low-moat bet selected anyway says why (for example, “lighthouse”) |
| 5 | Learning horizon | The question, and the date it is answered by | A deliverable or a quarter alone fails |
| 6 | Promotion class | Prototype tier, or promoted product on the standard application foundation | Changes only through the promotion crossing |
| 7 | Kill clause | A date plus a measurable condition, read against the field 3 baseline | Signed by the business sponsor before construction |
| 8 | Named humans | Product lead, wrangler, adoption lead | Three names, no blanks. The wrangler carries the agent’s bounded agency into the team’s accountability (agent lifecycle) |
| 9 | Retirement recipe | The identifier of the recipe that decommissions the product, and the date it was last run | Required, with a run date, before promotion |
Fields 1 to 5 are the product manager’s selection work. Field 6 is the promotion boundary. Fields 7 and 9 make the bet an option rather than a commitment. Field 8 ties the bet to named accountability.
Admission and promotion thresholds (the page’s reading of Ch 21 §5):
| Transition | Fields that must be complete |
|---|---|
| Candidate → admitted to the agent lane (prototype tier) | 1–5, plus 8 |
| Prototype → promoted product | 1–9, with field 7 signed and field 9 showing a run date |
| Any → killed | Field 7’s condition met or the date passed without it being met; retirement recipe executed |
Illustrative record layout. Book 2 defines the nine fields but no machine-readable schema. A layout such as the following keeps the card diffable next to the BOM; the headings are the source’s field names, the layout is this page’s suggestion and is not an AES schema.
# Bet card — <product name>
1. Problem (dated):
2. Who it serves (population, size):
3. Validation evidence (observation, sample, period, falsifier) + baseline (value, date, source):
4. Moat rank:
5. Learning horizon (question; answered by <date>):
6. Promotion class: prototype tier | standard application foundation
7. Kill clause (date; condition vs field 3 baseline; sponsor signature):
8. Named humans: product lead / wrangler / adoption lead
9. Retirement recipe (id; last run <date>; elapsed time):
Give field 3 a shape. To prevent discovery theatre (below), the page recommends recording for every validation claim: what was observed, how many users, over what period, and what result would have falsified the hypothesis. If nothing could have falsified it, no validation occurred (Ch 21 §8).
The kill clause
An option has value because you can decline to exercise it. A bet you cannot stop is a commitment with optimistic paperwork (Ch 21 §4.4).
How to write it.
- Take the condition from the baseline recorded in field 3. Do not invent it when the clause is written; a condition invented at that moment ends up unfalsifiable.
- State the date and the expected movement of the bar: “by
, below ”. - Have the business sponsor sign it before construction starts, at the point of enthusiasm rather than disappointment.
- Record it in field 7 and diarise the kill review on the portfolio-review cadence.
How it fires. Re-run the chart on the stated date. If the bar has not shrunk — the breaks did not fall, the queue is the same length, the reports are still late — the bet failed, however well the product was built and however much it is used. Adoption does not override the clause; this is what stops a well-liked system that changed nothing from living forever on its usage numbers.
Who can bring it forward. The wrangler and the adoption lead have a standing right to bring a kill clause forward. They see the telemetry first, and a selection body whose cards are never amended after admission has lost its feedback loop.
The retirement recipe, run before promotion
In a regulated firm, stopping is not free by default. A system that reached production carries obligations that outlive its usage. You can only cheaply retire what you can completely enumerate (Ch 21 §4.4): the BOM is what turns decommissioning into a recipe rather than archaeology.
What the recipe must do, taken from the obligations Book 2 lists:
| Step | Action | Evidence of completion |
|---|---|---|
| 1 | Dispose of data per its classification and retention rules | Disposal record per classified data domain in the BOM |
| 2 | Revoke entitlements and retire agent and service identities and secrets | Identity provider shows the identities disabled; no credentials issued after the retirement date |
| 3 | Unwind integrations and APIs | Consumers notified; routes removed from gateway and broker |
| 4 | Remove runtime services, cloud resources and pipeline templates | Resource inventory shows none tagged to the product |
| 5 | Close and retain evidence | Evidence outputs closed and held for their retention period (audit evidence) |
| 6 | Amend the register entry | Register shows the product retired, with date and reason |
| 7 | If the product was ever assessed as supporting a critical or important function, prepare the supervisory conversation | Assessment updated; regulatory routing via the regulatory map |
The BOM must enumerate, at minimum, the repositories, runtime services, cloud resources, classified data domains, APIs, identities and secrets, pipeline templates and evidence outputs (Ch 21 §4.4). If any are missing, the recipe cannot be complete.
Run it before promotion. Rule 4 requires the recipe to have been executed against the prototype. If you cannot take it down, you have not shown you can afford to have put it up. Record the run date and elapsed time in field 9. If no retirement recipe exists, price the bet as the irreversible commitment it is.
First quarter (Ch 21 §7). Pick a low-stakes product nobody would miss and decommission it end to end by recipe. Time it. That figure is the true price of every option in the portfolio; until it exists, every kill clause is an untested promise. Agent-specific decommissioning steps are on agent lifecycle.
The promotion boundary
The dangerous artefact is not the failed prototype, which is a cheap lesson. It is the successful prototype that accretes users, data and dependencies without ever crossing into production, and six months later is load-bearing, unowned and unevidenced (Ch 21 §4.5). The standard application foundation therefore does a second job as the promotion boundary, and the governing law is: promotion must be cheaper than evasion.
| Below the boundary: prototype tier | Above the boundary: standard application foundation | |
|---|---|---|
| Data | No production data | Classified data under the product’s approvals |
| Lifetime | Time-limited, automatic eviction | Lifecycle governed by the kill clause and retirement recipe |
| Visibility | Instrumented so the platform team sees what is built and how much it is used | Registered, with BOM, owner, usage telemetry and evidence |
| Controls | Sandbox isolation | Identity, authorisation, telemetry, deployment and evidence inherited from the foundation |
Threshold. Measure promotion elapsed time. If it exceeds two weeks, fix that before anything else (Ch 21 §7, 3,000-person firm). The crossing checklist, the per-cloud prototype-tier services (Azure, AWS, Google Cloud) and the advice to close the low-code default environment only after the crossing is fast are on application foundation and landing zone.
Treat shadow builds as validated demand. A prototype someone built outside the factory and that people use daily is often the strongest field 3 evidence available. The response is a fast crossing, not a switch-off.
Composite example (Asset Manager X, Book 2 Ch 21 §6). An analyst built a lease-extraction tool in an unclosed low-code environment: fourteen fields per lease, used daily by nine people, roughly 1,400 leases processed, some under confidentiality provisions, with no owner, evidence, entitlement model or tests. It was found through token telemetry. It crossed: the foundation was stamped, extraction ran through the governed pipeline with entitlement-aware retrieval, a wrangler and adoption lead were named, and the extraction schema and confidence thresholds went into the card. Decision to promoted product took nine working days, mostly on the entitlement model. It became release one of the top moat-ranked product.
Run the portfolio review
Cadence and forum. Quarterly portfolio review, with the CIO in the chair and the CRO in the room (Ch 21 §7, 3,000-person firm). The bet card is the only admissible input. Put a scheduled kill review on the same quarterly cadence.
Agenda the page recommends, assembled from Ch 21 §5–§7:
- Defect charts and accepted cause chains for each family; causes routed out of the factory.
- New candidates ranked by moat (Rule 1), with exceptions named.
- Admission decisions: field 3 non-empty, field 5 a dated question.
- Promotion decisions: kill clause signed, retirement recipe run, promotion time measured.
- Kill review: every clause whose date has passed; every clause brought forward by a wrangler or adoption lead.
- The two discovery metrics (below), read together with lane utilisation.
- Deferrals, each with a written reason and trigger.
Composite reading (Asset Manager X). Eleven candidates. Translation ranked last on moat and went to buy; report drafting ranked lower than its sponsorship suggested because its inputs were published documents; real-assets intelligence ranked first. The knowledge assistant ranked low on moat and was selected anyway as a lighthouse, because it exercised the whole chassis end to end and countered shadow AI; field 4 said so. Four candidates were killed or reshaped at the table: one to buy, one deferred a quarter with a written reason, one folded into another product’s scope, and one held until the review console had two quarters of operating history, because the risk function would not sign an autonomy ladder without it.
Two numbers that make the claim measurable
Book 2 (Ch 21 §5) defines two metrics, read together:
- Validated-intent lead time — median elapsed time from a problem entering discovery to a bet card with fields 1 to 5 complete and admitted to the agent lane. It ends where the intent-to-production lead time begins.
- Intent coverage ratio — validated intent admitted per period divided by agent-lane capacity consumed per period, in one consistent unit.
| Ratio | Reading |
|---|---|
| Comfortably above 1 | Discovery is ahead; the lane is the constraint (the old world) |
| At or below 1 | Discovery is the constraint, where a working factory ends up |
| Far below 1 with the lane running flat out | The lane is being filled with unvalidated intent; confirm in the field 3 column |
The composite first reading was 31 days and 0.6. Instrumentation, scorecard placement and thresholds are on factory measurement.
Scale the practice to the firm
| Firm (Book 2 Ch 21 §7) | Do this | Watch for |
|---|---|---|
| About 1,000 people | Put the bet card in existing tooling; make field 3 mandatory before anything enters the agent lane; accept a validated-intent lead time in weeks. Do not build a discovery function | The promotion boundary staying informal. Write the two tiers down even if the prototype tier is one subscription and a naming convention |
| About 3,000 people | Quarterly review with CIO chairing and CRO present; fund one adoption lead per product family; build the prototype tier deliberately (sandboxed, auto-evicting, no production data, fully visible) | Promotion taking more than two weeks |
| About 50,000 people | Push the bet card down to product families. Pull the moat ranking and the retirement recipe up to the centre. Give the centre one veto (decline a bet that duplicates another division’s) and one obligation (guarantee promotion in days) | A centre that only vetoes, which becomes a tollbooth |
Everyone, week one. Put usage telemetry beside every product shipped last quarter and count how many are used weekly by more than a defined threshold of people. Then find where non-engineers build — the low-code default environment, personal subscriptions on expense claims, notebooks on unmanaged laptops — looking for validated demand that had nowhere to go, not offenders. Week two: write the selection rule.
Failure modes and recovery
| Failure mode (Ch 21 §8) | How to recognise it | Recovery |
|---|---|---|
| Empty-lane panic | A utilisation target appears for the agent lane; cards enter the lane with field 3 blank | Show the coverage ratio to the same audience as utilisation. An idle lane costs tokens not spent; a misdirected lane costs review attention, run cost and a permanent tail |
| Discovery theatre | Artefacts with sentiment but no observation (quotes, no instrumented behaviour); validated-intent lead time lengthening without better admitted intent | Give field 3 its shape: what was observed, how many users, over what period, and what would have falsified it |
| Solution-shaped request | Field 1 names a system; field 3 cites the sponsor’s seniority rather than a measurement | Turn the request back into a problem with a cause chain and evidence before solution shaping starts; treat this as the first increment of intake, not a gate |
| Selection board that does not build | Bet cards never amended after admission; builders cannot challenge selection | Give the wrangler and adoption lead a standing right to bring a kill clause forward |
| Landfill | Rising count of production systems with an empty field 9, or whose named humans have left. Book 2 calls this the leading indicator, preceding the pain by about a year | Refuse promotion without a run retirement recipe; hold a scheduled kill review each quarter |
| Tollbooth boundary | Promotion elapsed time exceeds the time a business unit takes to build outside; shadow tools discovered later | Cut the crossing to days by supplying controls by inheritance first; only then tighten the supply side. Closing the default environment before a fast crossing exists moves shadow AI somewhere with less telemetry |
Acceptance checks
- A one-page selection rule exists, names its exceptions and states the evidence a problem must arrive with.
- A defect Pareto runs at least weekly per product family, and each accepted cause chain carries an evidence pointer per link and a named acceptor.
- Every item in the agent lane has a bet card with fields 1 to 5 and 8 complete; no admitted card has an empty field 3.
- Every promoted product has a signed kill clause dated before its first construction commit and a field 9 run date earlier than its promotion date.
- At least one real product has been retired end to end by recipe, and the elapsed time is recorded.
- Promotion elapsed time is measured; the median is under two weeks.
- The count of production systems with an empty field 9, or with a departed named human, is reported each quarter and is not rising.
- Validated-intent lead time and the intent coverage ratio are on the scorecard; there is no agent-lane utilisation target.
- The quarterly kill review has minutes showing clauses fired, extended with reasons, or brought forward.
What this page does not decide
It does not choose the product’s architecture; once a card is admitted, choose the AI product shape and classify the risk tier. It does not set the buy-versus-build decision for low-moat candidates; see build, configure, consume. Intake increments, triage and the capacity-bounded roadmap are on running the factory.
Source notes. The four rules, nine fields, two metrics and six failure modes are from Book 2 Chapter 21. The admission and promotion thresholds table, the retirement evidence column, the review agenda, the record layout and the acceptance checks are this page’s operationalisation of that chapter. Book 2 gives the composite lease tool as roughly 1,400 leases; Book 09 (Ch 10) describes a similar composite at roughly 1,500 (cited on application foundation). The figures are illustrative either way.
Source material
- Book 2 (The AI Delivery Factory), Chapter 21, Product Development at Agent Speed (§2–§8: validated intent, selection, kill clause, promotion boundary, the discovery-paced portfolio, bet card)
- Book 2 (The AI Delivery Factory), Appendix A, Glossary (bet card, promotion boundary)
- Book 2 (The AI Delivery Factory), Chapter 5, Bills of Material and Recipes (the BOM as the enumeration that makes retirement possible)
- Book 2 (The AI Delivery Factory), Chapter 6, The Standard Application Foundation (the foundation as the governed starting point and promotion boundary)
Related guidance
-
Choose the Shape of an AI ProductA least-agency decision sequence for choosing a product design: agency classes and their trade-offs, the eight questions, evidence for every step up, the consequence matrix that caps action scope, the cut-set review for agents, the other selection dimensions, and the design record. -
Build a Living Product Bill of Material and RecipeDescribe what an AI product contains and how the firm builds it in one versioned specification shared by pipelines, agents and assurance teams, including the classification line, readiness grades and reuse generations. -
Build the Standard Application FoundationRun the shared identity, telemetry, deployment, data and AI capabilities as a funded product that humans and agents both start from, with a contract, evidence-computed trust, a cheap promotion boundary and reuse measured by consumption. -
Run the AI Delivery FactoryOperate demand, capacity, the two worker classes, exceptions and evidence as one service: read two leading numbers, keep a handover backlog, gate construction on three conditions, run queues and alerts at agent volume, and count unused products as cost. -
Measure the FactoryRead usage first, then the seven-dimension scorecard; emit a served-request event that separates human from agent callers; label every reported number by evidence tier; watch the two leading numbers of a control gap; treat an unexplained sharp improvement as an alarm; and never turn a participation metric into a target. -
Operate the Agent LifecycleInventory, register, sign, zone, review, suspend, recover and retire agents through a six-stage lifecycle with named human ownership, zone-specific authority and evidence at every transition. -
Build, Configure or ConsumeThe layered sourcing decision: a signed portfolio threshold (8 products, 24 months, 3 shared patterns including shared data) decides whether building is available at all; Research Paper 7's four gates decide each capability; Book 2's five weighted dimensions score the contested cases. Covers the annual estate sort, the audit-log export clause for rented workflows, the model-sourcing breakeven test and register, and who owns what is built. -
Prepare a Governed Landing Zone for AI DeliveryDesign the management-group hierarchy, enforce guardrails in deny mode, give humans and agents the same paved workspace, integrate the eight agent-platform components, close the perimeter, and prove the zone is done with seven observable exit criteria and a second-line sign-off. -
Classify Use Cases into Risk TiersThe T0–T4 use-case taxonomy as a working decision reference: the prohibited-use gate, four intake questions, data and action boundaries per tier, the twelve-area control matrix, retention and payload rules, runtime enforcement, approval gates, regulatory routing and re-tiering.