Browse the reference ⌄
Decision Guide published

Run a Discovery-Paced Portfolio

How to decide which problems become products when construction is cheap: the selection rule and its evidence, the nine-field bet card, the kill clause, a retirement recipe run before promotion, and the promotion boundary, with owners, failure modes and acceptance checks.

Version
0.1.0
Owner
DEIXEI
Last reviewed
Reading time
19 min

For Cto · Product Leader · Ai Architect · Platform Engineer · Risk And Assurance · Internal Audit

Sources and details
Section
Design
Source works
Book 2
Source chapters
5, 6, 21
Source appendices
02:A
Refresh trigger
source change

Full source list ↓ Source index →

When a governed internal application costs weeks of factory time rather than a year of programme budget, the factory stops being paced by how fast it can build. It is paced by how fast the firm can learn what is worth building. Book 2 (Ch 21 §3) calls the supply of that learning validated intent: a problem that has been observed, measured and stated precisely enough to steer agents. An agent lane that never waits for validated intent is not efficient; it is shipping untested intent at scale, with evidence records proving that the unused thing was built under full control.

Use this page at the front of the design stage, before choosing the product shape. The output of using it is, for each candidate, one of three recorded decisions:

  • Admit — a bet card with fields 1 to 5 complete (Book 2’s admission point) and, as this page recommends, field 8 named, admitted to the agent lane at the prototype tier.
  • Promote — a bet card with all nine fields complete, a kill clause signed and a retirement recipe that has been run, crossing onto the standard application foundation.
  • Decline, defer, buy or route out — with the reason written down, and, for a deferral, the trigger that reopens it.

Status of the model. The four rules, the bet card and the two metrics are the author’s operating proposal in Book 2 Chapter 21, not an industry standard or regulatory requirement. Asset Manager X and its staff are the book’s composite institution and people. The page’s own recommendations are labelled as such.

Why discovery now sets the pace

Book 2 (Ch 21 §4.1) traces the scarce skill moving upstream with each software-factory wave: construction in the classic factories, domain analysis in product lines, product judgement inside teams in platform engineering, and now specification and selection. Construction still matters; it has become something people supervise.

The costs of feeding the factory unvalidated intent land in four places (Ch 21 §2):

CostWhat it consists ofWhy it matters in a regulated firm
DirectBuild, run and token cost, plus the permanent operational tail of anything that reaches productionShipped-but-unused is the factory’s cost trap (running the factory)
Reviewer attentionEvery item in the agent lane consumes human review at the same rate, valuable or notReview capacity is the one input that has not become elastic
SupervisoryEach production system keeps its register entry, owner, evidence trail, resilience test and, where relevant, its critical-or-important-function assessment, whether or not anyone uses itForty half-used applications are one large, unbudgeted, permanent commitment
PeopleProduct managers trained to ration construction, designers asked whether they still have a jobUnanswered, the market answers by hiring them away

Two role changes follow and belong in the operating model, not only in job titles:

  • The product manager moves from rationing to selection and specification. Ranking thirty requests when twenty can be built is a formality. The job becomes deciding which problems deserve to exist as products at all, and writing a testable statement of “good” that an agent cannot misread (agent-speed SDLC).
  • The designer becomes the curator of the constraint set. The design system becomes a versioned, machine-readable practice injected into every agent thread that builds a user-facing surface. It must be written to be obeyed, not consulted: “card padding is 16px; deviations require a recorded exception” is checkable, “use whitespace generously” is not (Ch 21 §4.3). In a supervised firm the constraint set also carries control conventions: how confidence and citations are shown on generated output, how approvals are captured, and what a user sees when an agent acted on their behalf.

The four rules

Book 2 Ch 21 §5 states the discipline as four rules and one artefact.

RuleWhat it requiresWho owns itEvidence that it is being applied
1. Rank by moat depth, not demand volumeScore each candidate on how much of the firm’s own proprietary data and process it draws on. Commodity data → probably buy (build, configure, consume); data only you hold → where the factory’s advantage compoundsProduct lead proposes; portfolio review decidesField 4 filled on every card; exceptions named as exceptions
2. Size every bet to a learning horizon, not a scopeState the question the bet answers and the date it is answered byProduct leadField 5 is a question with a date, not a deliverable
3. Write the kill clause before construction startsA date and a measurable condition, agreed by the business sponsor while enthusiasm is highBusiness sponsor signsField 7 signed and dated before the first construction commit
4. No promotion without a retirement recipe that has been runRun, not written: executed at least once against the prototypeProduct lead with the platform teamField 9 holds a recipe identifier and a run date earlier than the promotion date

Rule 2 in practice (Book 2’s example): a six-week question about whether operations will accept an agent drafting the lease abstract is a bet; “Q1 delivery of the lease abstraction module” is a project dressed as one.

Select from evidence, not from volume

A selection rule needs an ordering of problems that does not depend on who is loudest. Book 2 (Ch 21 §4.2, with the intake mechanism of Ch 24) builds it from evidence the firm already produces.

  1. Chart the defects continuously. Every function produces a record of its own defects: reconciliation breaks, exception queues, manual touches, rework, late reports, complaints, NAV corrections. Run a standing agent job that pulls the operational data weekly and produces a Pareto chart ranked by volume and by cost, with the tail visible.
  2. Work the top two or three bars back to causes across six categories: people, process, policy, data, system, measurement. A cause-and-effect diagram and five whys are enough.
  3. Attach evidence to every “because”. Each link in the chain carries a pointer to its evidence. Agents may draft the chain (minutes rather than days), but a drafted chain is plausible by construction, so a named human accepts it, pointer by pointer, before it routes anywhere.
  4. Route the causes. Causes owned outside the factory go to their owners. Only what is left becomes a candidate product. Saying “this one is not ours, and here is the evidence” protects reviewer attention.
  5. Record the baseline. The bar chosen for attack is measured, dated and recorded before construction. It is the same number read at intake, in production and on the kill date (see one baseline, read three times).

Write the selection rule down (Ch 21 §7). One page: what makes a problem worth becoming a product here, ordered by proprietary-data depth, with exceptions named; and what evidence a problem must arrive with — the defect data showing the bar and its size, and a cause chain whose remaining causes are the factory’s. A selection rule without an evidence requirement is a preference.

Composite example (Asset Manager X, Book 2 Ch 21 §6). The largest bar for the client-reporting family was late quarterly factsheets. A reporting-exceptions console had already been built for exactly that pain, in six weeks, and nobody outside the squad had opened it. An agent drafted the cause chain in about eleven minutes; two people spent about an hour accepting it. The causes were a benchmark feed landing after an old vendor cut-off and a sign-off sequenced after the feed: a data cause and a process cause, no system cause. The fix went back to the business and never entered the factory.

The bet card

The bet card is a one-page operating artefact (Book 2 classes it Level 2), held next to the product’s bill of material. It is the only admissible input to the portfolio review.

#FieldWhat it must holdCompletion rule
1ProblemThe user problem in one sentence, in the user’s words, datedMust state a problem, not a system. “We need a breaks dashboard” fails
2Who it servesA named population and its size”The business” fails
3Validation evidenceWhat has been observed (interviews, an instrumented experiment, telemetry from an adjacent product) and the measured baseline the bet is meant to move, datedEmpty means the bet is not yet validated intent and may not enter the agent lane
4Moat rankThe candidate’s position on the proprietary-data ordering (Rule 1)A low-moat bet selected anyway says why (for example, “lighthouse”)
5Learning horizonThe question, and the date it is answered byA deliverable or a quarter alone fails
6Promotion classPrototype tier, or promoted product on the standard application foundationChanges only through the promotion crossing
7Kill clauseA date plus a measurable condition, read against the field 3 baselineSigned by the business sponsor before construction
8Named humansProduct lead, wrangler, adoption leadThree names, no blanks. The wrangler carries the agent’s bounded agency into the team’s accountability (agent lifecycle)
9Retirement recipeThe identifier of the recipe that decommissions the product, and the date it was last runRequired, with a run date, before promotion

Fields 1 to 5 are the product manager’s selection work. Field 6 is the promotion boundary. Fields 7 and 9 make the bet an option rather than a commitment. Field 8 ties the bet to named accountability.

Admission and promotion thresholds (the page’s reading of Ch 21 §5):

TransitionFields that must be complete
Candidate → admitted to the agent lane (prototype tier)1–5, plus 8
Prototype → promoted product1–9, with field 7 signed and field 9 showing a run date
Any → killedField 7’s condition met or the date passed without it being met; retirement recipe executed

Illustrative record layout. Book 2 defines the nine fields but no machine-readable schema. A layout such as the following keeps the card diffable next to the BOM; the headings are the source’s field names, the layout is this page’s suggestion and is not an AES schema.

# Bet card — <product name>
1. Problem (dated):
2. Who it serves (population, size):
3. Validation evidence (observation, sample, period, falsifier) + baseline (value, date, source):
4. Moat rank:
5. Learning horizon (question; answered by <date>):
6. Promotion class: prototype tier | standard application foundation
7. Kill clause (date; condition vs field 3 baseline; sponsor signature):
8. Named humans: product lead / wrangler / adoption lead
9. Retirement recipe (id; last run <date>; elapsed time):

Give field 3 a shape. To prevent discovery theatre (below), the page recommends recording for every validation claim: what was observed, how many users, over what period, and what result would have falsified the hypothesis. If nothing could have falsified it, no validation occurred (Ch 21 §8).

The kill clause

An option has value because you can decline to exercise it. A bet you cannot stop is a commitment with optimistic paperwork (Ch 21 §4.4).

How to write it.

  1. Take the condition from the baseline recorded in field 3. Do not invent it when the clause is written; a condition invented at that moment ends up unfalsifiable.
  2. State the date and the expected movement of the bar: “by , below ”.
  3. Have the business sponsor sign it before construction starts, at the point of enthusiasm rather than disappointment.
  4. Record it in field 7 and diarise the kill review on the portfolio-review cadence.

How it fires. Re-run the chart on the stated date. If the bar has not shrunk — the breaks did not fall, the queue is the same length, the reports are still late — the bet failed, however well the product was built and however much it is used. Adoption does not override the clause; this is what stops a well-liked system that changed nothing from living forever on its usage numbers.

Who can bring it forward. The wrangler and the adoption lead have a standing right to bring a kill clause forward. They see the telemetry first, and a selection body whose cards are never amended after admission has lost its feedback loop.

The retirement recipe, run before promotion

In a regulated firm, stopping is not free by default. A system that reached production carries obligations that outlive its usage. You can only cheaply retire what you can completely enumerate (Ch 21 §4.4): the BOM is what turns decommissioning into a recipe rather than archaeology.

What the recipe must do, taken from the obligations Book 2 lists:

StepActionEvidence of completion
1Dispose of data per its classification and retention rulesDisposal record per classified data domain in the BOM
2Revoke entitlements and retire agent and service identities and secretsIdentity provider shows the identities disabled; no credentials issued after the retirement date
3Unwind integrations and APIsConsumers notified; routes removed from gateway and broker
4Remove runtime services, cloud resources and pipeline templatesResource inventory shows none tagged to the product
5Close and retain evidenceEvidence outputs closed and held for their retention period (audit evidence)
6Amend the register entryRegister shows the product retired, with date and reason
7If the product was ever assessed as supporting a critical or important function, prepare the supervisory conversationAssessment updated; regulatory routing via the regulatory map

The BOM must enumerate, at minimum, the repositories, runtime services, cloud resources, classified data domains, APIs, identities and secrets, pipeline templates and evidence outputs (Ch 21 §4.4). If any are missing, the recipe cannot be complete.

Run it before promotion. Rule 4 requires the recipe to have been executed against the prototype. If you cannot take it down, you have not shown you can afford to have put it up. Record the run date and elapsed time in field 9. If no retirement recipe exists, price the bet as the irreversible commitment it is.

First quarter (Ch 21 §7). Pick a low-stakes product nobody would miss and decommission it end to end by recipe. Time it. That figure is the true price of every option in the portfolio; until it exists, every kill clause is an untested promise. Agent-specific decommissioning steps are on agent lifecycle.

The promotion boundary

The dangerous artefact is not the failed prototype, which is a cheap lesson. It is the successful prototype that accretes users, data and dependencies without ever crossing into production, and six months later is load-bearing, unowned and unevidenced (Ch 21 §4.5). The standard application foundation therefore does a second job as the promotion boundary, and the governing law is: promotion must be cheaper than evasion.

Below the boundary: prototype tierAbove the boundary: standard application foundation
DataNo production dataClassified data under the product’s approvals
LifetimeTime-limited, automatic evictionLifecycle governed by the kill clause and retirement recipe
VisibilityInstrumented so the platform team sees what is built and how much it is usedRegistered, with BOM, owner, usage telemetry and evidence
ControlsSandbox isolationIdentity, authorisation, telemetry, deployment and evidence inherited from the foundation

Threshold. Measure promotion elapsed time. If it exceeds two weeks, fix that before anything else (Ch 21 §7, 3,000-person firm). The crossing checklist, the per-cloud prototype-tier services (Azure, AWS, Google Cloud) and the advice to close the low-code default environment only after the crossing is fast are on application foundation and landing zone.

Treat shadow builds as validated demand. A prototype someone built outside the factory and that people use daily is often the strongest field 3 evidence available. The response is a fast crossing, not a switch-off.

Composite example (Asset Manager X, Book 2 Ch 21 §6). An analyst built a lease-extraction tool in an unclosed low-code environment: fourteen fields per lease, used daily by nine people, roughly 1,400 leases processed, some under confidentiality provisions, with no owner, evidence, entitlement model or tests. It was found through token telemetry. It crossed: the foundation was stamped, extraction ran through the governed pipeline with entitlement-aware retrieval, a wrangler and adoption lead were named, and the extraction schema and confidence thresholds went into the card. Decision to promoted product took nine working days, mostly on the entitlement model. It became release one of the top moat-ranked product.

Run the portfolio review

Cadence and forum. Quarterly portfolio review, with the CIO in the chair and the CRO in the room (Ch 21 §7, 3,000-person firm). The bet card is the only admissible input. Put a scheduled kill review on the same quarterly cadence.

Agenda the page recommends, assembled from Ch 21 §5–§7:

  1. Defect charts and accepted cause chains for each family; causes routed out of the factory.
  2. New candidates ranked by moat (Rule 1), with exceptions named.
  3. Admission decisions: field 3 non-empty, field 5 a dated question.
  4. Promotion decisions: kill clause signed, retirement recipe run, promotion time measured.
  5. Kill review: every clause whose date has passed; every clause brought forward by a wrangler or adoption lead.
  6. The two discovery metrics (below), read together with lane utilisation.
  7. Deferrals, each with a written reason and trigger.

Composite reading (Asset Manager X). Eleven candidates. Translation ranked last on moat and went to buy; report drafting ranked lower than its sponsorship suggested because its inputs were published documents; real-assets intelligence ranked first. The knowledge assistant ranked low on moat and was selected anyway as a lighthouse, because it exercised the whole chassis end to end and countered shadow AI; field 4 said so. Four candidates were killed or reshaped at the table: one to buy, one deferred a quarter with a written reason, one folded into another product’s scope, and one held until the review console had two quarters of operating history, because the risk function would not sign an autonomy ladder without it.

Two numbers that make the claim measurable

Book 2 (Ch 21 §5) defines two metrics, read together:

  • Validated-intent lead time — median elapsed time from a problem entering discovery to a bet card with fields 1 to 5 complete and admitted to the agent lane. It ends where the intent-to-production lead time begins.
  • Intent coverage ratio — validated intent admitted per period divided by agent-lane capacity consumed per period, in one consistent unit.
RatioReading
Comfortably above 1Discovery is ahead; the lane is the constraint (the old world)
At or below 1Discovery is the constraint, where a working factory ends up
Far below 1 with the lane running flat outThe lane is being filled with unvalidated intent; confirm in the field 3 column

The composite first reading was 31 days and 0.6. Instrumentation, scorecard placement and thresholds are on factory measurement.

Scale the practice to the firm

Firm (Book 2 Ch 21 §7)Do thisWatch for
About 1,000 peoplePut the bet card in existing tooling; make field 3 mandatory before anything enters the agent lane; accept a validated-intent lead time in weeks. Do not build a discovery functionThe promotion boundary staying informal. Write the two tiers down even if the prototype tier is one subscription and a naming convention
About 3,000 peopleQuarterly review with CIO chairing and CRO present; fund one adoption lead per product family; build the prototype tier deliberately (sandboxed, auto-evicting, no production data, fully visible)Promotion taking more than two weeks
About 50,000 peoplePush the bet card down to product families. Pull the moat ranking and the retirement recipe up to the centre. Give the centre one veto (decline a bet that duplicates another division’s) and one obligation (guarantee promotion in days)A centre that only vetoes, which becomes a tollbooth

Everyone, week one. Put usage telemetry beside every product shipped last quarter and count how many are used weekly by more than a defined threshold of people. Then find where non-engineers build — the low-code default environment, personal subscriptions on expense claims, notebooks on unmanaged laptops — looking for validated demand that had nowhere to go, not offenders. Week two: write the selection rule.

Failure modes and recovery

Failure mode (Ch 21 §8)How to recognise itRecovery
Empty-lane panicA utilisation target appears for the agent lane; cards enter the lane with field 3 blankShow the coverage ratio to the same audience as utilisation. An idle lane costs tokens not spent; a misdirected lane costs review attention, run cost and a permanent tail
Discovery theatreArtefacts with sentiment but no observation (quotes, no instrumented behaviour); validated-intent lead time lengthening without better admitted intentGive field 3 its shape: what was observed, how many users, over what period, and what would have falsified it
Solution-shaped requestField 1 names a system; field 3 cites the sponsor’s seniority rather than a measurementTurn the request back into a problem with a cause chain and evidence before solution shaping starts; treat this as the first increment of intake, not a gate
Selection board that does not buildBet cards never amended after admission; builders cannot challenge selectionGive the wrangler and adoption lead a standing right to bring a kill clause forward
LandfillRising count of production systems with an empty field 9, or whose named humans have left. Book 2 calls this the leading indicator, preceding the pain by about a yearRefuse promotion without a run retirement recipe; hold a scheduled kill review each quarter
Tollbooth boundaryPromotion elapsed time exceeds the time a business unit takes to build outside; shadow tools discovered laterCut the crossing to days by supplying controls by inheritance first; only then tighten the supply side. Closing the default environment before a fast crossing exists moves shadow AI somewhere with less telemetry

Acceptance checks

  • A one-page selection rule exists, names its exceptions and states the evidence a problem must arrive with.
  • A defect Pareto runs at least weekly per product family, and each accepted cause chain carries an evidence pointer per link and a named acceptor.
  • Every item in the agent lane has a bet card with fields 1 to 5 and 8 complete; no admitted card has an empty field 3.
  • Every promoted product has a signed kill clause dated before its first construction commit and a field 9 run date earlier than its promotion date.
  • At least one real product has been retired end to end by recipe, and the elapsed time is recorded.
  • Promotion elapsed time is measured; the median is under two weeks.
  • The count of production systems with an empty field 9, or with a departed named human, is reported each quarter and is not rising.
  • Validated-intent lead time and the intent coverage ratio are on the scorecard; there is no agent-lane utilisation target.
  • The quarterly kill review has minutes showing clauses fired, extended with reasons, or brought forward.

What this page does not decide

It does not choose the product’s architecture; once a card is admitted, choose the AI product shape and classify the risk tier. It does not set the buy-versus-build decision for low-moat candidates; see build, configure, consume. Intake increments, triage and the capacity-bounded roadmap are on running the factory.

Source notes. The four rules, nine fields, two metrics and six failure modes are from Book 2 Chapter 21. The admission and promotion thresholds table, the retirement evidence column, the review agenda, the record layout and the acceptance checks are this page’s operationalisation of that chapter. Book 2 gives the composite lease tool as roughly 1,400 leases; Book 09 (Ch 10) describes a similar composite at roughly 1,500 (cited on application foundation). The figures are illustrative either way.

Source material

  • Book 2 (The AI Delivery Factory), Chapter 21, Product Development at Agent Speed (§2–§8: validated intent, selection, kill clause, promotion boundary, the discovery-paced portfolio, bet card)
  • Book 2 (The AI Delivery Factory), Appendix A, Glossary (bet card, promotion boundary)
  • Book 2 (The AI Delivery Factory), Chapter 5, Bills of Material and Recipes (the BOM as the enumeration that makes retirement possible)
  • Book 2 (The AI Delivery Factory), Chapter 6, The Standard Application Foundation (the foundation as the governed starting point and promotion boundary)

← All AI Delivery Factory guidance

On this page

  1. Why discovery now sets the pace
  2. The four rules
  3. Select from evidence, not from volume
  4. The bet card
  5. The kill clause
  6. The retirement recipe, run before promotion
  7. The promotion boundary
  8. Run the portfolio review
    1. Two numbers that make the claim measurable
  9. Scale the practice to the firm
  10. Failure modes and recovery
  11. Acceptance checks
  12. What this page does not decide

All pages in Design →