1. Introduction
The subject of this paper is a regulated financial institution — a bank, an insurer or an asset manager — in the EU, the UK or Switzerland that is building an AI factory: the organisational and technical capability that delivers AI applications repeatedly from shared, governed parts. The reader I have in mind runs technology for such a firm and is being told, by vendors and by their own engineers, that agents have changed the economics of software.
For every capability the factory needs there are three ways to source it.
- Build as code. An agent writes it: a web form, a workflow engine, a document store.
- Configure a platform. The capability is assembled on a low-code or integration platform: an app builder, a workflow service, a ticketing or collaboration suite.
- Consume as a foundation. The capability is bought or subscribed to and treated as part of the factory floor. It is used through its interfaces and never re-implemented: identity, payments, electronic signature.
The business case for an AI factory often assumes, without saying so, that because agents make code cheap the answer is the first option. The thesis of this paper is that this assumption does not survive the evidence of 2025 and 2026. What the evidence supports is narrower. Agent-driven, code-first delivery has measured wins in one category, the rewrite and migration of legacy systems. It has a cost structure that applies in every category: review, rework and a stability tax. And the choice between building and configuring is itself dissolving, because platform vendors are turning their platforms into governed hosting layers for code that agents write.
What is evidence and what is argument. Section 2 describes how the evidence was gathered and checked, and what kind of checking it was. Sections 3 to 6 report published evidence, with my reading marked as such. Section 7 is argument. Section 8 lists the claims that failed verification. Section 9 is a design proposal, the four-gate framework, and has not been tested in a controlled way. Section 10 applies the same logic to model sourcing and is mostly my judgement, with the few facts it relies on cited. Section 11 states what the paper does not establish.
A note on two names. “DORA” in this paper means Regulation (EU) 2022/2554, the Digital Operational Resilience Act. The Google-owned DevOps Research and Assessment body of work, which shares the acronym, is called “the DORA research programme” throughout.
Terms. An agent here is an AI system that writes or changes software with some autonomy, under a named human owner who is accountable for what it does. A machine-operable surface is a way of operating a product without its graphical interface: an API, a command line or a protocol endpoint. A critical or important function is a term defined in DORA, and I use it only in that sense.
2. Method, and what kind of verification this is
2.1 The first pass, August 2026
The material behind this paper was gathered on 2 August 2026 by an AI research workflow. It ran five search angles, fetched 22 sources and extracted 106 claims. Twenty-five of those claims were then put to panels of three AI agents each, with the instruction to try to refute them. A claim passed on a majority vote. Eighteen passed and seven failed.
That procedure is useful and it is not peer review. No human expert read the sources at that stage. The panels were instances of language models reading the same pages the extraction had read; they can share a misreading, and they can vote down a claim that is true. Section 2.2 reports cases of both.
2.2 The second pass, 10 November 2026
The first pass itself concluded that a citation in this field has a half-life measured in months. By November its numbers were three months old. For this version an AI research agent opened the primary source for every figure that remains in the paper and re-dated it. Where a page could not be opened, the claim was cut, restated as my judgement, or marked. That is also not peer review, and the paper has not yet had a legal or practitioner review.
The second pass changed the following.
- Two of the seven refutations were wrong. The panels had rejected a claim that a retailer cancelled a vendor contract after rebuilding the product with a general-purpose model, on the ground that it could not be found in the source. It is in the source, on page 102 [23]. They had also rejected, as superseded, the finding that AI adoption is associated with higher individual productivity. Google’s own summary of the 2025 report still says so [8]. Section 8 gives the detail.
- Three “confirmed” items needed correction. A sample size attributed to one year’s survey is a ten-year cumulative figure [6]. A follow-up trial was reported with one of its two estimates missing [2]. And two of the three measured figures attributed to a June 2026 vendor press release are not in that release [26].
- Several items were cut because no primary source could be opened: an analyst firm’s forecast of project cancellations, a named bank’s in-house code-modernisation tool known only through press coverage, a set of practitioner-blog statistics on low-code sprawl and integration cost, and a reference to government guidance on agent authentication whose text could not be read.
I draw one conclusion from this for the reader’s own use of AI-assisted research. A vote among agents is a filter. It removes some errors and adds a few of its own, and it does not replace opening the source.
3. Evidence: what agent-written code costs
3.1 The one randomised trial, and the perception gap
The best-controlled evidence on AI-assisted coding is still METR’s randomised trial. Sixteen experienced open-source developers worked on 246 real issues in repositories they knew, with each issue randomly assigned to allow or disallow AI tools, primarily Cursor Pro with Claude 3.5 and 3.7 Sonnet. With AI allowed they took 19% longer. Before the study they expected a 24% speed-up, and afterwards they still believed they had been 20% faster [1].
METR now displays a notice on that study: “These results are out of date.” Its follow-up of 24 February 2026, on the tools of late 2025, estimates a speed-up of −18% for developers returning from the first study (confidence interval −38% to +9%) and −4% for newly recruited developers (−15% to +9%). METR says the new data “gives us an unreliable signal” and that it believes developers are sped up more now than in early 2025 [2].
Two things follow, and both are my reading. First, a self-reported productivity gain is not admissible as an input to a return-on-investment model. The gap between what the developers believed and what was measured was about forty percentage points, in the wrong direction. Second, the current causal effect of coding agents on experienced developers is unmeasured. Any business case that states it as a number is stating an estimate.
3.2 A structural failure rate
On Scale AI’s SWE-Bench Pro public leaderboard, as opened on 10 November 2026, the top entry resolves 61.5% (±3.1) of 731 public tasks, and the next four entries resolve between 46.1% and 59.1% [3]. A task counts as resolved only if the patch makes failing tests pass without breaking tests that already passed. The benchmark’s problems are drawn from actively maintained repositories and are described by its authors as long-horizon tasks [16].
Roughly two in five such tasks therefore fail at the first attempt for the best published configuration. I read that as a cost of production. Human review and rework are inputs to code-first delivery, to be budgeted like any other input; they are not an exception path. The figure is from one benchmark and it moves: an earlier figure of about 23% for the same benchmark was already out of date in August (section 8).
3.3 A maintenance liability
GitClear analysed 211 million changed lines written between January 2020 and December 2024, from repositories it describes as owned by Google, Microsoft, Meta and enterprise companies. It reports copy-pasted lines rising from 8.3% to 12.3% of changed lines between 2021 and 2024, lines associated with refactoring falling from 25% to under 10%, and 2024 as the first year in which copy-pasted lines exceeded moved lines [4]. The report’s headline, “4x Growth in Code Clones”, cannot be derived from those two percentages and I do not rely on it.
This is one vendor’s data set, the vendor sells a code-metrics product, and the data cannot attribute an individual line to an AI tool. It is a correlation over a period in which AI assistants spread. With those limits stated, it is the best published measure of a plausible mechanism: code that is cheap to write is duplicated more and consolidated less, and the cost arrives later.
3.4 The stability tax
The DORA research programme’s 2024 report found AI adoption associated with worse software delivery performance on both of its measures. Google’s announcement gives the estimates: as AI adoption increased by 25%, delivery throughput fell by an estimated 1.5% and delivery stability by an estimated 7.2% [5]. The same report found that AI adoption “significantly increases individual productivity, flow, and job satisfaction” [6]. The figure of 39,000 respondents that is sometimes attached to the 2024 survey is the programme’s own cumulative count across ten years of reports [6]; I could not open the 2024 report’s methodology section and give no sample size for that year.
The 2025 report, published as State of AI-assisted Software Development, draws on survey responses from nearly 5,000 technology professionals. Its authors write: “Unlike last year, we observe a positive relationship between AI adoption on both software delivery throughput and product performance. However, AI adoption does continue to have a negative relationship with software delivery stability” [7]. On the individual level, the announcement reports that more than 80% of respondents believe AI has increased their productivity [7], and Google’s summary of the report states that “AI is boosting individual performance” [8].
So the accurate statement, as of November 2026, has three parts. The negative throughput finding belongs to 2024 and was reversed in 2025. The negative stability finding appears in both years. The positive individual finding also appears in both years, and it is a self-reported belief, which section 3.1 gives reason to discount. All of these are survey associations, not controlled effects.
3.5 Net
Agent-written code is not free. Its cost is tokens, plus review capacity, plus a rework and stability penalty, plus a maintenance liability that compounds. A comparison with a configured platform that leaves those terms out will favour code for the wrong reason.
4. Evidence: the build-or-configure choice is dissolving
4.1 The platform as a governance host
The question “should an agent write this form in code, or should someone configure it on the low-code platform?” assumes that code and platform are alternatives. Microsoft, whose Power Platform is the example I use for the configure option, is removing that assumption from the platform side.
In its release plan for October 2025 to March 2026, Microsoft describes Power Apps as built with “a team of agents” and lists “Enterprise-ready Vibe coding” as a headline capability [10]. The accompanying post by the responsible corporate vice-president says the agents outline user stories and requirements, propose a data model and can “code a full-stack application in minutes” [9]. The speed claim is the vendor’s and is unmeasured. What matters is the product direction: the output is code.
On 5 February 2026 Microsoft announced that code apps in Power Apps were generally available [11]. A code app is a web application written in a code-first development environment, in a framework such as React or Vue, and then published to the platform. According to the documentation, it gets Microsoft Entra authentication and authorisation, access to the platform’s connectors (the page says more than 1,500 as of its September 2026 revision; the February announcement said more than 1,400), and “adherence to your organization’s Managed Platform policies”, which the page lists as sharing limits, Conditional Access and data-loss-prevention policy [12].
My reading is that the platform is being repositioned from an alternative to code into a host for code. The decision for the firm then changes. It is less “code or platform” and more “which governance layer will host the code the agents write”: the platform’s inherited identity, data-loss-prevention and connector controls, or a landing zone the firm assembles itself.
Microsoft’s own documentation gives reasons for caution. The same page lists, as current limitations, that code apps do not support the platform’s Git integration and that their compiled assets are served from a publicly accessible endpoint that does not support restriction by IP address. End users need a premium licence or pay-as-you-go billing [12]. A governance host whose artefacts sit outside its own source-control integration has not finished becoming one.
4.2 What cannot be claimed about agents and graphical tools
It is tempting to argue that agents are structurally bad at products configured through a graphical interface, and that this settles the matter in favour of code. The evidence that used to support that argument has expired. The OSWorld benchmark paper of 2024 reported that humans completed 72.36% of its computer-use tasks and the best model 12.24% [13]. By September 2025 one model vendor was reporting 61.4% on the same benchmark, up from 42.2% four months earlier [14]. That is a vendor-reported figure and I cite it only to show the direction and the speed.
The position I can defend is this. As of November 2026 I have found no independent measurement of agents building or maintaining applications on low-code platforms, in either direction. What exists is vendors making the question moot by making their platforms prompt-driven and addressable through interfaces. The useful question about a platform is therefore not whether an agent can click through its designer. It is whether the platform exposes a machine-operable surface, produces artefacts that can be compared line by line, and emits evidence of what was done. A firm can assess that for each platform today. The sibling paper Technology Management and Agent Readiness (Working Paper 04) makes the same test a procurement requirement.
4.3 The AWS and Google Cloud equivalents
The pattern is not specific to Microsoft, although the packaging is. On AWS the nearest integration equivalent is Step Functions, and on Google Cloud it is Workflows and Application Integration. All three have documented ceilings, and the documented remedies have the same shape as Microsoft’s (section 6.1). A Step Functions standard workflow fails when its execution history reaches 25,000 events, and a task’s input or output is limited to 256 KiB; AWS’s own guidance is to start a new execution, to run iterations as child workflows, and to pass a pointer to object storage in place of the payload [19]. Google’s Workflows caps a definition file at 128 KB and the memory for all variables in one execution at 512 KB [20]. Application Integration allows 100 tasks in an integration and tells the reader to split the work across several integrations beyond that [21].
Where they are better: these services are defined in text files from the start, which makes them easier to review, compare and regenerate than a workflow drawn in a designer. Where they are worse: in the documentation I opened I found no AWS or Google equivalent of Microsoft’s arrangement, in which professionally written code is published into the low-code platform and inherits its connector and data-loss-prevention policies. On those clouds the governance host for agent-written code is the cloud landing zone itself. AWS’s low-code builder, App Studio, is documented as available in two regions, US West (Oregon) and Europe (Ireland) [22], which is a narrower footprint than an EU, UK or Swiss institution may need. A firm on AWS or Google Cloud therefore reaches the same decision with fewer packaged controls and less to exit from.
5. Evidence: where code-first demonstrably wins
5.1 Legacy rewrite and migration
The strongest documented cases are in the Stanford Digital Economy Lab’s study of 51 deployments, dated April 2026 [23]. Two are relevant.
- An insurance firm “found that AI could rewrite legacy systems from scratch faster than refactoring them”. A project quoted at 5,000 hours with a team of seven, scheduled for completion in 2027, was finished in 600 hours with a team of three. By my arithmetic that is about an eighth of the quoted effort.
- A fintech with over 100 million customers needed to migrate millions of lines of legacy code. The traditional estimate was 18 months with over 1,000 engineers. With AI coding agents, business units began completing migrations in weeks, with engineers reviewing and merging changes in place of writing them.
The study does not name either firm. A case study published by the vendor of one coding agent describes a migration with the same parameters — an 18-month estimate, more than 1,000 engineers, a monolith of over six million lines — and names the customer as Nubank [24]. The match is my inference. That vendor page also claims efficiency multiples of eight to twelve times and cost savings of over twenty times; those are marketing figures and I do not rely on them.
The caveats belong with the numbers. The study states that it “relies primarily on self-reported data from interview participants” and has a selection bias towards successful deployments [23]. The comparison in each case is against an estimate for work that was never done the old way.
The scope matters more than the size. These are wins in the migration and rewrite of existing systems, where the old system serves as the specification and verification is a comparison of old behaviour with new. They are not evidence about a new capability with no predecessor, and they say nothing about code versus low-code for such a capability. That rewriting can beat refactoring when agents do the work is the clearest support the evidence gives to the code-first position, and it is bounded by category.
5.2 A licence count is not an outcome
A test for separating evidence from marketing can be shown on a live example. On 11 December 2025 Microsoft announced that four large IT services firms would each deploy more than 50,000 Copilot licences, over 200,000 in total. The release contains no measured productivity or return figure [25]. Figures appeared in a follow-up on 3 June 2026 covering three of the four firms: one reports “20–25% productivity improvements in research and content production tasks”, another “more than 250,000 FTE days saved every quarter”, alongside usage rates above 85%. No methodology, baseline or sample is given, and nothing is described as audited [26].
Six months separated the deployment announcement from the first numbers, and the numbers, when they came, were task-level and self-reported. The rule I take from it: date every such claim, treat a licence or usage count as an input, and ask for measured delivery outcomes with a stated baseline.
6. Platform ceilings and the rebuild trigger
6.1 Documented ceilings, documented workarounds
A configured platform comes with limits that the vendor documents. Azure Logic Apps is a convenient example because Microsoft’s reference page is precise. As of the page revision dated 30 October 2026, a workflow may contain 500 actions with a nesting depth of 8; a workflow on the multi-tenant Consumption plan may run 100,000 action executions in a rolling five-minute interval, or 300,000 in a high-throughput mode that is in preview; and a message may be 100 MB, with the note that “some connectors and APIs don’t support chunking or even the default limit” [17]. The Standard plan, which runs single-tenant, has no limit on action executions per interval [18]. Connectors add their own limits; the Azure Service Bus connector, for example, permits up to 6,000 calls per minute [18].
The interesting part is the remedies. Microsoft’s guidance for throttling is to refactor a workflow into a parent and child workflows, to set up multiple connections for a single action, and to partition the data by hand with take() and skip() expressions so that two loops each process half of it [18]. For the action and nesting limits, the documented extension is nested workflows [17]. Each remedy is a piece of architectural rework carried out inside the platform.
That leads to the pattern I want to name. A platform ceiling does not announce itself as a reason to rebuild. It announces itself as a workaround. Each workaround brings code-like complexity into the configured artefact while keeping the platform’s constraints, which is the worst of both options. The operating rule I propose is: the second structural workaround is the rebuild trigger. When a configured workflow needs its second structural workaround for throughput, payload or nesting limits, the capability has outgrown the platform, and the rebuild decision should be taken explicitly at that point. The alternative is that it is deferred one workaround at a time and never taken. The threshold of two is my judgement. I have no data set of rebuild decisions to calibrate it against.
6.2 The estate-governance cost
The configure option has its own unpriced cost. Low-code estates built by many hands grow apps and flows that nobody owns. The platform ships sharing limits, app quarantine and data-loss-prevention enforcement [12], which I read as the vendor’s acknowledgement of the problem. I have not found a primary source that quantifies the redundancy, and the practitioner figures in circulation are not citable. I therefore state it only as a term that must appear in the comparison: someone has to inventory, rationalise and retire what the platform makes easy to create. It is the platform-side counterpart of the code-clone liability in section 3.3.
6.3 Exit, under DORA
For the regulated reader there is a constraint the spreadsheet does not capture. As of November 2026, DORA requires financial entities to have exit strategies for ICT services that support critical or important functions (Article 28(8)), and requires the contracts for those services to include exit strategies with a mandatory adequate transition period (Article 30(3)(f)) [29].
The consequence is practical. A workflow configured into a proprietary platform, if it supports a critical or important function, brings that platform into the firm’s exit planning: what would replace it, how long the transition takes, and whether the configured logic can be extracted in a usable form. The same capability built as code on commodity compute has a much simpler exit. I read this as a regulator-imposed weight on the build side of the scale for components that support such functions. It is a weight and not a verdict; the firm’s legal and risk functions decide how heavy it is. UK and Swiss firms face outsourcing and operational-resilience rules with similar intent, which this paper does not analyse.
7. What never to build
This section is argument. I found no regulated institution that has published a policy on what it will not build, and I do not cite one.
The useful starting point is the old distinction between commodity and differentiating capabilities, applied at a finer grain than the product. A single “solution” usually contains both. Authentication is the standard example: the protocol flows, multi-factor mechanics, session handling and token rotation are commodity; the authorisation model, the user model and the tenancy rules are the firm’s own. Buying the first and owning the second avoids both re-implementing a security product and handing the firm’s control model to a vendor. The identity of agents, as distinct from people, is in my judgement still unsettled ground, which is a reason to consume it from a provider that is tracking the changes.
The market gives mixed signals, and the mix is the finding. A venture firm’s survey of 495 US enterprise AI decision-makers, published in December 2025, reports that 47% of AI solutions were built internally in 2024 and that 76% of AI use cases were purchased in 2025 [27]. A survey of 817 builders by a vendor of internal-tool software, published in February 2026, reports that 35% had already replaced at least one SaaS tool with a custom build and that 78% expected to build more in 2026 [28]. Both sources have an interest and both samples are narrow; the first is US-only. They point in opposite directions over the same period. The boundary is moving both ways at once, and a static answer will be wrong.
One case supports the build side and deserves a careful statement. The Stanford study quotes an executive at a retail company that had built a custom solution on top of a specialised vendor: “We ended up redeveloping the same code in Claude. And we cancelled the contract and we have our own proprietary solution” [23]. It is one anonymous, self-reported sentence, offered by the study’s authors as an illustration of total cost of ownership. It shows that such substitutions happen. It does not show how often, or what they cost to maintain.
The position I can defend is that the never-build list is defined by three properties held together, not by component type. The capability is (a) a commodity that confers no differentiation; (b) load-bearing for security or compliance evidence that auditors expect from an established third party; and (c) alive, meaning subject to continuous external change in regulation, protocols or attack surface that someone else already pays to track. Identity, payments, electronic signature, document retention and core collaboration meet all three. Consume them.
Workflow and approval logic fails the first test. An approval flow encodes the firm’s own control model, and in a regulated firm that is differentiation. This is why the workflow question never resolves to “just use the ticketing tool”. The engine is commodity. The control model is not. Buy or configure the engine, and own the definition of the control model as versioned, testable code that an agent can read.
8. The claims that failed
The first pass rejected seven claims. I keep the table because it is the most useful exhibit in the paper: these are statements a careful person would have written in good faith in 2025. Each was re-checked on 10 November 2026.
| # | Claim rejected in August 2026 | Panel vote | Reason given then | Status on 10 November 2026 |
|---|---|---|---|---|
| 1 | Agents succeed on only 12.24% of computer-use tasks against 72.36% for humans (OSWorld, 2024) | 0–3 | Outdated | Rejection holds. The figures are the 2024 paper’s [13]. A vendor reported 61.4% in September 2025 [14]. I could not read the current official leaderboard. |
| 2 | Agents are structurally disadvantaged on GUI-configured platforms | 0–3 | Rested on claim 1 | Rejection holds, for the same reason. No measurement on low-code platforms exists in either direction (section 4.2). |
| 3 | Frontier models resolve only about 23% on SWE-Bench Pro | 0–3 | Outdated | Rejection holds. The live leaderboard shows 61.5% (±3.1) at the top [3]. The current abstract of the benchmark paper gives no score [16]. |
| 4 | Systems strong on SWE-bench fail to generalise to visual and JavaScript tasks (12% on SWE-bench Multimodal) | 0–3 | Outdated snapshot | Rejection holds, with a qualification. The 12% is the October 2024 paper’s top result [15]. The project’s leaderboard data now shows a top entry of about 36%, with the latest submission dated November 2025 [15]. The gap has narrowed and has not closed. |
| 5 | AI adoption significantly increases individual productivity and flow | 0–3 | 2024 finding, superseded in 2025 | Rejection does not hold. The sentence is the 2024 report’s [6], and Google’s summary of the 2025 report still states that AI is boosting individual performance [8]. What 2025 reversed was the throughput finding. The claim is admissible if dated and described as self-reported. |
| 6 | A retailer cancelled a vendor contract after rebuilding the product with a general-purpose model | 1–2 | Not found in the source as described | Rejection does not hold. The passage is on page 102 of the study [23]. It supports less than the claim suggests (section 7). |
| 7 | An order-of-magnitude gap on SWE-bench Multimodal shows a general failure to transfer | 0–3 | Same source as claim 4 | Rejection holds. As claim 4. |
Five of the seven rejections (1, 2, 3, 4 and 7) were deaths by staleness. The statements were accurate when their sources were published and wrong within about a year. That is the lesson I would keep if I kept only one: in this field a figure without a date is not a fact, and a paper’s own numbers are on the same clock. Section 3.2 will be out of date before this version is revised.
The other two (5 and 6) are a different lesson. The panels of AI agents rejected two claims that the sources support. Neither error was dramatic. Both were found only by opening the source again.
9. A proposal: four gates, decided once
No published framework that I found maps technologies and capabilities to what agents can deliver on them. What follows is a proposal built on the evidence above.
9.1 The gates
Each candidate capability passes through four gates in order and stops at the first gate that decides it.
flowchart TD
C[Candidate capability] --> G1{Gate 1: Differentiation<br/>Commodity, evidence-bearing, alive?}
G1 -- Yes --> F[Consume as a foundation]
G1 -- No --> G2{Gate 2: Agent-operability<br/>Machine-operable, comparable, evidenced?}
G2 -- No --> X[Keep at arm's length<br/>or exclude from the factory]
G2 -- Yes --> G3{Gate 3: Ceilings and exit<br/>Limits clear of load? Exit realistic?}
G3 -- No --> B[Build as code]
G3 -- Yes --> G4{Gate 4: Honest return<br/>Full cost on both sides}
G4 --> B
G4 --> P[Configure the platform]
Gate 1 — Differentiation. Is this a commodity that confers no advantage, whose compliance evidence auditors expect from an established provider, and which changes continuously at someone else’s expense? If so, consume it and stop. The factory builds on it. Agents use it through its interfaces and never re-implement it.
Gate 2 — Agent-operability. Does the candidate platform expose a machine-operable surface, artefacts that can be compared line by line, integration with source control, and a record of each change that the firm can keep as evidence? This is the gate the agent era adds. A platform that can be configured only through a graphical interface is not impossible for agents; section 4.2 explains why that claim is dead. It is unrepeatable and hard to audit at the pace a factory works, which disqualifies it for a different reason. A platform that fails this gate may still be used at arm’s length. It should not become a shared part of the factory. Partial passes are real: a low-code platform with programmable interfaces and exportable definitions passes, while a hosting feature whose artefacts are not covered by the platform’s own source-control integration (section 4.1) passes with a noted gap.
Gate 3 — Ceilings and exit. Do the documented platform limits sit comfortably clear of the capability’s projected load, and, for a component that supports a critical or important function, is the exit realistic? I use a margin of ten times projected load as a working heuristic; it is a judgement and should be set by the firm. Performance and security enter here as specifics: documented throttle, payload and nesting limits against projected load, and the cost of the exit plan against the cost of running code on commodity compute. The rule from section 6.1 applies to capabilities already in service: the second structural workaround is the rebuild trigger.
Gate 4 — Honest return on investment. Only now compare building as code with configuring the platform, and only with the full cost on both sides.
| Side | Terms that must appear |
|---|---|
| Code | Tokens; human review capacity, sized for a first-attempt failure rate of the order of two in five on realistic tasks (section 3.2); a stability penalty (section 3.4); a maintenance liability (section 3.3). |
| Platform | Licences; estate governance (section 6.2); the risk and cost of ceiling workarounds (section 6.1); exit cost (section 6.3). |
| Both | Measured outcomes only, since perceived speed is not admissible (section 3.1). Category matters: migration and rewrite have documented large wins, and new capabilities do not (section 5.1). |
9.2 The catalogue
The business case for the factory does not depend on agents winning every comparison. It depends on the comparison being made once, by architects, with evidence, and then reused. Otherwise it is made again on every project by whoever is writing the prompt. In practice that means three things.
- A foundations layer of consumed capabilities — identity, document management, collaboration, payments, electronic signature — with the access patterns agents are to use specified in advance.
- A capability catalogue of pre-decided answers. Workflow and approval: engine configured on the firm’s platform, control model owned as versioned code. Forms and user interfaces: agent-written code hosted on a governance layer. Integrations: governed connectors first, custom code past documented ceilings.
- A scorecard for each entry: agent-operability, documented ceilings, exit cost and a date on which the entry is re-decided. Section 8 is the reason for the date. I would review entries every three to six months, prototype before deciding anything close to a boundary, and avoid multi-year commitments on capabilities whose economics are still moving.
The factory then does not choose between build, configure and consume at run time. It decides in the catalogue, records the evidence, dates the decision and decides again on a clock.
10. Model sourcing: an economics axis and a control axis
The model is the one capability every AI factory consumes. Nobody in the audience for this paper will train a frontier model, so Gate 1 is already answered. What remains is how to consume: rented from a hosted provider, rented with dedicated capacity, or run on infrastructure the firm controls. This section is mostly my judgement. Where a fact carries the argument it is cited.
10.1 Two questions that are usually merged
Every model-sourcing decision answers two independent questions.
- Economics. How cheaply and flexibly can the firm buy this capability? This is a question about cost and elasticity.
- Control. How much does the firm need to own the boundary, the availability and the continuity of this capability? This is a question about risk.
The expensive mistakes come from arguing one on the other’s ground. “We will self-host to save money” is a control decision argued as economics, and it is usually false. “Everything is sensitive, so nothing leaves” is an economics decision argued as control, and it is usually over-cautious and costly.
10.2 Economics favours renting
Three observations support this, and the conclusion is mine.
First, the price of a fixed level of capability has fallen fast. Epoch AI’s analysis of March 2025 found that the price of reaching a given benchmark threshold fell by between 9 and 900 times per year depending on the threshold, and cautioned that the fastest declines were recent and might not persist [30]. That analysis is twenty months old and uses list prices. I cite it for direction only.
Second, hosted providers offer discounts that a self-hosted deployment has to beat as well. One provider’s published price list, for example, charges a tenth of the base input price for reading cached prompt content on most of its models and gives a 50% discount for asynchronous batch processing [31].
Third, the number that governs unit cost is cost per token at the firm’s actual utilisation, and it is the number most self-hosting cases get wrong. A hosted provider spreads each accelerator across many tenants. A firm’s own accelerator serves the firm’s own uneven demand and costs the same whether busy or idle. The cloud providers’ own dedicated-capacity products make the point: Microsoft documents that its provisioned deployments are billed at an hourly rate “regardless of the number of tokens consumed” and recommends them for predictable traffic, with pay-per-token deployments remaining the better fit for low or highly variable volume [32].
My judgement is that self-hosting is almost never justified on token cost alone, and that a proposal to self-host for cost should be treated as unproven until a specific high, steady workload passes the test in section 10.5.
10.3 Control is the honest reason to own
Two things a metered interface cannot guarantee.
The boundary. For some data the question is whether it may leave the firm’s boundary at all, and the price per token is irrelevant. Whether that is so for a given data class is a question for the firm’s legal and data-protection functions, and the answer depends on what “the boundary” is taken to mean. It need not mean the firm’s own hardware. The Stanford study found that among the enterprises it interviewed, sovereignty “was addressed through contracts and data minimization, not through model architecture”, and that more than two thirds of those who discussed selection criteria cited capability, not cost, as the primary reason for their model choice [23]. Hosted providers also offer deployments that keep processing within a region or a geographic zone [32]. The discipline is to confine the premium for control to the data that needs it. My view, offered as an opinion, is that this slice is usually smaller than first assumed.
Continuity. A model the firm runs itself cannot be withdrawn, repriced or rate-limited by a supplier. That is worth paying for as insurance and as a negotiating position. It is a floor, kept modest and kept working. It is not a way to run the whole factory.
10.4 Four questions in order, and a routing matrix
For each class of workload, ask in order. The first two are control questions and override the others.
- Sensitivity. May this data lawfully and safely be processed outside the firm’s defined boundary? If not, it goes to a deployment inside the boundary, and cost is secondary.
- Continuity. If the primary provider were unavailable, throttled or sharply repriced tomorrow, would the failure of this workload be unacceptable? If so, a second provider or an in-boundary fallback is mandatory behind a routing layer.
- Difficulty. Does the task need the most capable model available, or will a smaller one pass the firm’s quality test? Rent the most capable model for the first kind. Use the cheapest tier that passes for the second.
- Volume and steadiness. Is the volume high, steady and predictable? Only then can self-hosting be considered on cost, subject to section 10.5.
| Routine task | Hard task | |
|---|---|---|
| Data may leave the boundary | Cheapest hosted tier that passes the quality test, with caching and batch pricing. Self-host only if the breakeven test clears. Governed by economics. | Most capable hosted model. Governed by economics. |
| Data may not leave the boundary | In-boundary deployment sized to this volume. Governed by control. | In-boundary deployment of the most capable model the boundary permits, accepting a capability gap or a higher price. Governed by control. |
A workload that is also continuity-critical adds a fallback in whichever cell it sits: a second provider where data may leave, a second in-boundary deployment where it may not. If a firm is self-hosting in the top row, it should be able to say which control it is buying. If it cannot, it is paying a control premium for an economics problem.
10.5 The breakeven test at real utilisation
Before approving self-hosting on cost grounds, ask for five numbers.
- The fully loaded hourly cost of the serving capacity: hardware or cloud instance, power, and a share of the people who operate it.
- Sustained throughput in tokens per second for the chosen model on that capacity.
- Expected utilisation: the share of hours in which the capacity is doing work. This is the figure most often overstated.
- Effective cost per token: hourly cost ÷ (throughput × utilisation × 3,600).
- The hosted price per token for equivalent capability, after caching and batch discounts.
If (4) is not comfortably below (5) at honest utilisation, with margin for the capability gap and the operating burden, the case fails on cost and must be made on control or dropped.
10.6 What to standardise
The provider and the self-hosted model will change within quarters. The lasting investment is the layer that hides them: a router that classifies each request by the four questions and dispatches it, with failover for continuity-critical classes. The Stanford study reaches the same conclusion from its cases: “The durable advantage is in the orchestration layer, not the foundation model” [23].
The facts behind these decisions move at two speeds. Prices, model versions, measured quality and the firm’s own utilisation move monthly and should be reviewed on a fixed cadence; I suggest quarterly. Data classifications, contractual terms and regulatory constraints move rarely and should be reviewed on an event: a new rule, a changed contract term, an outage. Each decision should carry the date it was last validated, for the reason section 8 gives.
11. Limitations
- The framework is a proposal. The four gates, the catalogue and the rebuild trigger have not been tested in a controlled comparison. The threshold of two workarounds and the ten-times load margin are judgements.
- No independent measurement exists of agents operating low-code platforms. Gate 2 uses a machine-operable surface as a proxy because nothing better is published.
- No end-to-end case of a rebuild forced by platform limits was found. The vendor documentation maps the ceilings. How organisations decide when they hit them is undocumented.
- No regulated institution’s “never build” policy was found. Section 7 is reasoning.
- The central productivity input is unmeasured. METR marks its 2025 result as out of date and its 2026 data as unreliable. The DORA research programme’s findings are survey associations, and its individual-level finding is self-reported. I read the 2025 report through its authors’ announcement and Google’s summary; the report body was not opened.
- Much of the evidence comes from interested parties. GitClear sells a metrics product. The benchmark leaderboard is run by a data vendor. The Stanford cases are self-reported, selected towards success and compared against estimates. The two market surveys are from a venture firm and a tools vendor; one is US-only. Every Microsoft, AWS and Google statement is the vendor’s own documentation or marketing.
- Microsoft is the worked example. The AWS and Google Cloud comparison in section 4.3 rests on four documentation pages and on what I did not find, which is weaker than what I found.
- The model-sourcing section is largely judgement. It contains no price comparison, deliberately, because any figure would be out of date on publication. The one price-trend source is from March 2025.
- Regulatory statements. The regulatory statements are a technologist’s reading as of November 2026 and are not legal advice. UK and Swiss rules are mentioned and not analysed.
- All quantitative claims decay. The paper’s own table of failed claims is the evidence. Every figure carries its date and should be re-checked before reuse.
- Verification. The first pass was made by panels of AI agents in August 2026. The source checks for this version were made by an AI research agent that opened the pages listed below on 10 November 2026. Neither is peer review. Items cut because their sources could not be opened are listed in section 2.2.
References
All web sources accessed 10 November 2026.
- METR, “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity”, 10 July 2025. https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/
- METR, “We are Changing our Developer Productivity Experiment Design”, 24 February 2026. https://metr.org/blog/2026-02-24-uplift-update/
- Scale AI, “SWE-Bench Pro” public leaderboard. https://labs.scale.com/leaderboard/swe_bench_pro_public
- GitClear, “AI Copilot Code Quality: 2025 Look Back at 12 Months of Data”. https://www.gitclear.com/ai_assistant_code_quality_2025_research
- Google Cloud Blog, “Announcing the 2024 DORA report”, 23 October 2024. https://cloud.google.com/blog/products/devops-sre/announcing-the-2024-dora-report
- DORA research programme, Accelerate State of DevOps Report 2024, report page, https://dora.dev/research/2024/dora-report/ ; publication record with abstract, Google Research, https://research.google/pubs/dora-accelerate-state-of-devops-2024-report/
- N. Harvey and D. DeBellis, “Announcing the 2025 DORA Report”, Google Cloud Blog, 23 September 2025. https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report
- Google, “How are developers using AI? Inside our 2025 DORA report”, The Keyword, 23 September 2025. https://blog.google/technology/developers/dora-report-2025/
- R. Cunningham, “Inside the new Power Apps: The future of app development”, Microsoft Power Platform Blog, 18 November 2025. https://www.microsoft.com/en-us/power-platform/blog/2025/11/18/inside-the-new-power-apps-the-future-of-app-development/
- Microsoft Learn, “Overview of Power Apps 2025 release wave 2” (archived by Microsoft as of October 2026). https://learn.microsoft.com/en-us/power-platform/release-plan/2025wave2/power-apps/
- J. Chodak, “Generally available: host and run code apps in Power Apps”, Microsoft Power Platform Blog, 5 February 2026, https://www.microsoft.com/en-us/power-platform/blog/power-apps/generally-available-host-and-run-code-apps-in-power-apps/ ; and “What’s new in Power Platform: February 2026 feature update”, 17 February 2026, https://www.microsoft.com/en-us/power-platform/blog/power-apps/whats-new-in-power-platform-february-2026-feature-update/
- Microsoft Learn, “Power Apps code apps: features and requirements” (page dated 23 September 2026). https://learn.microsoft.com/en-us/power-apps/developer/code-apps/overview
- OSWorld benchmark, project site carrying the 2024 paper’s abstract (reached by redirect from https://os-world.github.io/). https://osworld-v1.xlang.ai/
- Anthropic, announcement of Claude Sonnet 4.5, 29 September 2025. https://www.anthropic.com/news/claude-sonnet-4-5
- SWE-bench Multimodal benchmark paper, arXiv:2410.03859 (submitted 4 October 2024), https://arxiv.org/abs/2410.03859 ; current leaderboard data from the project’s public repository, https://github.com/SWE-bench/swe-bench.github.io (file
data/leaderboards.json). - X. Deng, J. Da, E. Pan et al., “SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?”, arXiv:2509.16941, version 2 (November 2025). https://arxiv.org/abs/2509.16941
- Microsoft Learn, “Limits and configuration reference guide — Azure Logic Apps” (page dated 30 October 2026). https://learn.microsoft.com/en-us/azure/logic-apps/logic-apps-limits-and-config
- Microsoft Learn, “Handle throttling problems or 429 Too Many Requests — Azure Logic Apps” (page dated 10 July 2026). https://learn.microsoft.com/en-us/azure/logic-apps/handle-throttling-problems-429-errors
- Amazon Web Services, AWS Step Functions Developer Guide, “Step Functions service quotas”, https://docs.aws.amazon.com/step-functions/latest/dg/service-quotas.html ; and “Best practices for Step Functions”, https://docs.aws.amazon.com/step-functions/latest/dg/sfn-best-practices.html
- Google Cloud, Workflows documentation, “Quotas and limits”. https://docs.cloud.google.com/workflows/quotas
- Google Cloud, Application Integration documentation, “Quotas and limits”. https://docs.cloud.google.com/application-integration/docs/quotas
- Amazon Web Services, “AWS App Studio FAQs”. https://aws.amazon.com/appstudio/faqs/
- E. Pereira, A. W. Graylin and E. Brynjolfsson, The Enterprise AI Playbook: Lessons from 51 Successful Deployments, Stanford Digital Economy Lab, April 2026, PDF pp. 6, 10, 63, 99 and 102. https://digitaleconomy.stanford.edu/app/uploads/2026/03/EnterpriseAIPlaybook_PereiraGraylinBrynjolfsson.pdf
- Cognition, “Nubank” customer case study (undated). https://devin.ai/customers/nubank
- Microsoft, “Cognizant, Infosys, TCS and Wipro emerge as Frontier Firms with Microsoft, deploying Copilot and agentic AI across the enterprise”, Source Asia, 11 December 2025. https://news.microsoft.com/source/asia/2025/12/11/cognizant-infosys-tcs-and-wipro-emerge-as-frontier-firms-with-microsoft-deploying-copilot-and-agentic-ai-across-the-enterprise/
- Microsoft, “Infosys, TCS and Wipro scale Microsoft 365 Copilot to over 300,000 employees”, Source Asia, 3 June 2026. https://news.microsoft.com/source/asia/2026/06/03/infosys-tcs-and-wipro-scale-microsoft-365-copilot-to-over-300000-employees
- Menlo Ventures, “2025: The State of Generative AI in the Enterprise”, 9 December 2025. https://menlovc.com/perspective/2025-the-state-of-generative-ai-in-the-enterprise/
- Retool, build-versus-buy report for 2026 (report page), published 17 February 2026. https://retool.com/blog/ai-build-vs-buy-report-2026
- Regulation (EU) 2022/2554 of the European Parliament and of the Council of 14 December 2022 on digital operational resilience for the financial sector (DORA), OJ L 333, 27.12.2022, p. 1. https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32022R2554 Articles 28(8) and 30(3)(f). Read in the Official Journal text on EUR-Lex, 10 November 2026.
- Epoch AI, “LLM inference prices have fallen rapidly but unequally across tasks”, Data Insight, 12 March 2025. https://epoch.ai/data-insights/llm-inference-price-trends
- Anthropic, Claude API documentation, “Pricing” (prompt-caching multipliers and Batch API discount). https://platform.claude.com/docs/en/about-claude/pricing
- Microsoft Learn, “Provisioned throughput for Foundry Models” (page dated 15 July 2026). https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/provisioned-throughput