AI-Native Engineering - An Engineering Goal Closure Pilot Blueprint
The finding. As of July 13, 2026, vendors sell nearly every component of AI-native engineering — runtimes, memory, context graphs, governance, coding agents, analytics. No reviewed generally available product closes the loop: a durable cross-team goal with a predeclared evidence contract, builder-independent verification through production, and an outcome verdict against the original intent. This blueprint names that gap engineering goal closure, tests whether an incumbent vendor stack can fill it before building anything, and specifies the smallest thing worth owning if none can.
Freshness contract: The market landscape and availability statements in Section 5 were audited as of July 13, 2026. They are perishable. Recheck them before an executive, security, architecture, or procurement decision and regularly during active vendor selection.
1. Executive summary
Frontier AI has reduced the cost of implementation. The scarce resources in engineering are increasingly intent, context, coordination, verification, outcome judgment, and risk ownership.
Vendors now provide much of the substrate required for AI-native engineering: long-running agent runtimes, sandboxes, memory, enterprise context graphs, multi-agent orchestration, policy controls, evaluations, observability, agent identities, audit, and cost or ROI analytics.
What this audit did not find as a generally available native product is the exact engineering closure loop:
A durable cross-team engineering goal, a predeclared evidence contract, builder-independent verification spanning code through production, and a post-deployment outcome verdict traced to the original intent.
This blueprint calls that combination engineering goal closure.
The claim is deliberately narrow. OpenAI, Atlassian, ServiceNow, IBM, AWS, Anthropic, GitHub, Microsoft, Salesforce, Sierra, and engineering-metrics vendors already cover substantial parts of the design. Public documentation may also omit enterprise-only capabilities. The pilot must therefore begin with a falsifiable vendor proof-of-fit, not with the predetermined conclusion that a custom gateway is necessary.
The proposed strategy is:
- Define a company-owned, provider-neutral goal/evidence/outcome object model.
- Test the same real engineering closure workload against the closest vendor platforms.
- Rent every capability that passes the test.
- Build only the residual closure kernel and adapters that the test proves are missing.
- Measure value using achieved outcomes, safety, rework, review focus, and cost - not token volume, agent sessions, or PR count.
This is a recombination of shipped capabilities, not a request to invent a new foundation-model stack. A private trained model is not required for this pilot. The durable asset is the organization's permissioned context and evidence history, not model ownership.
Section 2 states the gap precisely; Section 6 defines the test that decides what, if anything, gets built.
2. The thesis, stated precisely
2.1 What the market already sells
The market already sells or openly documents:
- agent execution, memory, scheduling, identity, sandboxes, and multi-agent work;
- enterprise context and relationship graphs connecting goals, work, people, code, systems, and funding;
- agent governance, evaluation, observability, policy, audit, cost, and ROI tooling;
- structured specification and acceptance-criteria workflows;
- source-control-native coding agents and multi-vendor mission control;
- operational agents that observe systems and act on incidents;
- engineering-delivery telemetry associated with AI adoption.
2.2 The residual gap found in this audit
As of July 13, 2026, no reviewed generally available product was confirmed to natively unify all four of these engineering capabilities:
- Durable cross-team goal record - the original intent, accountable owner, constraints, decisions, work, evidence, deployment, observation, and verdict remain joined after merge.
- Predeclared evidence contract - the required proof and pass conditions exist before a builder begins.
- Builder-independent closure - a builder cannot mark its own evidence complete merely by producing an artifact or passing its own model-based review.
- Production outcome verdict - an observation window resolves the goal as achieved, missed, mixed, or invalidated against the original intent.
Several vendors cover large subsets. The defensible differentiation is the join and enforcement model, not every underlying component.
2.3 What would falsify the thesis
The thesis is falsified if a vendor can demonstrate, export, and govern the complete four-part loop on the pilot workload without an organization-owned closure kernel. If that happens, the correct decision is to buy the capability and keep only a portable export or adapter seam.
3. Goals and non-goals
Goals
- Prove one real intent -> goal + evidence contract -> execution -> independent verification -> deployment -> observation -> outcome verdict loop.
- Determine which parts should be bought, configured, adapted, or built using evidence rather than vendor assumptions.
- Preserve one coherent decision layer with temporary, bounded workers - one brain, many hands.
- Produce the organization's own baseline and pilot measurements: cycle time, review focus, rework, escaped defects, safety events, cost per goal, and cost per achieved outcome.
- Validate security, data governance, identity, audit, interrupt, and rollback at pilot scale.
- Keep goal, evidence, decision, deployment, observation, and verdict schemas provider-neutral and organization-agnostic.
- Exercise self-management safely: the platform records and assists with its own operations as a governed standing goal.
Non-goals
- Building another coding assistant.
- Replacing source control, issue tracking, CI, observability, or existing approval rules.
- Predetermining a winning model, runtime, or control-plane vendor.
- Training or fine-tuning a private model for the pilot.
- Removing humans from intent, judgment, accountability, contract authority, or irreversible decisions.
- Claiming causal productivity gains before a baseline and observation design exist.
- Building generic identity, sandbox, memory, routing, audit, or evaluation infrastructure that a selected vendor already satisfies.
4. Assumed starting footprint
This blueprint assumes a mid-to-large engineering organization with most of the following:
- AI coding tools available to engineers, with approved connectors to source control, issue tracking, documentation, and selected internal systems.
- An internal AI or platform owner able to approve tools, distribute skills or extensions, enforce access policy, and monitor spend.
- Standard repositories, CI, pull requests, approval rules, deployment systems, and observability.
- Permissioned identity and secrets management.
- A real non-sensitive repository and production-shaped goal suitable for a controlled pilot.
- Named engineering, platform, and risk owners willing to make decisions during the pilot.
The closer the existing footprint matches, the more the pilot becomes integration and policy work rather than a new platform rollout.
5. Market landscape - audited July 13, 2026
Availability labels are intentionally explicit: GA, public beta/preview, limited enterprise access, and announced/pre-GA are not interchangeable.
| Platform or project | Current relevant capability | What the pilot still must prove |
|---|---|---|
| OpenAI Frontier | Enterprise business context, durable institutional memory, production agent execution, agent identity and permissions, audit, evaluation and optimization, and third-party agent support. Its launch material described limited customer access; current access is sales-led rather than confidently public GA. | Whether Frontier can represent, enforce, export, and close the exact four-part engineering goal loop. Do not predetermine the answer. |
| Atlassian Strategy Collection + Teamwork Graph | Strongest reviewed overlap for goals, priorities, work, teams, systems, funding, costs, benefits, forecasts, blockers, and delivery health. Teamwork Graph connects more than 150 billion objects and relationships and can expose context to external agents. | Whether it supports a predeclared engineering evidence contract and mandatory independent code-to-production closure rather than strategic reporting alone. |
| ServiceNow AI Control Tower | Cross-system agent discovery, observation, governance, security, evaluation, kill controls, cost, and ROI. Some May 2026 enhancements were in Innovation Lab with GA expected in August 2026. | Whether currently available components can carry the engineering goal/evidence/outcome object and whether pre-GA capabilities satisfy pilot timing and risk. |
| IBM watsonx Orchestrate + IBM Bob | Agent scheduling, catalogs, policy, audit, observability, evaluation, cost/outcome optimization, multi-agent software delivery, isolated subagents, and modernization workflows. | Whether IBM's control plane can join original engineering intent to a portable production verdict rather than only govern agents and workflows. |
| AWS Bedrock AgentCore | Multi-model harness, isolated runtime, long-running sessions, memory, identity, gateway/MCP, policy, observability, evaluation, and registry capabilities. | Whether a separate strategic goal graph and closure ledger are required and what the workload-based cost would be. |
| Anthropic Managed Agents | Public-beta long-running sessions, memory, schedules, multi-agent work, webhooks, hosted or self-hosted execution, and Outcomes rubric grading in a separate context. | Whether Outcomes can consume the evidence contract; whether independent context is sufficient for the risk tier; and whether beta, retention, control-plane data flow, ZDR, and HIPAA limitations are acceptable. Anthropic's retention documentation says Managed Agents are not ZDR-eligible or HIPAA-ready. |
| GitHub Agent HQ | Multi-vendor coding-agent mission control, repository-native work, PR flow, enterprise controls, sessions, and audit. The control plane reached GA while third-party agents remained public preview in the reviewed material. | Whether repository work can remain attached to a durable goal and outcome record after merge and production observation. |
| Microsoft Azure SRE Agent | Persistent operational knowledge, background analysis, scheduled workflows, subagents, approval/autonomous modes, managed identity, isolated execution, and incident action. | Whether its operational machinery can be configured to answer “did the intended outcome occur?” without overstating incident automation as product-outcome causality. |
| Google Managed Agents + Gemini Enterprise Agent Platform | Preview managed execution with ephemeral environments and resumable sessions, plus a broader platform for building, governing, and optimizing enterprise agents. | Whether Google supplies or can be configured to supply the durable engineering evidence and production-verdict object. |
| Salesforce Agentforce | Agent observability, command-center operations, enterprise context, customer-agent performance, and outcome-oriented domain metrics. | Transferability from customer workflows to general engineering closure. |
| Sierra Pinecone | Strong internal existence proof: one employee-facing agent over Claude Code and Codex, persistent work, a 37-system MCP gateway, inherited permissions, per-call policy, audit, systems of record underneath, 75,000+ sessions, 600+ people, and 70% of PRs opened through it. | Sierra explicitly says it does not yet have a good general way to measure the outcomes it actually cares about. This is evidence for the front half and the unresolved measurement gap, not a purchasable general closure product. |
| GitHub Spec Kit and SDD tools such as Kiro, BMAD, and Tessl | Portable structured intent, requirements, acceptance criteria, planning, tasks, implementation, and some task-level verification. | Extending the artifact beyond implementation so it remains authoritative through deployment and observed outcome. |
| DX, Jellyfish, LinearB, and Swarmia | AI-adoption and delivery analytics across cycle time, review flow, throughput, cost, and impact. These are separate products and datasets, not one “700-company benchmark.” | Joining delivery telemetry to the organization's specific original intent, constraints, evidence history, and final verdict. |
| NVIDIA and Cadence ChipStack | NVIDIA reports more than 40x faster verification cycles in a bounded semiconductor workflow using autonomous agents, simulation, and formal verification. | Generalizability. This is vendor-reported, domain-specific evidence that autonomous verification can work, not a general engineering productivity multiplier. |
Evidence behind the urgency
- GitLab's 2026 AI Accountability Report, conducted by Harris Poll across 1,528 respondents in six countries, reports that 85% agreed AI shifted the bottleneck from writing code to review and validation. IBM cited this research; IBM did not originate it.
- Sierra distinguishes activity from outcome and states that it cannot yet measure the general outcomes it ultimately values, even after substantial internal adoption.
- DX reports a median PR-throughput improvement of about 8% across more than 400 companies, while Jellyfish reports much larger results for top adopters across a separate population. The variance is a reason to measure locally, not to select a preferred headline.
Landscape conclusion
The market invalidates the broad statement that no one sells “the layer above.” It supports a narrower, dated conclusion: the reviewed public products did not demonstrate the complete engineering goal-closure loop as one native generally available capability.
6. Phase 0: incumbent-default proof-of-fit
The first deliverable is not code. It is a controlled proof-of-fit using one real, non-sensitive, production-shaped engineering goal.
The vendors in Section 5 are not seven competitors selling the same product; they each sell a different partial slice of the substrate. No reviewed generally available product was confirmed in public material to natively provide the complete closure loop, so Phase 0 is not a bake-off to pick a winner. It answers a narrower question: can the platforms the organization already runs be configured to close the loop, and if not, exactly what is missing?
The incumbent-default rule
If a platform is already procured, deployed, and inside the security boundary, it wins ties against every vendor that would require new onboarding. For most organizations matching the Section 4 footprint, the incumbent agent runtime, incumbent tracker and context graph, and incumbent source-control platform's coding agents receive the first hands-on test. A non-incumbent advances to hands-on evaluation only when desk evidence indicates it can satisfy a material criterion the incumbent fails, or when security or strategic requirements justify the additional onboarding.
Two tracks, hard timebox
Track 1 - desk audit of alternatives (zero procurement). Score every non-incumbent Section 5 candidate against the acceptance test below using public documentation, trials, and sales-engineering conversations only. Record pass / partial / fail / unknown per criterion with evidence links. This is the receipts file: it converts "nobody does this" from a hunch into a documented, dated, falsifiable claim, at a small fraction of the cost of a full procurement cycle. No contracts, no data integration, no security review.
Track 2 - hands-on proof-of-fit on the incumbent stack (timeboxed at kickoff). Run the acceptance test for real on the incumbent platforms, concentrating on the criteria that documentation cannot answer - typically: whether the runtime's evaluation facility can consume a predeclared evidence contract, whether a goal record survives intact from tracker intent to production verdict, whether a builder can be structurally prevented from self-approving evidence, and whether the complete record exports without semantic loss. Any capability requiring vendor access that does not materialize inside the timebox is scored from desk evidence and the phase moves on; no vendor's sales calendar controls the pilot schedule.
Standing falsification clause
The thesis remains permanently open to disproof: any vendor, at any time, that can demonstrate, export, and govern the complete four-part closure loop on the pilot workload earns a formal replacement evaluation. If it passes the workload, security, cost, support, and exportability gates, the correct decision is to buy it and keep only the portable export seam. A one-time audit goes stale; this standing offer is what keeps the claim honest after Phase 0 ends, and it costs nothing to maintain.
One common acceptance test
Both tracks score against the same ten criteria - hands-on for the incumbent stack, desk evidence for the rest. Each candidate must demonstrate whether it can:
- Ingest one cross-team engineering intent and preserve it through production observation.
- Represent the evidence contract before any builder executes.
- Prevent a builder from self-approving every evidence row.
- Accept deterministic CI, security, benchmark, and state evidence.
- Accept an independent model/vendor or human judgment where deterministic proof is unavailable.
- Compile expected outcomes and guardrails into a defined observation window.
- Record
achieved,missed,mixed, orinvalidatedafter observation. - Connect model spend, external-service spend, and human work to the goal.
- Enforce least privilege, attribution, approval, interrupt, rollback, and audit.
- Export the complete record without losing semantics or provenance.
Decision rule
- Pass: rent the capability and retain a portable export seam.
- Partial pass: rent the working portion and implement only the missing adapter or rule.
- Fail: record the evidence and place the residual capability inside the closure kernel.
7. Architecture: portable closure over rented platforms
The architecture is a hypothesis until Phase 0 completes.
Human intent and accountable owner
|
v
Provider-neutral closure record
goal | constraints | evidence contract
decisions | work links | deployment
observation | verdict | provenance
|
v
Closure rules and policy seam
state machine | independence checks
approvals | budgets | export | audit join
|
+--------+---------+-------------------+
| | |
v v v
Rented agent Deterministic Systems of record
control plane evidence systems tracker | source control
and workers CI | scanners deployment | observability
multiple vendors benchmarks | SRE identity | vault
+------------------+-------------------+
|
v
Observed outcome verdict
Company-owned boundary
The organization should own the semantics and portable history of:
- goals and original intent;
- constraints and accountable owners;
- evidence contracts and results;
- decisions and approvals;
- deployments and observation windows;
- final verdicts, confidence, and known confounders;
- provenance and export.
Ownership does not necessarily mean a custom service. A vendor may store and enforce these objects if it passes exportability and control requirements.
Capabilities expected to be rented
Unless Phase 0 proves otherwise, rent:
- model inference and routing;
- agent execution, scheduling, and sandboxes;
- generic identity, permissions, and secrets integration;
- general memory and context retrieval;
- source-control-native coding workers;
- generic MCP or tool gateways;
- observability storage and agent traces;
- delivery analytics;
- commodity evaluation infrastructure.
Likely residual closure kernel
If no candidate passes end to end, the smallest plausible custom kernel is:
- Canonical goal/evidence/outcome ledger.
- State machine and closure rules.
- Evidence-producer interface and independence policy.
- Outcome-watch compiler.
- Adapters to the chosen control plane, tracker, source control, CI, deployment, and observability.
- Portable export and audit join.
This replaces the original assumption that an all-purpose “Agent Gateway” is automatically the only build.
8. Closure object model
The record extends specification-driven development, but it does not die at merge.
goal:
schema_version: 1
id: G-0001
intent: "<observable outcome that matters>"
owner: "<human accountable for intent>"
constraints: ["<hard boundary>"]
acceptance_criteria: ["<artifact or behavior criterion>"]
risk_tier: low # allowed: low, medium, high
evidence_contract:
- id: E-01
question: "<what must be proven?>"
threshold: "<pass condition declared before execution>"
producer_class: deterministic # or independent_model, stateful, human
independence_required: true
status: pending # allowed: pending, pass, fail, waived
result: null
provenance: null
approvals:
- action: "<controlled action>"
approver_role: "<accountable role>"
status: pending # allowed: pending, approved, rejected
budget:
expected_usd: null
ceiling_usd: null
actual_usd: 0
escalation: "owner approval with evidence"
outcome_contract:
expected: "<production outcome>"
guardrails: ["<must not regress>"]
observation_window: "<duration and start condition>"
confidence_method: "<how uncertainty will be handled>"
links:
tracker: []
prs: []
decisions: []
deployments: []
incidents: []
observation:
observed: null
guardrail_results: []
confounders: []
confidence: null
verdict: pending # allowed: pending, achieved, missed, mixed, invalidated
status: proposed # allowed: proposed, approved, active, verifying,
# awaiting_approval, deployed, observing, closed, aborted
Record rules
- Every state change is attributed to a human or governed service identity.
- Evidence checks are named contracts, not hardcoded vendor integrations.
- Required evidence exists before execution; retrospective evidence additions require an attributed decision.
- A waived check requires a named human, reason, and expiry.
- The goal cannot close while required evidence is pending or failed.
- The verdict includes confidence and confounders; an outcome association is not automatically proof of causality.
- The complete record must be exportable in a documented, provider-neutral format.
9. Verification and evidence independence
The closure rule
The builder cannot be the sole judge of its own work. The closure layer bookkeeps and enforces; it does not manufacture a passing verdict.
Evidence producers
- Deterministic: CI, tests, type checks, security scanners, policy checks, benchmarks, schema validation, deployment state, and measured service state.
- Independent model: a separately invoked evaluator with isolated context; for consequential claims, prefer a different vendor or model family from the builder.
- Stateful: compare the expected world state with staging or production state across repeated checks.
- Human: architecture, product judgment, ethics, legal, risk, ambiguous tradeoffs, and irreversible approval.
An isolated grader context is useful but is not automatically independent enough for medium- or high-risk closure. Risk policy determines when a different vendor/model, deterministic check, or human judgment is required.
Evidence hierarchy
Prefer evidence in this order when the question permits it:
- Deterministic system evidence.
- Repeated state evidence.
- Independent cross-vendor/model evaluation.
- Isolated same-vendor model evaluation.
- Builder-produced narrative, used only as supporting context.
Human authority sits outside the hierarchy: it is mandatory wherever policy assigns accountability to a person.
10. Production outcome closure
Operational tooling usually asks, “Is the system broken?” Engineering goal closure adds, “Did the intended change occur without violating its constraints?”
Each approved outcome_contract compiles into:
- the target metric or observable state;
- guardrail metrics that must not regress;
- start condition and observation window;
- minimum data sufficiency;
- rollback or escalation conditions;
- known confounders and confidence method.
The standing service observes the window and prepares a verdict. Deterministic cases may close automatically when policy permits. Ambiguous or consequential cases route the evidence to the accountable human.
An incident can be operationally resolved while the goal remains missed. A deployment can be healthy while its intended outcome remains unachieved. The durable distinction between those states is the blueprint's most differentiated capability.
11. Policy, autonomy, memory, and self-management
Goal-scoped policy
Every consequential tool call should carry a goal ID, governed identity, requested action, risk tier, and budget context. Authorization becomes explainable: a worker may act on repository X because it is executing approved goal G-12 within declared constraints - not because it possesses standing broad access.
Evidence-earned autonomy
Each action class begins at propose-and-approve. A promotion to execute-with-rollback requires:
- a declared evidence threshold;
- a sufficient history of approved actions;
- no unresolved policy violation or missed safety outcome;
- a proposed policy change;
- approval from the named risk owner;
- a tested rollback and automatic demotion condition.
Autonomy is earned per action class and risk context, not granted globally to an agent.
Durable memory
Recommended scopes are personal, team, product, and company. Shared memory accepts published decisions or goal-linked evidence with provenance, permissions, and review dates. Secrets remain in a vault. The authoritative closure record remains company-controlled and exportable even when general memory is rented.
Governed self-management
Standing goal SG-0 makes the platform its own first customer. It may monitor health, cost, errors, stale memory, vendor API changes, and configuration drift; prepare maintenance changes; and report on a regular cadence. It does not replace accountable platform ownership. Privileged changes, vendor contracts, risk policy, and security exceptions remain human-authorized.
12. Operating model
Principles
- Humans own intent, constraints, acceptable risk, and irreversible decisions.
- A durable goal becomes the primary unit of coordination; tickets and PRs remain systems-of-record artifacts underneath it.
- One coherent decision layer dispatches temporary bounded workers.
- Memory lives outside any individual model session.
- Verification outranks implementation.
- Humans review evidence, uncertainty, architecture, and tradeoffs rather than predictable trivia.
- Work is complete when policy accepts the production verdict, not merely when code merges.
The engineer's day
- Start: The engineer opens an approved AI coding surface into the shared goal state, current constraints, decisions, work links, and pending evidence.
- Build: The engineer and agent perform the work while bounded workers investigate, benchmark, test, or review in isolated environments.
- Coordinate: New decisions and collisions attach once to the goal and become available to other permissioned sessions.
- Review: The PR arrives with its evidence status, gaps, uncertainty, and goal context. Existing approval rules remain in force.
- Deploy: Deployment events attach to the goal. Rollback and escalation follow existing controls plus goal-specific guardrails.
- Observe: The system watches the declared outcome window and prepares the verdict.
- Close: The accountable owner accepts, rejects, or qualifies the verdict; lessons remain attached to the durable record.
Multiple teams on one product
For two or more engineering teams sharing one repository and product, goals span teams while execution remains permission- and ownership-bound. Managers do not operate separate competing “main agents.” Each person has a permission-bound session over the same authoritative goal state. Temporary workers are scoped to a goal, repository area, evidence task, budget, and lifetime. Team-specific conventions live in team memory; product decisions and closure evidence live in product scope.
Pilot roles
| Role | Accountability |
|---|---|
| Goal owner | Intent, constraints, tradeoffs, and acceptance of the final verdict |
| Platform intent owner | SG-0, vendor integration, operational escalation, and continuity |
| Risk owner | Risk tiers, approval thresholds, autonomy promotion, and emergency interrupt |
| Approvers | Existing ship/no-ship authority applied to the goal's evidence |
| Internal platform team | Distribution, identity, policy integration, observability, and co-ownership |
| Vendor owners | Proof-of-fit evidence, terms, availability, support, and exit/export path |
13. Value case - hypotheses to test
The pilot tests value mechanisms; it does not promise percentages in advance.
Candidate value mechanisms
- Higher context floor: shared goals and decisions reduce rediscovery, handoff loss, and insufficient-context review failures.
- Less coordination labor: agents maintain tracker links, evidence status, and briefs while humans own decisions.
- Shorter dead time: bounded unattended investigation, tests, and observation continue between human work periods.
- Earlier defect discovery: independent evidence arrives before approval instead of relying entirely on line review.
- Better review focus: humans spend more attention on architecture, risk, uncertainty, and tradeoffs.
- Durable outcome knowledge: the organization learns which engineering goals achieved their intended effect and under what constraints.
- Tighter governance: actions become attributable to a goal, identity, policy decision, and evidence trail.
- Portable institutional memory: decisions and evidence survive sessions, laptops, PTO, and personnel changes.
Claims deliberately removed
This version does not assert:
- 20-30% effective capacity recovery;
- zero ticket labor or zero dead time;
- a specific monthly pilot cost;
- a guaranteed one-quarter delivery;
- outcome knowledge unavailable from every vendor at any price.
Those may become measured results, bounded planning scenarios, or falsified assumptions. They are not current evidence.
14. Measurement and cost
Baseline before build
Measure the pilot team's current:
- intent-to-production and intent-to-observed-outcome cycle time;
- review latency and review time by activity type;
- rework, escaped defects, rollback, and failed change rate;
- interruptions, handoff loss, and coordination effort;
- deployment-to-outcome observation time;
- existing AI, tooling, and human cost where measurable.
Delivery-analytics products may provide part of the baseline. Their data must be joined to the pilot's declared intent and verdict rather than treated as the outcome by itself.
Pilot success criteria
| Dimension | Minimum acceptable evidence |
|---|---|
| Loop completeness | At least one real goal reaches a production verdict after its declared observation window |
| Evidence integrity | Every required shipped evidence row has an allowed independent producer or an attributed human waiver |
| Review focus | A measurable shift from predictable line-level checks toward evidence, architecture, risk, and tradeoffs |
| Cycle time | No material regression versus baseline; improvement is upside, not the only gate |
| Safety | No unattributed action; no unhandled policy violation; interrupt and rollback exercised successfully |
| Self-management | SG-0 handles at least one real operational event under policy and human accountability |
| Portability | Complete goal/evidence/outcome record exported and reconstructed without semantic loss |
| Cost | Actual cost per goal and per achieved outcome recorded with vendor and human components |
Explicit non-success metrics
Token volume, session count, tool calls, lines of code, and PRs opened are adoption and diagnostic signals. They are not proof that an outcome improved.
Workload-based cost model
Estimate cost only after vendor selection and workload definition:
pilot cost = platform commitments
+ model inference
+ search and external tools
+ sandbox/runtime time
+ observability and storage
+ implementation and integration labor
+ security/procurement effort
+ operating and review labor
Budget per goal, enforce ceilings, cap retries and worker spawning, and require evidence-backed escalation. Idle persistent state may be inexpensive, but vendor minimums and human operating cost prevent a responsible universal “idle equals zero” claim.
15. Security, governance, and procurement
Pre-build gates
- Select one non-sensitive repository and have the risk owner confirm its suitability.
- Name the goal, platform, and risk owners.
- Review each candidate's availability state, data flows, retention, training-use terms, regional processing, subprocessor posture, incident terms, and deletion/export behavior.
- Document whether tool inputs and outputs cross a vendor control plane even when execution is self-hosted.
- Approve the risk-tier, action-class, approval, autonomy, interrupt, and rollback matrix.
- Run the Phase 0 proof-of-fit without a predetermined vendor conclusion.
- Confirm the complete closure record can be exported before accepting any vendor as its authoritative store.
Always-on controls
- SSO, SCIM or equivalent group control, least privilege, and short-lived scoped credentials.
- Goal-scoped policy on consequential tool calls.
- Read-only defaults and explicit write boundaries.
- Human approval for high-risk actions until evidence supports a narrower promotion.
- Isolated execution and repository/network boundaries.
- Vault-only secrets.
- Full attributed audit and provenance.
- Prompt-injection defenses for low-trust content.
- Tested emergency interrupt, rollback, and recovery.
- Measured false-positive and false-negative behavior for approval and risk gates.
- Provider-neutral schemas and export to reduce vendor, policy, and availability concentration risk.
The system never operates as an unbounded global administrator.
16. Pilot phases
| Phase | Deliverable | Exit condition |
|---|---|---|
| 0 - Baseline and proof-of-fit | Baseline, pilot goal, risk gates, incumbent-stack proof-of-fit, desk-audit receipts, scored decision record | Selected rented stack - incumbent by default, with any exception documented - and confirmed residual gap |
| 1 - Portable closure record | Goal/evidence/outcome schema, state machine, export, identity attribution | SG-0 and the pilot goal can be represented and reconstructed |
| 2 - Execution integration | Selected runtime/control plane connected to the repository and tracker | One bounded task executes with full attribution and existing approvals |
| 3 - Independent evidence | Deterministic, model/human, and at least one stateful producer | A real change is gated by the declared evidence contract |
| 4 - Production closure | Deployment, observation window, guardrails, and final verdict | One real goal closes as achieved, missed, mixed, or invalidated |
| 5 - Decision report | Costs, outcomes, incidents, portability, autonomy history, and recommendation | Stakeholders decide scale, adjust, buy differently, or stop |
The pilot's duration is deliberately not fixed in advance. Procurement, security review, vendor access, data integration, and the chosen observation window may dominate the schedule. Phase 0 should produce the first credible timeline, and the timeline is owned as a decision, not assumed.
17. Open decisions, ordered by blocking power
- Internal platform owner and distribution path.
- Non-sensitive pilot repository and real cross-team goal.
- Named risk owner.
- Baseline data source and measurement design.
- Criteria for escalating a non-incumbent from desk audit to hands-on evaluation.
- Proof-of-fit scoring and required export format.
- Required independence level by risk tier.
- Production outcome and observation window for the pilot goal.
- Internal-only versus publishable final report.
18. Beyond the pilot
Within engineering
Different engineering domains get distinct evidence vocabularies and tool surfaces over the same closure semantics. Product services, ML systems, infrastructure, data systems, mobile, security, and hardware should not be forced into identical checks.
Across the organization
The schema is organization-agnostic: intent, constraints, evidence, accountable decisions, observation, and verdict can extend to sales, finance, operations, legal, or support. Expansion is justified only after the engineering pilot proves the closure model and each domain defines its own evidence and risk vocabulary.
Strategic position
Models, agent runtimes, control planes, search providers, and coding workers will keep improving and competing. The durable company-owned asset is the permissioned history connecting intent, decisions, evidence, deployments, outcomes, and learning.
The moat is not owning a model. It is building trustworthy organizational memory about what was intended, what was done, what proved it, what happened, and who accepted the result.
This pilot preserves the full ambition: one coherent engineering intelligence, permission-bound personal sessions, shared durable goal state, temporary bounded workers, builder-independent verification, systems of record underneath, accountable human governance, evidence-earned autonomy, production outcome closure, and a platform that helps operate itself. It begins at the smallest production-shaped scale capable of proving whether the residual engineering goal-closure layer is valuable enough to own.