Agent Engineering

Locate the dynamic decision

A workflow specifies operations and the rules governing transitions between them. Those rules can include branches, parallel work and repetition. A workflow that checks a result and follows a prescribed error branch is still a workflow; responding to observations does not by itself establish delegated discretion. [1]

Here, an agent is a system given discretion to select some subsequent work toward a goal using observations. The distinction concerns who determines the next operation: application-defined transitions or a decision process that can choose among approaches. This is a narrower engineering convention than the broader artificial-intelligence definition, which also includes simple prescribed controllers. [2][3]

Routine record conversion usually has known transitions: validate a record, convert supported fields, and reject invalid input. Investigating a failed job can require a less predictable sequence. A missing-file error may justify inspecting the producing job; an incompatible-field error may justify comparing schemas. These examples locate the design choice. Use dynamic selection where interpreting new information usefully changes the work, while accounting for additional latency, variable execution paths and opportunities for mistakes. [2]

The boundary can be small. A prescribed incident workflow can delegate investigation to an agent, then require a structured finding before proceeding. Conversely, the agent can invoke a fixed diagnostic procedure as one tool. Agents vs Workflows: Why Not Both? illustrates this composition: an agent can occupy a workflow step, and a workflow can supply an agent capability. Tool count and model-call count do not identify where discretion lives. [4]

Define the permitted task

Useful discretion requires something stable to pursue. A task specification states the required outcome, starting information, permitted effects and operating constraints. Keep these separate from a proposed procedure. An investigation may change its search strategy while preserving the same target and reporting obligation. Permission to investigate does not implicitly include permission to repair. [5]

Write success conditions in terms that can later be examined. “Investigate thoroughly” leaves both action selection and completion ambiguous. “Identify the failed stage and support the diagnosis with relevant records, or explain what remains unresolved” gives the agent a target for its observations. Instructions communicate these requirements; Specify the intended task develops that interface. [5][6]

Distinguish missing external facts from missing intent. A lookup can establish when a job ran; it cannot establish which of two plausible jobs the requester meant. DiscoBench's clarification-aware search tasks make this distinction explicit by withholding a distinguishing constraint that a simulated user can supply. Ask when resolving that ambiguity changes the target or acceptable action. An incidental choice within an already defined task need not reopen the delegation. [7][8]

Feed results into the next choice

The environment comprises the external systems and conditions relevant to the task. A policy is the rule or model-based process that selects actions using observations and retained information. An action can inspect the environment or change it. The resulting observation then becomes input to a subsequent decision. [3]

A tool call requests an operation exposed by the application. The model supplies the proposed operation and arguments; application code validates, authorizes and dispatches it, then associates the returned result with the call. A structurally valid request does not establish permission or successful execution. Data, requests and effects explains these boundaries. [9]

Closed-loop execution uses observations of action results to guide later choices. A failed test can lead to inspecting a different function; a missing prerequisite can lead to obtaining it before continuing. This adaptation does not require further training. The model's weights, the numerical parameters learned during training, can remain fixed while each invocation receives new information. Changing inputs, keeping weights fixed explains this distinction. [10][11]

Expand ⤢

The distinction becomes concrete in What if the harness mattered more than the model?. An initial coding agent proposed a median-function fix without editing the file. After adding test execution and instructions to use feedback, the demonstrated agent ran the failing test, edited the file and reran the test successfully. Both the tools and prompt changed. The useful lesson is the completed feedback path, rather than an isolated claim about either intervention. [12]

Decision-making and execution machinery have different responsibilities. The policy chooses work; the runtime carries out requests, returns observations and supports continuation. Agent Runtimes and Harness Engineering covers that machinery. Feedback also depends on timing: waiting for a complete tool response simplifies turn-taking but can delay reaction to a changing process. Generated code may itself inspect results and branch, so producing several instructions at once does not necessarily make their execution open-loop. [13][14][1]

Separate observations from state

Partial observability means the agent receives only some of the information relevant to the task. The environment has a state; the agent has observations of that state and interpretations of them. Kaelbling, Littman and Cassandra's 1998 account of planning under partial observability formalizes this distinction and shows why an information-gathering action can be useful before committing to another action. A language-model transcript is not automatically the probabilistic belief state used in that formalism. [15]

Maintain a working account that distinguishes an observed fact, an inferred explanation and an unresolved outcome. “The query returned no rows” is an observation. “The record does not exist” is a stronger conclusion that depends on the query's scope and freshness. Contradictory responses may concern different versions or moments. Before acting on them, identify what was inspected and when. Context as a selected view and State beyond the transcript explain how this account differs from the input supplied to one model call. [16][15]

Consider a delayed operation whose last status response says running. The server subsequently completes the work, but the agent has not yet received another response. The correct working status is still “last observed running; current completion unresolved.” Google's long-running operation pattern provides a concrete interface for tracking such work through an operation reference and subsequent status requests. [17]

Task records should therefore preserve pending outcomes and remaining dependencies alongside completed work. A self-maintained to-do list makes intended progress inspectable, but crossing off an item supplies no independent confirmation. Anthropic's long-running agent experiments used explicit feature requirements and tests before marking features complete, addressing premature completion and poorly documented partial work. [18][19]

A short history of agent architectures

Planning anticipates dependencies, reaction responds to change, and commitment preserves useful decisions. Agent architectures developed ways to combine these needs long before language models. These turning points explain why the approaches still coexist.

Language models enter the loop

Reiichiro Nakano and OpenAI collaborators' WebGPT, 2021, trained GPT-3 to research answers through a text browser. The question and browser state guided commands to search, follow links, scroll and collect references. A model command or imposed limit ended browsing, followed by an answering phase. Language-based action selection thus worked within prescribed phases and bounds. [28]

Shunyu Yao and colleagues' ReAct, submitted in October 2022 with an ICLR camera-ready revision in March 2023, made interleaved reasoning, action and observation a reusable pattern. New observations could change subgoals and subsequent actions. Studies covered question answering, fact verification and interactive decisions, but also found reasoning errors, uninformative searches and unproductive loops. This supplied a language-model mechanism for feedback; explicit dependencies, execution checks and stopping limits still required attention. [29][30][10]

Maintain an executable plan

A plan is a proposed arrangement of actions and subgoals. A subgoal is an intermediate condition that helps satisfy the overall goal. Decomposition makes work easier to inspect, but each new subtask introduces another decision or handoff. Choose a level at which prerequisites and results are clear without turning one coherent operation into many unnecessary choices. [31]

An action's preconditions specify what must hold before it is applicable; its effects describe the facts its modeled execution adds or removes. STRIPS uses these descriptions to make prerequisite ordering explicit, while distinguishing a modeled action from actual robot execution. A step can name the right operation yet occur too early. Valmeekam and colleagues' 2023 planning investigation tested models on tasks with explicit starting states, goals and action rules. The models often used permitted action names but produced sequences that an external validator rejected. Checking the actions in a plan is therefore different from checking whether their ordering can achieve the goal. [21][32]

Later models showed stronger planning capability, though results remained sensitive to how the task was represented. A September 2024 PlanBench study reported that o1-preview solved 587 of 600 Blocksworld tasks: rearranging blocks under specified action rules. These problems used three to five blocks and required solutions of two to sixteen steps. On a semantically equivalent set with renamed concepts, it solved 317 of 600. The improvement over earlier models and the difficulty with renamed tasks support separate conclusions: planning capability can improve, while executable-plan checks remain valuable. [33]

Planning horizon

The last approach resembles receding-horizon planning in model predictive control: optimize a future sequence, execute its first input, obtain a new state estimate and solve again. The control-theory formulation requires predictive dynamics, objectives and constraints. Applying its pattern to language agents does not transfer its feasibility or stability results. Deliberation and search can help select the continuation; their computational tradeoffs belong in Reasoning and Test-Time Compute. [37]

Commitment and reconsideration

A plan also saves work by settling choices provisionally. Michael Bratman, David Israel and Martha Pollack's 1988 account of resource-bounded practical reasoning explains how commitment focuses later decisions and filters incompatible alternatives. Reconsidering everything at every step consumes time while the environment continues changing. Plans should remain partial where future conditions are unknown, and revisable when new information invalidates them. [38]

The Procedural Reasoning System made this balance operational: select procedures using current beliefs and goals, retain adopted procedures as intentions, and elaborate near-term steps while leaving later details open. New observations could activate another procedure or interrupt existing work. Its use with SRI's Flakey robot demonstrated this combination of goal-directed planning and reaction. [25]

In belief–desire–intention architectures, beliefs represent information about the environment, desires represent objectives and priorities, and intentions retain the selected course of action. Commitment rules determine when to reconsider. Rao and Georgeff's OASIS example retained an aircraft arrival sequence until completion or until a relevant timing condition could no longer be met. It was evaluated alongside airport operations using live radar data; it did not establish operational control of traffic. [26]

Selective revision

Sehoon Kim and colleagues' LLMCompiler, submitted in 2023 and published at ICML 2024, arranges tool calls proposed by a large language model into a dependency graph. Independent calls can execute together; a dependent call waits for the results it needs. Execution observations can return to the planner to produce another graph. This avoids requiring a model decision between every independent operation. FactSet's planning-agent adaptation separates a high-level workflow outline from detailed planning and execution, then combines task results before deciding whether to replan or finish. [39][40]

When a prerequisite changes, revise the affected continuation. Suppose an input has been checked, but the intended processing service becomes unavailable. The input check can remain useful while the agent investigates another permitted method. The goal is blocked only if no feasible continuation remains. Rechecking everything wastes work; retaining every assumption ignores the new evidence. Monitored plan repair makes that distinction explicit. [22][36]

PLANEX checked conditions needed by the remaining plan against its current world description. Observed progress could make planned steps unnecessary; a short recovery plan could restore conditions needed to resume the original plan. It requested replanning when no usable continuation remained. This separates repairing a prerequisite from replacing the whole strategy. [22]

Choose a useful next action

The action space is the set of operations available to the policy. Choosing within it requires more than recognizing the appropriate function. The agent must supply suitable arguments, satisfy prerequisites and choose an operation useful at this point in the task. Choose a call or another response explains immediate selection; sequential selection adds the consequences for later work. [41][42]

Value of information is the benefit of learning something before choosing a subsequent action. An inspection earns its cost when the expected improvement in subsequent decisions outweighs its cost and delay. Checking a dependency can distinguish “proceed” from “repair the prerequisite.” Another search that cannot change the chosen action adds delay without the same benefit. The metareasoning literature similarly evaluates computation through the decisions it can improve, including its cost and delay. [43]

Tool granularity determines how many choices the agent must make and what each choice reveals. John Yang, Carlos Jimenez and colleagues' SWE-agent, 2024, studied this through the commands and observations used to search, view and edit repositories. On 300 repository-repair tasks from SWE-bench Lite with GPT-4, an interface returning search summaries resolved 18% of tasks versus 12% for an iterative interface using next/previous commands. Repeated inspection sometimes exhausted the iterative interface's budget. The comparison held model weights fixed and changed the interface; its result supports evaluating action granularity for the task rather than assuming that finer control always helps. [44]

The useful unit is an understandable operation with informative feedback. A task-oriented tool may combine a frequently repeated procedure, while an interactive terminal may need character-level control. Evaluate granularity against the actual task: too little feedback hides decisions, while unnecessary micro-operations multiply them. [45][42]

Responding while other work continues

Responsiveness also depends on how behaviors share control. In Brooks's subsumption architecture, concurrent layers contain asynchronous finite-state modules that respond to inputs without advancing together. Suppression temporarily replaces a module's normal input with another signal. Inhibition temporarily blocks an output, discarding messages emitted during that interval. Lower layers keep running while higher layers intervene at selected connections. [23]

Directed travel could suppress random wandering while obstacle avoidance remained active; wandering could also be inhibited during observation collection. These signal paths let basic behavior continue without waiting for higher-level results. The robot still had limitations around fast obstacles and clutter. [23]

Brooks's 1991 exposition, Intelligence without representation, explained the broader architectural choice: build complete sensing-to-action behaviors incrementally instead of depending on separately developed modules meeting through an accurate central world model. This approach reduced dependence on that representation. It did not establish that planning was unnecessary; it identified a different way to organize responsive behavior. [46]

Constrain accumulated effects

Bounded autonomy gives the agent freedom to select actions within an explicit task, authority and resource allowance. Delegated authority specifies the operations it may perform on particular resources for the requester. This boundary applies to the whole attempt. A collection of individually permitted operations can still accumulate excessive effects or move beyond the original goal. [47][48]

Specify resource scope, allowed operations, cumulative change limits, elapsed time and action budgets separately. Ten inexpensive reads and ten large model requests do not consume equivalent resources. A count limit cannot substitute for every other limit, and keeping each call small does not bound total work. OWASP's unbounded-consumption guidance motivates accounting across inputs, repeated requests and downstream operations. [49]

A documented AgentCore temporal-policy example counts requested amounts over five minutes within a session. Requests of 1000 amount units permit the first two; the third reaches the denied threshold of 3000. This counts requests, not confirmed transfers, and does not establish a limit across separately created sessions. [48]

Behavioral instructions help the policy choose appropriately, but trusted code must enforce protected boundaries. The agent cannot grant itself more authority because a different action appears helpful. Enforce at protected boundaries develops the security principles. In a data-pipeline remediation example, a learned policy proposes a response while an external safety layer can override it and escalate critical or unknown cases. [50][51]

Prompt injection occurs when untrusted material is interpreted as instructions that redirect the system. A log entry, retrieved page or tool response may contain such instructions, but receiving that content cannot enlarge the delegation. The application must continue to enforce access and action limits independently. When content becomes instructions explains the attack mechanism. [52]

Request the right human decision

Human intervention can supply three different things: information about intent, permission to act, or responsibility for work the agent cannot complete. Mixed-initiative interaction means that people and software share control. Eric Horvitz's 1999 principles connect intervention to uncertainty about user goals, the consequences of a mistaken action and the cost of interrupting the person. [8]

Ask a focused clarification when its answer changes the plan. Request approval at the point where a reviewable consequential action is ready. A changed target or material payload needs renewed authorization; broad assent does not approve every later variation. See Approval has a scope. Enforcement must sit on the execution path, as illustrated by n8n's tool-interception workflow, rather than depending on the agent choosing an optional review tool. [53][55]

A handoff also needs accepted ownership. PagerDuty's incident lifecycle distinguishes notification, acknowledgment and resolution: acknowledgment means a responder has claimed the still-unresolved issue. An agent that merely sends an escalation message should retain the fact that ownership is unconfirmed. Its remaining work must respect that waiting state and the original scope. [56]

Review is fallible. In Duolingo researchers' 2025 fabricated-alert studies, reviewers accepted some erroneous alerts even after guidelines emphasized independent video evidence. The studies concerned exam review, not agent transaction approval, but demonstrate why a human checkpoint alone cannot establish correctness. Give reviewers the information needed for independent judgment, and account for their attention and workload. Review meaningful changes covers that presentation. [57][58]

Establish the achieved outcome

The task's success conditions determine what must be checked after action. A postcondition is a required property of the resulting state after successful work. A verifier is a procedure checking a stated property. It may inspect an artifact, execute a test or read authoritative application state. The strength of the conclusion depends on what it actually checks; Checks and their limits explains that boundary. [59][60]

Match the check to the actual target and result. For the export, inspect the requested account, period and fields, and establish that the artifact can be used as required. Preserve the identity of the checked artifact; a check of an earlier version does not establish a later version's properties. Acceptance, readiness and task satisfaction are separate facts even when an API exposes them through similarly named statuses. [59][16]

Use direct outcome evidence where possible. A tool-call record establishes what was requested; the resulting database state can establish whether the intended record exists. A passing unit test establishes its tested behavior, while an interactive workflow may require additional checks. Anthropic's coding experiments reported failures visible through browser interaction that simpler checks missed. [62][19]

When direct verification is unavailable, identify the proxy and the judgment it leaves open. Resemblance to a trusted reference can help assess a document, but cannot establish suitability for every new situation. Report completion as a supported account of the artifact, scope, checks and unresolved work. What Does Done Even Mean? develops this view of completion as several claims rather than a single status flag. [63][61]

Decide whether to continue

A termination condition ends the current attempt. Verified completion is one such condition; exhausted limits, cancellation or inability to proceed within scope are others. Waiting for useful information differs from ending the attempt. Define these dispositions explicitly so the system can distinguish work that is complete, deferred, blocked or merely no longer being pursued. [64][65][66]

Progress detection asks whether observations establish movement toward unmet requirements. More calls, a longer report and a revised to-do list are weak substitutes. Repeating an action without obtaining new information suggests a stall; alternating between approaches without resolving their failure suggests oscillation. The next action should have a feasible reason to improve the task's position. Otherwise, revise the approach, seek intervention or end with an explicit account of what remains. [67][68]

Microsoft's Magentic-One, 2024, provides a concrete pattern: a task ledger retains facts and a provisional plan, while a progress ledger checks completion, repetition and forward movement. Detected stalls can trigger strategic replanning. These remain model assessments, and independent attempt or time limits can terminate work with only a best guess available. [67]

A limit bounds execution without establishing success. LangGraph's recursion limit counts graph super-steps, which need not equal model calls or tool calls. An application can exit gracefully with a partial result before reaching the limit. Absence of an exception therefore does not establish fulfillment of the original task. [66]

Record the stopping reason, task outcome and outstanding operations separately. Ending action selection does not stop an already running external process. Temporal's activity documentation, for example, describes cooperative cancellation that an activity can ignore. Agent Runtimes and Harness Engineering covers continuation and cancellation mechanics; the agent's report must preserve pending work until its status is resolved. [69]

Repair the strategy from known effects

Recovery starts by distinguishing confirmed success, confirmed failure without an effect, and an unknown outcome. Recovery after execution introduces these cases. A timeout can leave the client uncertain even though the operation completed. Repeating the request under that uncertainty can create a second effect. [69]

Reconciliation checks authoritative state against the intended operation. After a lost acknowledgment, retain the operation identifier and inspect its status before considering a potentially duplicating action. A confirmed effect preserves progress. Confirmed absence may permit another attempt. If the evidence remains inconclusive, keep the outcome unresolved and apply the remaining time and authority limits. [70]

Idempotency can make repeated requests represent one logical operation when the receiving API enforces that contract. A caller-provided identifier distinguishes a retry from a deliberate second operation with identical parameters. It is not an exactly-once guarantee for arbitrary tools. The implementation belongs in Agent Runtimes and Harness Engineering; strategically, duplicate protection and determining whether the task succeeded remain separate needs. [71]

Once effects are known, match the repair to the obstacle. Correct a request rejected before execution. Obtain information missing from a decision. Refresh state whose assumptions no longer hold. Choose another method when the current one lacks a required capability. Repeating an infeasible approach consumes the allowance without repairing the plan. Retain completed work where it remains valid, and reconsider the dependent continuation. [64][36][72]

The Remote Agent flight experiments illustrate two recovery boundaries. An injected camera-switch failure prompted onboard replanning. A separate real execution deadlock required ground investigation and termination of the first experiment; a later six-hour experiment completed validation. Operators had supplied goals and constraints, but remained responsible for a fault the onboard system could not resolve. Successful recovery from one disturbance did not establish autonomous recovery from every failure. [27]

Compensation performs new actions to counter earlier effects. It differs from atomic rollback: another actor may have changed the state, cancellation may incur a cost, and an already observed message cannot be made unseen. Microsoft's compensating-transaction pattern therefore treats recovery as business-specific work that can itself fail. Any remedy requires its own applicable authority; an earlier mistake does not grant permission for unlimited corrective action. [73][50]

Escalation is a legitimate recovery outcome. In the data-pipeline remediation design, deterministic checks establish incident facts, a bounded policy selects a response, and an external layer escalates unknown or high-risk cases. Measuring success only by avoidance of escalation would penalize intended behavior. The relevant question is whether the system made the best permitted progress and left unresolved work with a usable next step. [74][51]

Explain the consequential divergence

A trajectory is the ordered observations, selected actions and results from one task attempt. Examine both the final outcome and the decisions along the way. Several trajectories can satisfy the same task, so exact agreement with a reference path can reject useful alternatives. Conversely, reaching the right final state can conceal a prohibited intermediate action. [60][75]

Locate the earliest consequential divergence rather than the first superficial difference. First establish whether the needed information reached the decision-maker. A fact absent from a diagnostic log may still have appeared in the effective model input; inspect the actual input boundary before attributing an information gap. If the observation was available, examine whether the selected action was justified by it. Record inputs, outputs and versions explains the recording requirements. [76][77]

Consider two attempts given the same observation: a required input has not yet been produced. One inspects the upstream job; the other immediately starts the dependent operation, which rejects the missing input. Their action choices diverge under shared information. The second choice is inconsistent with the known prerequisite, but its trace alone does not establish whether the model ignored or misunderstood the observation. [32]

Generated explanations are additional outputs, not direct access to the cause of a decision. Anthropic's reasoning-faithfulness experiments found cases where injected hints changed answers without being acknowledged in the generated reasoning. Such accounts can suggest hypotheses, but cannot by themselves prove that a particular observation caused the action. [78]

Execution conditions can also explain the outcome. Anthropic's infrastructure-noise study varied resources while holding the model, harness and tasks fixed, changing both infrastructure failures and task success. A failed dependency installation could reflect a resource-heavy strategy interacting with a memory limit. Use Turn failures into hypotheses to test such explanations through controlled changes. [79]

Across attempts, group paths by meaningful decisions and compare their outcomes. A recurring low-performing sequence can reveal a missing prerequisite or an unhelpful tool choice, but association is a lead for investigation. Preserve enough identifiers, outcomes and selected diagnostic fields to test that lead. Do not make unrestricted retention of prompts, retrieved documents or tool responses the price of debugging; OWASP's logging guidance requires protecting or excluding sensitive data. [80][81]

Measure the value of discretion

Return to the original architectural choice with whole-task evidence. Compare credible alternatives: a prescribed procedure, a procedure containing one bounded dynamic stage, and broader model-directed execution where that design is plausible. Define each variant by the decisions it delegates. Restore starting states separately, and state tool access, time limits, retry allowances and human assistance. Tasks, attempts and outcomes and Baselines, budgets and repeated attempts establish the comparison method. [2][60]

AI Agents That Matter, 2024, provides a narrow reason to take simple baselines seriously. Across 164 HumanEval function-generation tasks and five runs per system, a procedure that regenerated after test failure while increasing sampling temperature had no significant accuracy difference from the best-performing complex architecture, while costing less than several agents. This concerns short coding tasks, not open-ended investigations or human supervision costs. [82]

τ-bench, 2024, broadened assessment to interactions among tools, agents and simulated users. It checks final database state and required user-facing information, and measures consistency across repeated trials. Its pass^k measure concerns all k independent trials succeeding, whereas pass@k requires at least one success. The paper explicitly warns that a successful final-state check can miss a trajectory violation such as acting without required confirmation. [75]

Separate task utility from unwanted consequences. AgentDojo uses explicit simulated application state and distinct checks for intended task success and attacker objectives. That design permits a satisfactory answer and an unwanted side effect to be recorded separately. Each checker still covers only its encoded conditions. [83]

Exercise disturbances deliberately. ReliabilityBench combines repeated executions, task variations and injected tool failures with final-state predicates. For your application, include blocked prerequisites, stale information, unknown effects and exhausted allowances alongside ordinary completion. Distinguish an error injected before execution from an acknowledgment lost after a mutation; they require different recovery. Test-case coverage must reflect the intended users and operating conditions, as emphasized by NIST's measurement guidance. [84][85]

A changed action needs a consistent continuation. Once a candidate takes a different branch, feeding it the old run's subsequent observations may describe a world its actions never produced. Recompute downstream state and observations through an appropriate executor or simulator. Offline, replay, shadow and live evidence explains this limit; RL Environments and Simulators covers the environment mechanics. [86][87]

Simulator behavior also changes the difficulty. In Build Evals That Actually Matter, Lyft's initial simulated users patiently supplied unusually complete explanations. Evaluation became harder after the team incorporated production examples and trained the user simulator to resemble customer language. Lower scores alone did not prove production validity, but exposed how cooperative simulation could hide failures. [88][89]

Use the comparison to change a specific delegation. If adaptive investigation improves acceptable completion but unrestricted repair adds unwanted effects or review burden, keep investigation dynamic and constrain repair. If a prescribed procedure handles the workload adequately, broader discretion needs another demonstrated benefit. The engineering result is a justified boundary around particular decisions, supported by outcomes under the conditions in which the system will operate. [2][85]

Open questions

1. 01

   The useful boundary of discretion remains workload-dependent. Broader planning can adapt to unfamiliar conditions while adding execution and supervision costs. Progress would mean matched comparisons that isolate delegated decisions and measure acceptable completion, unwanted effects and human correction effort together. [82][4]

2. 02

   Progress detection remains difficult when intermediate results are ambiguous or delayed. Model-written ledgers can preserve both useful commitments and mistaken assumptions. Better methods would detect consequential stalls early, without discarding valid work or repeatedly interrupting productive attempts. [67][95]

3. 03

   Human intervention must improve decisions within finite review capacity. Approval interfaces can encourage deference, while frequent interruptions consume attention. Progress would establish when clarification, independent review and handoff improve whole-task outcomes after accounting for the person's effort. [58]

4. 04

   Simulations must remain informative when agents choose unfamiliar paths. Consistent tool state is necessary, but realistic user behavior and disturbances are also difficult to reproduce. Progress would show that changes which improve simulated outcomes predict improvements under representative live conditions. [87][89]