Context Engineering: Choosing What a Model Knows at Each Step

The input the model sees

For one invocation, model context is the complete model-visible input: application instructions, the current request, selected conversation turns, retrieved material, tool definitions and schemas, tool results, examples, and structural markers added during serialization. These components eventually become an ordered token sequence. Count the complete request explains that final representation; Reserve room for generation explains its capacity bound. [1][2][3]

The context window bounds the sequence available to the invocation. It does not contain everything the application can access, and prompt caching does not enlarge it: a reused prefix still occupies capacity. Likewise, accepting a long input does not guarantee that the model will use every included fact reliably. Controlled long-context experiments have found position-sensitive behavior in several historical models and tasks, so capacity and effective use are different claims. [3][4][5]

Context changes computation at inference time, not the model's stored parameters. Removing an instruction or example from the next call removes that invocation-time influence unless the application supplies it again. What a prompt changes develops this distinction; parameter updates belong to training and post-training, not context engineering. [6][7]

The practical question is therefore not “How much can we fit?” but “What should this call be allowed and expected to use?” An application may possess a document that is relevant but unauthorized, a tool result that is permitted but stale, a correct record that contributes nothing to the present decision, or a large catalog whose definitions crowd out the task itself. Exclusion is part of the design, not evidence that the information was lost. [8][9][10]

State beyond the prompt

Once a call is understood as a temporary projection, the fuller task record needs another home. Authoritative task state is the application-controlled account of goals, constraints, accepted decisions, artifacts, observations, pending operations, and completion status. It may live in a database, event log, repository, workflow record, or another durable store. The model can receive a selected view of that state, but its generated claims do not modify the authoritative record until application code validates and accepts them. [11][12][13]

A conversation transcript is useful evidence about what was said, but it is not automatically a database of what is true. A user may correct an earlier statement; a tool result may report a transient observation; a model may assert that work completed when the external system still shows it running. Separate observations from state explains why observations need interpretation, while Preserve state at acceptance supplies the broader rule for accepting changes. [13][14]

Persistent memory is different again. It retains selected information across interactions—such as a preference, prior event, or reusable fact—and makes that information available for later retrieval. It is a candidate source for a call, not the call itself and not necessarily the authoritative state of the present task. Memory formation, correction, expiry, and forgetting belong in Agent Memory. [15][16]

Expand ⤢

Three different lifetimes

From sources to a request

A context assembler constructs the complete input for a specific model call. It first identifies the decision the model must make—such as answering a question, choosing a tool, assessing an artifact, or proposing the next step. It then finds the information that decision depends on, excludes sources that are not permitted or valid, and converts the selected material into an ordered model-visible request. [1][11]

Assembly stages

A system instruction is application-supplied model-visible instruction carried with provider-specific authority semantics. It is not a universal wire format. Similarly, a tool definition is context that describes an available operation; its executable implementation and authorization policy remain outside the model. Tool definitions shape available choices owns that interface in detail. [19][20]

The final request is a derived view. A concise context record should identify the assembly-policy version, source identities and versions, material transformations, ordering, and token count. This does not make the request true or safe by itself. It makes its construction inspectable and allows a failure to be reproduced without pretending that the prompt was the authoritative source. [18][11][21]

Roots of programmable context

The recent label combines several older engineering problems. Limited active capacity, relevance-ranked information access, explicit interaction state, invocation-time task specification, and external knowledge were developed along partly independent lines. The milestones below are useful antecedents, not a claim that one directly caused the next or that newer techniques replaced older ones. [22][23][12]

Must-know developments

These developments address different constraints. Working sets concern active capacity; retrieval concerns candidate selection; dialogue state and blackboards concern explicit state; prompting concerns task information supplied at invocation time; RAG concerns evidence-conditioned generation; tiered-memory systems concern reacquisition after information leaves the active window. Modern context engineering must combine these concerns because an agent request may contain instructions, evidence, state projections, tool choices, and untrusted material at once. [25][27][26]

Larger windows reduce some pressure but do not decide permission, truth, freshness, provenance, or which dependencies a decision requires. Nor do they guarantee stable use of information across positions and tasks. Context engineering remains an application concern because those questions depend on the application, not only on the model architecture. [4][28][29]

Select for the decision

Selection begins with eligibility, not relevance. A record is eligible only if current policy permits its use for this actor, purpose, and task. Among eligible candidates, the assembler asks whether a set is sufficient for the current decision. A useful passage can still be excluded because it is stale, redundant, too sensitive, or unnecessary; a highly ranked set can still be unusable because it omits a required connection. [30][31][32]

Two retrieval failures need different diagnoses. Low precision includes irrelevant material that can distract generation. Low recall omits evidence needed to answer. Raising top-k may help recall while worsening precision, disclosure, and budget use; nearest-neighbor rank also returns candidates when a corpus has no valid answer. Search, indexing, candidate generation, and reranking belong in Search and Retrieval. Here the concern is which candidates become model-visible. To answer whether the current build passed, a result must be associated with the commit being checked. [33][34][35]

Candidate dispositions

Record meaningful omissions. “Excluded because unauthorized,” “superseded by version 12,” and “not loaded because the current decision does not require it” imply different uncertainty and recovery paths. Omission records also make evaluation possible: a failed answer can be traced to absent evidence rather than vaguely blamed on the model. [18][30]

Reveal detail on demand

Progressive disclosure exposes enough metadata to discover a resource before loading its full contents. A compact catalog entry might carry a name, purpose, scope, source, and retrieval reference. If the task makes the resource relevant, the assembler can load a focused excerpt; only a later need justifies the full artifact. This is useful for large documentation sets, skill libraries, tool catalogs, and long tool results. [38][39][40]

An Agent Skill is one concrete staged format: discovery metadata is available first, instructions load when the skill is activated, and referenced resources load separately as needed. The general mechanism is broader than skills. File trees, indexes, manifests, graph views, and deferred tool definitions all separate discoverability from immediate inclusion. [38][41][39]

Deferral creates a new dependency: the path back to detail must work. A preview can omit the answer, an entry can expire, a vague description can prevent discovery, and repeated reads can add latency. A retained reference is not recoverability unless storage still contains the source, permissions still allow access, and the agent has a functioning retrieval operation. Controlled research also suggests that gains depend on corpus size, metadata design, task, and the harness's existing navigation behavior. [37][42][43]

Order and label the material

After selection, the model receives an ordered serialization rather than an unordered set. Application instructions, the current task, demonstrations, retrieved sources, tool observations, and output expectations occupy positions and roles. Changing their order or labels changes the input, even when their human-readable facts remain the same. Order and format are interventions covers the broader prompt-sensitivity evidence. [4][44][5]

Role metadata describes authority assigned by the application. A retrieved document cannot promote itself by claiming to be a system message. Conversation history and retrieved evidence should remain distinguishable, and derived statements should retain their links to source versions. Time labels matter when a current request includes historical records or observations collected at different moments. [19][45][18]

Controlled representation changes

There is no universal best template. Stable conventions make changes reviewable, but their behavior still needs workload-specific tests. Repeating every instruction may improve salience while consuming budget and adding conflicts; placing all evidence first may help one task and obscure another. Preserve complete serialized inputs so an ordering experiment can actually be reproduced. [4][44]

Allocate the request budget

A context budget is an application policy over the complete counted request. Instructions, the current task, evidence, history, tools, structural tokens, and generated output share capacity. Let $P$ be the complete serialized input, $R$ the generation allowance reserved by the application, and $C$ the supported context capacity. The basic guard is: [2][3]

The symbols describe capacity, not quality. Count the complete provider-specific serialization with the intended model. Hosted counting endpoints may provide exact counts or estimates under different contracts, and provider-added material can affect the result. Reserve output before spending the remainder on input; reasoning models may require additional provider-specific accounting. [21][2]

A practical allocation policy

A fitting request can still perform poorly because required relationships are scattered, distractors remain, or useful material occupies a position the model handles weakly. Conversely, a shorter request may incur extra retrieval latency or omit a dependency. When longer context helps covers architectural behavior; the application must evaluate the allocation on its own task distribution. [28][4][42]

Refresh, supersede, remove

Freshness is not one age threshold. Observed-at time records when the application obtained a value. Effective time records when the source says the value applies. A version identifies a particular representation; an expiry or validation rule says when reuse requires another check. Authority identifies which source may decide the fact. These dimensions can disagree. [47][32][48]

Consider a build status cached at 10:00. A cache hit at 10:05 proves reuse of the stored representation, not that the build is still running. A newer observation from an unofficial mirror may also be less authoritative than an older record from the build system. The assembler needs a source-specific rule: refresh on read, conditional revalidation, a time-to-live, event-driven invalidation, or rejection when current validity cannot be established. [32][49][13]

Staleness can corrupt downstream decisions even when the original write succeeds. In one reported multi-agent deployment, a new credit score reached PostgreSQL but a shared cache was not invalidated; a later agent read the older cached record and made its decision from stale evidence. The lesson is architectural: source success, cache coherence, and downstream decision correctness are separate claims. [50]

Different lifecycle operations

When required freshness or current permission cannot be established, fail closed for the affected claim or operation. Historical information may remain useful as history, but it must not silently authorize a current action. Maintain fitness through change covers general source change, while Enforce current authority covers permission decisions. [32][31]

Keep untrusted content in role

Retrieved documents, webpages, attachments, memory records, and tool results can be relevant without being trusted instructions. Prompt injection occurs when attacker-controlled content causes an instruction-following model to cross the application's intended task or instruction boundary. An indirect injection arrives through material the application meant to treat as data. When content behaves like instruction develops the attack mechanism. [10][53]

Assembly should retain source identity, derivation, trust classification, current permission, and data role. Delimiters, role labels, encodings, and classifiers may improve behavior, but they are instructions to a fallible model rather than a hard isolation boundary. A Base64 spotlighting counterexample illustrates the general limitation: changing representation does not by itself prove that content cannot influence instruction following. [19][54][55]

Any model-proposed external effect must cross a separate enforcement boundary. Trusted application code binds the authenticated actor to the requested operation, target resource, current state, and delegated scope. A valid tool call, persuasive explanation, or provenance label cannot grant permission. Controls belong on attack paths and Authorization belongs at the protected operation cover those controls. [31][53]

Apply least disclosure before the call as well as least privilege at execution. If the immediate task does not require a secret, private record, internal identifier, or destructive capability, do not expose it merely because the application can. Reducing unnecessary context narrows both distraction and the material available for exfiltration. Minimize without losing the task supplies the broader governance rule. [56][10]

Compact, but preserve recovery

Compaction replaces a larger active history with a smaller representation. Summarization is one lossy compaction method, but it is not the only one. An application can remove duplicated structure, mask old tool observations, retain selected extracts, write an abstractive summary, build hierarchical summaries, or extract explicit task state. Each method preserves different information. [57][58][59]

Methods and losses

Define invariants before compacting. For long tasks these often include the current goal, accepted decisions and their rationale, unresolved obligations, safety constraints, pending operation identities, artifact references, source versions, and known failures. Preserve authoritative raw records or recoverable artifacts outside the compacted prompt. A fluent summary cannot certify that every later-relevant detail survived. [62][59][60]

Expand ⤢

Compaction can also remove governing instructions or collapse distinctions between current and historical state. Some implementations therefore pause after compaction so the application can reinsert selected instruction-oriented or recent material. Others reset the active context and reconstruct it from durable handoffs. Neither approach is universally superior: both depend on the quality, provenance, and completeness of what survives outside the window. [57][63][64]

Reconstruct the next call

Long-running work survives when continuity flows through durable records rather than hidden model persistence. Useful records include task identity, accepted state version, current plan, artifacts, source versions, unresolved questions, pending operation identities, and verification results. Each call receives a fresh projection sufficient for its next decision. [65][62][17]

The model returns observations and proposed changes. Application code validates them against the current state before acceptance. Before dispatching an external effect, it assigns a stable operation ID and atomically compares the expected state version while recording the operation as pending. This durable reservation closes the gap in which another worker could advance the task after a version check but before dispatch. After reconciling the effect by operation ID, the application atomically accepts the resulting transition only if the pending state still has the expected version; otherwise it preserves the pending record for further reconciliation and rebuilds context from current state. [13][66][67][68]

An interruption around an external mutation creates an additional problem: the effect may be unknown. Cancellation does not necessarily roll back server work, and a missing response does not prove failure. Preserve a logical operation identifier, query authoritative status, and retry only under an idempotent contract. Represent uncertain external outcomes explains this boundary. [68][67]

A conceptual acceptance boundary

def run_step(task_id):
    state = load_authoritative_state(task_id)
    projection = assemble_context(state, policy_version="ctx-7")
    proposal = call_model(projection)
    checked = validate_proposal(proposal, state)

    if not checked.requires_external_effect:
        return compare_and_accept(
            task_id, expected_version=state.version, proposal=checked
        )

    operation_id = new_operation_id()
    pending = compare_and_record_pending(
        task_id,
        expected_version=state.version,
        operation_id=operation_id,
        arguments=checked.arguments,
    )
    if pending is None:
        return "stale_projection"

    dispatch_with_idempotency_key(operation_id, checked.arguments)
    outcome = query_authoritative_status(operation_id)
    reconciled = reconcile_effect(checked, outcome)

    accepted = compare_and_accept_reconciled(
        task_id,
        expected_version=pending.version,
        operation_id=operation_id,
        proposal=reconciled,
    )
    return "accepted" if accepted else "needs_reconciliation"

Runtime scheduling, checkpoint implementation, cancellation propagation, and worker recovery belong in Agent Runtimes and Harness Engineering. Context engineering owns the reconstructable projection: which durable facts and references the next invocation must receive after an interruption, compaction, handoff, or model replacement. [69][14]

Test the context policy

Treat the assembler as versioned software. A context manifest is a useful application contract even though no provider-neutral standard defines one. It can record the policy version, current decision, source identities and versions, eligibility results, selected and omitted components, transformations, ordering, token counts, trust labels, and a privacy-controlled reference to the final serialization. [18][70][11]

Test deterministic assembly separately from model behavior. Unit and snapshot checks can establish that unauthorized records are absent, required components precede serialization, source versions are recorded, overflow follows declared priorities, and compacted records retain named invariants. These checks do not establish that a model will use the resulting context correctly. [70][36]

Claim-to-check map

Activation gates matter. A short conversation can score perfectly in a compaction evaluation simply because it never triggered compaction. One reported context-policy experiment checked trigger activation separately and found that single-turn cases often failed to exercise the strategies under study. Similarly, a freshness test is uninformative if the stale cache was bypassed, and a progressive-disclosure test says little if the full resource was already loaded. [73][72]

For behavioral comparisons, keep the model, decoding settings, tools, task set, and grader fixed while varying the named context policy. Include an untouched-history baseline before assuming that compression helps. Use development tasks to choose the policy and held-out tasks for final assessment; repeated trials should reset memory when they are meant to be independent. Compare changes on matched work develops the comparison design. [74][36][75]

Telemetry should answer the operational question without collecting every prompt by default. Useful fields include input and output tokens, cached tokens, latency, tool calls, summarization events, policy version, source versions, and activation status. Full instructions, inputs, and outputs may be sensitive and large; Record meaning-changing boundaries and Minimize without losing the task cover controlled capture and minimization. [71][70]

Open questions

1. 01

   How can a context manifest become interoperable without pretending that every provider exposes the same message roles, token accounting, or model-visible request boundary? A useful standard would preserve source identity, versions, eligibility, transformations, trust, ordering, and final serialization while allowing provider-specific fields. [18][21]

2. 02

   How should an assembler optimize several constraints without collapsing them into an unjustified score? Relevance, dependency coverage, authorization, freshness, disclosure risk, latency, and token cost can conflict. Progress would require controlled, workload-specific policies that report each constraint and its failure modes separately. [30][35]

3. 03

   How much semantic and provenance loss accumulates across repeated compaction cycles? Existing work establishes that summaries can be lossy and variable, but a strong answer needs repeated-cycle experiments against an uncompacted authoritative record, including temporal relations, exceptions, permissions, and source links. [60][61]

4. 04

   How should systems measure context quality directly rather than infer it only from downstream task success? Useful progress would distinguish whether required information was present, whether its roles and versions were correct, and whether the model used it, without treating fluent output as proof. [76][30]

5. 05

   How can deletion and changed permissions propagate through source records, indexes, summaries, memories, caches, traces, and backups? Progress requires representation-specific ownership, durable tombstones or invalidation records, rebuild procedures, and evidence that each retained copy honored the change. [51][52]

6. 06

   How should context policies adapt when models change without turning every model-specific workaround into permanent harness complexity? Progress would pair replaceable policy components with model-version-specific evaluation, then remove resets, ordering tricks, or compaction rules when controlled evidence shows they no longer help. [77]