Purpose and development
Engineering around learned behavior
Traditional application code contains rules written directly by programmers. A machine-learning model instead derives some behavior from training data: it predicts or generates an output from patterns learned during training. Machine Learning Fundamentals develops that mechanism; here, the important consequence is that a permitted input can still receive an incorrect output.
That uncertainty does not make ordinary engineering irrelevant. The model may propose a category, message, or action, while application code decides whether the proposal is well formed, whether the caller is authorized, whether current state still permits it, and whether an external effect should occur. Code can enforce those responsibilities even though it cannot guarantee that every prediction is useful. Essential control flow, tool definitions, and information-flow restrictions therefore belong in explicit software rather than being left for a model to infer on each request.
Repeatability and correctness are separate properties. Repeating the same answer only shows stability under the tested conditions; correctness asks whether the answer satisfies the intended requirement. Model-containing systems also have dependencies that cross clean-looking code boundaries: data, configuration, feedback loops, and downstream consumers can make a model change affect behavior beyond its adapter. The practical goal is therefore not to pretend that uncertainty disappeared. It is to state each software guarantee precisely and collect separate empirical evidence for model behavior.
Must-know turning points
Several complementary lines of work shaped this approach. The dates below identify the cited publications or announcements, not universal invention dates, and their sequence does not imply that later practices replaced earlier ones.
| Date | Landmark | Contribution |
|---|---|---|
| October 1969 | An Axiomatic Basis for Computer Programming, C. A. R. Hoare | Connected assumptions about an initial state to properties established when a program terminates, including rules for composing program parts. |
| December 1972 | On the Criteria To Be Used in Decomposing Systems into Modules, David Parnas | Organized modules around design decisions they conceal, so callers need not change when a hidden representation changes. |
| September 2000 | Continuous Integration, Martin Fowler and Matthew Foemmel | Documented frequent integration with an automated build and test process, intended to expose interactions while changes remained small and recent. |
| 2015 | Hidden Technical Debt in Machine Learning Systems, D. Sculley and colleagues | Explained how data, configuration, feedback, and consumers can couple learned components beyond apparent code boundaries. |
| July 2023 | Introducing TypeChat, Microsoft | Applied TypeScript definitions and compiler diagnostics to checking model-generated records before application use. |
Interfaces and obligations
Specify what an operation promises
A type describes permitted values and operations. An interface exposes how another component can interact with a module. A type named SanitizedString, for example, is still an ordinary string unless some operation actually checks or constructs the promised property.
A precondition is an obligation placed on the caller before an operation begins. A postcondition is an obligation the implementation owes after successful completion. If one operation’s postcondition establishes the next operation’s precondition, the two can be composed without every caller knowing their internals. Hoare’s 1969 rules supplied a formal foundation for this reasoning; Bertrand Meyer’s Applying Design by Contract, published in October 1992, presented contracts as practical guidance for reusable components. These ideas do not require turning an application into a formal proof.
One interface, several obligations
| Element | Owner | Obligation |
|---|---|---|
| Input values | Caller | Supply an existing record identifier, the version used to produce the proposal, and a label from the declared set. |
| Acceptance condition | Implementation | Accept only if the authoritative record still has expectedVersion and the requested transition is allowed. |
| Successful result | Implementation | Store the label and advance the record version together. |
| Rejected result | Interface | Return an explicit reason such as invalid label, stale version, or unavailable storage. |
Static checking can reject incompatible program expressions before execution, but it cannot inspect a value that has not arrived yet. TypeScript, for example, erases ordinary type annotations when producing JavaScript. Runtime values from a model, HTTP request, database, or file therefore need runtime checks. Declared failures are also not automatically contract violations: an unavailable dependency can be an expected outcome represented by the interface, while silently returning malformed success violates the operation’s promise.
Validate external values
External bytes pass through several distinct questions. Parsing asks whether bytes can be interpreted as a value. Schema validation asks whether that value has the declared fields, types, required properties, and allowed values. Application rules ask whether the value is permitted in the current domain and state. Prediction assessment asks whether the model interpreted the input correctly. Each layer can reject a value, but passing one layer does not imply passing the next.
Each boundary establishes a narrower guarantee
Parsing, schema validation, domain acceptance, persistence, and task evaluation support different claims.
Read the diagram as text
- External bytes. A model or another external source supplies untrusted input.
- Parse. Interpret the bytes as a value or return a parsing failure.
- Validate schema. Check required fields, types, and allowed values; preserve explicit failure outcomes.
- Validated proposal. The record satisfies the checked runtime shape, but its meaning is not yet established.
- Apply domain and state rules. Check authorization, permitted transitions, and current authoritative state.
- Persist atomically. Commit the accepted mutation while preserving its state-transition contract.
- Authoritative accepted state. Software has established the promised state change, not the model’s general usefulness.
- Parsing failure. The bytes cannot be interpreted as the expected value.
- Invalid or unavailable result. Malformed output or an explicit dependency failure is not treated as successful data.
- Rejected operation. The proposal is disallowed, unauthorized, or stale for the current state.
- Representative labeled cases. Examples and conditions selected for the intended task.
- Task-specific oracle. Metrics or review criteria judge whether interpretations satisfy the task.
- Task-quality evidence. Observed evaluation results support bounded claims about model usefulness.
- External bytes → Parse: interpret.
- Parse → Validate schema: parsed value.
- Parse → Parsing failure: reported failure.
- Validate schema → Validated proposal: validated record.
- Validate schema → Invalid or unavailable result: explicit rejection.
- Validated proposal → Apply domain and state rules: candidate data.
- Apply domain and state rules → Persist atomically: accepted transition.
- Apply domain and state rules → Rejected operation: explicit rejection.
- Persist atomically → Authoritative accepted state: committed state.
- Validated proposal → Task-specific oracle: evaluated output.
- Representative labeled cases → Task-specific oracle: task conditions.
- Task-specific oracle → Task-quality evidence: assessment results.
Structure and meaning are different checks
| Returned value | Parses? | Matches schema? | Interpretation |
|---|---|---|---|
{"label": 7} | Yes | No | The label is not one of the permitted strings. |
{"label": "positive"} | Yes | Yes | Structurally acceptable, but inconsistent with the unambiguous complaint. |
{"label": "negative"} | Yes | Yes | Structurally acceptable and consistent with this input. |
{"status": "unavailable", "reason": "provider_timeout"} | Yes | Separate failure schema | No successful classification is claimed. |
The second record is the crucial case: a validator can establish that positive is allowed without establishing that it describes the input. Conversely, a provider timeout should not be converted into neutral, an empty label, or another fabricated success. The interface should preserve unavailable or rejected outcomes so downstream code cannot mistake absence of a prediction for a prediction about absence.
Microsoft’s TypeChat used the same TypeScript definitions to guide generated data and check it, feeding compiler diagnostics into a repair request when validation failed. That is a concrete boundary mechanism, not a guarantee that the model understood the request. Daniel Rosenwasser’s TypeChat talk demonstrates the approach. Structured Outputs and Tool Calling develops constrained generation, argument validation, and action handling in depth.
Boundaries, state and effects
Hide decisions behind useful boundaries
A module is a unit with a focused responsibility and an explicit interface. Information hiding places a changeable decision inside the module that owns it, so callers depend on the stable interface rather than the hidden representation. Cohesion groups logically related responsibility; limited coupling keeps one component from needing to know another component’s internal decisions.
In a model-based application, provider wire formats, application acceptance policy, and persistence often change for different reasons. A model adapter can translate provider responses into an internal result. Domain rules can decide whether that result is allowed. A storage adapter can make the accepted state transition. If a provider changes only its response envelope, the model adapter may contain the change. If the replacement model changes classification behavior, the internal interface may remain identical while broader evaluation is still necessary.
Boundaries contain knowledge, not all consequences
A provider-format change can remain inside the model adapter, while a behavior change can cross the same stable interface and still require broader evaluation.
Read the diagram as text
- Application coordinator. Sequences acceptance without owning provider or storage formats.
- Domain rules. Own permitted transitions and acceptance policy.
- Model adapter. Translates provider requests and responses into the internal result contract.
- Model provider. External inference service with its own wire format and behavior.
- Storage adapter. Owns persistence operations and database-specific behavior.
- Authoritative database. Stores accepted application state.
- Application coordinator → Domain rules: code: acceptance policy.
- Application coordinator → Model adapter: code: internal result contract.
- Application coordinator → Storage adapter: code: state interface.
- Model adapter → Model provider: data: provider wire format.
- Storage adapter → Authoritative database: data: persistence protocol.
Replaceable dependencies
Dependency injection means supplying a dependency to a consumer instead of constructing it inside the consumer. It creates an explicit replacement point for a real provider, a controlled test double, or another implementation; no framework or separate service is required. Large dependency graphs can nevertheless become hard to trace, so the provisioning path should remain visible. Matt Pocock’s software-fundamentals talk relates this to deep modules: substantial responsibility behind a simple interface. Depth is a relationship between functionality and interface complexity, not a target file size.
Decisions and effects
A pure function returns the same result for the same inputs and does not modify pre-existing external state. Gary Bernhardt’s July 2012 functional core, imperative shell pattern places value transformations and decisions in a testable core while a surrounding shell performs database, network, and terminal effects. This separation is useful when it clarifies ownership; adding shallow wrappers that merely forward calls creates navigation cost without hiding a meaningful decision.
Preserve state at acceptance
State is information retained between operations. A state transition changes that information. An invariant is a condition that must hold at the boundaries where the system claims to be in a valid state. An invariant may be temporarily false during an internal calculation, but every exported successful operation must re-establish it before making the new state visible.
Suppose a model proposes a label using record version 12. While inference runs, another operation changes the record and advances it to version 13. The proposal can remain well formed yet no longer be eligible for acceptance. Reading version 13 in a separate preflight check is insufficient if another writer can update the record between that check and the eventual write.
Optimistic acceptance
An atomic conditional update compares the expected version and performs the mutation as one database operation. PostgreSQL’s Read Committed behavior rechecks an UPDATE condition against a row changed by another transaction. The application can therefore update only where id and version = 12, increment the version in the same statement, and treat zero affected rows as stale-proposal rejection. Every relevant writer must participate by advancing the version. This single-row pattern does not establish arbitrary multi-row invariants.
Check and write must share one database operation
ExampleA preflight read can become stale before a later write; a conditional update rejects the proposal when its expected version no longer matches.
Both paths begin with proposal P tied to record version 12.
Read the diagram as text
- Proposal P · version 12. A label proposal bound to the state used during inference.
- Preflight reads version 12. The application observes a matching version before issuing a separate write.
- Concurrent writer creates version 13. Another accepted mutation advances the authoritative row.
- Later unconditional write. The separated write can apply P after its source version has become stale.
- Conditional UPDATE. Match record id and version 12; store the label and increment the version in the same operation.
- Database rechecks version 13. The authoritative row no longer satisfies the version-12 predicate.
- Zero rows affected. No state mutation occurs because the expected version does not match.
- Reject as stale. The application reports that P is no longer eligible for acceptance.
- Proposal P · version 12 → Preflight reads version 12: separate freshness check.
- Preflight reads version 12 → Concurrent writer creates version 13: intervening state change.
- Concurrent writer creates version 13 → Later unconditional write: stale write remains possible.
- Proposal P · version 12 → Conditional UPDATE: expected version supplied.
- Conditional UPDATE → Database rechecks version 13: database evaluates predicate.
- Database rechecks version 13 → Zero rows affected: version mismatch.
- Zero rows affected → Reject as stale: explicit result.
- Proposal targets version 12. Both paths begin with proposal P tied to record version 12. Active: Proposal P · version 12. New: Proposal P · version 12.
- Unsafe preflight succeeds. The unsafe path reads version 12, but the read and eventual write are separate operations. Active: Proposal P · version 12, Preflight reads version 12. New: Preflight reads version 12.
- Authoritative state advances. A concurrent writer commits version 13 after the preflight read. Active: Proposal P · version 12, Preflight reads version 12, Concurrent writer creates version 13. New: Concurrent writer creates version 13.
- Separated write is unsafe. An unconditional later write can apply the version-12 proposal after the row has advanced. Active: Proposal P · version 12, Preflight reads version 12, Concurrent writer creates version 13, Later unconditional write. New: Later unconditional write.
- Conditional update reaches the database. The safe path submits the record id, expected version 12, label mutation, and version increment as one update. Active: Proposal P · version 12, Concurrent writer creates version 13, Conditional UPDATE. New: Conditional UPDATE.
- Database rejects the stale proposal. The database rechecks the predicate against version 13, affects zero rows, and the application returns stale rejection. Active: Proposal P · version 12, Concurrent writer creates version 13, Conditional UPDATE, Database rechecks version 13, Zero rows affected, Reject as stale. New: Database rechecks version 13, Zero rows affected, Reject as stale.
Represent uncertain external outcomes
A side effect is an observable change outside a function’s returned value: writing a record, sending a message, charging a card, or publishing a file. A database transaction can group local changes so they commit together or roll back together, but it does not automatically include an unrelated remote API. The local record and the remote system therefore occupy different transaction boundaries.
A timeout creates partial failure when the caller lacks enough information to classify the whole operation. The remote service may have completed an effect while its response was delayed or lost. Cancellation likewise does not roll back completed changes. A useful result type distinguishes confirmed success, confirmed rejection, and unknown outcome; collapsing unknown into failure invites an unsafe duplicate operation.
Remote completion can coexist with caller uncertainty
ExampleA deadline describes what the caller observed, not whether the remote effect occurred.
The caller creates operation op-42 and sends it to the remote service.
Read the diagram as text
- Caller. Application component awaiting confirmation.
- Operation op-42. Stable logical identity retained across status checks or safe retries.
- Remote service. Authority over the external effect.
- Remote effect completed. The external change exists even though confirmation is absent.
- Caller outcome unknown. The deadline passed without authoritative confirmation.
- Caller → Operation op-42: creates identity.
- Operation op-42 → Remote service: request.
- Remote service → Remote effect completed: commits effect.
- Operation op-42 → Caller outcome unknown: deadline without confirmation.
- Send identified request. The caller creates operation op-42 and sends it to the remote service. Active: Caller, Operation op-42, Remote service. New: Caller, Operation op-42, Remote service.
- Remote effect completes. The remote service adds a completed effect for op-42; the caller has not received confirmation. Active: Caller, Operation op-42, Remote service, Remote effect completed. New: Remote effect completed.
- Deadline expires. The caller adds an unknown outcome. Remote completion remains visible as a separate fact. Active: Caller, Operation op-42, Remote service, Remote effect completed, Caller outcome unknown. New: Caller outcome unknown.
Safe repetition requires a contract
Idempotency means that repeating the same logical operation does not add another effect. It requires a stable operation identity and enforcement by the receiving boundary; merely logging a key is not enough. Stripe, for example, documents provider-specific behavior that stores the first executing request’s result for an idempotency key and rejects parameter mismatches. Other services can have different retention and concurrency rules. When the outcome is unknown, preserve the operation identity and reconcile against authoritative provider state before deciding whether another attempt is safe.
Simple synchronous code is easy to follow when success and failure are promptly confirmed. Explicit pending and unknown states add complexity, but they become necessary when external effects can outlive the caller’s observation. Durable retries, checkpoints, scheduling, and long-running recovery belong to Agent Runtimes and Harness Engineering.
Testing the right claims
Match checks to claims
A test oracle is the rule used to decide whether an observed result satisfies an expectation. The relevant distinction between software testing and model evaluation is the claim supported by that oracle—not whether the run uses a real model, a substitute, exact code, or a model judge. A software test can call a real model to exercise parsing. An evaluation can use deterministic code to score an exact task outcome.
| Claim | Exercise | Oracle | What remains unknown |
|---|---|---|---|
| Invalid data is never committed in these cases | Acceptance and storage path | Rejected result and unchanged authoritative state | Cases and invariants not exercised by the suite |
| The adapter handles the provider interaction | Serialization, response handling, and documented errors | Expected request and accepted or rejected response | Usefulness of the provider’s model answers |
| A selected user path is connected | Application wiring and user-visible boundary | Expected observable outcome | Other paths and production conditions |
| Classifications are useful for the intended task | Representative labeled cases, repeated where behavior varies | Task-specific metrics and review criteria | Performance outside the evaluated distribution |
A regression is an unintended loss of behavior or performance that was previously supported. A changed mean score, a failed assertion, or one different answer is not automatically a regression; the interpretation depends on the baseline, expected variability, and acceptance criterion. Nathan Sobo’s CI and stochastic-evals talk shows broad coding-agent failures being narrowed into focused stochastic checks and then conventional tests for deterministic surrounding defects. Reported run counts are local acceptance choices, not universal reliability guarantees.
Evals and Benchmarks develops representative task selection, graders, human assessment, uncertainty, and comparison. This chapter’s narrower concern is preserving the boundary: an application test supports the properties it asserts, while evidence about useful model behavior must cover the intended task and conditions.
Control dependencies in focused tests
A unit test is a focused check of a small unit’s behavior. A fixture provides preparation, cleanup, or stored test data; a suite groups checks; and a runner executes them and reports outcomes. Explicit dependencies let the test provide a model response, clock value, or external failure rather than waiting for a live system to produce the desired condition.
A test double replaces a real dependency. A stub supplies predetermined answers. A fake implements a simplified working version, such as an in-memory repository. These checks establish how the application responds to the supplied behavior; they do not establish that the real dependency will produce it, or how often it will do so. Focused real-implementation checks must cover that separate risk.
Test observable obligations
| Supplied dependency result | Expected application result | State assertion |
|---|---|---|
| Valid label for version 12 | Accepted | Label changes and version advances once |
| Malformed label record | Rejected as invalid output | Record remains unchanged |
| Provider timeout | Unknown or unavailable result | No fabricated label is stored |
| Valid label based on stale version 12 | Rejected as stale | Current version 13 remains unchanged |
Assertions should target public behavior callers rely on. Exact text equality is appropriate when exact text is the contract; it is brittle when punctuation or explanatory wording is irrelevant. Likewise, interaction assertions about internal call order should be used only when that order is itself an obligation.
Vary inputs, keep the oracle explicit
Property-based testing expresses an executable property and generates many inputs to check it. Koen Claessen and John Hughes introduced QuickCheck at ICFP 2000 to make controlled generation and executable properties practical in Haskell. A useful property here might say that any rejected proposal leaves the record unchanged. Generators, distributions, and the property itself can still be incomplete or wrong; finite successful cases are evidence, not a proof.
Exercise real integration boundaries
An integration test exercises collaborating components. A contract test checks that an implementation satisfies an agreed interaction. An end-to-end test exercises a selected complete path through the application. These categories overlap, but they answer different boundary questions. None replaces focused tests that localize failures cheaply.
Check both sides of an interaction
A substitute can encode an obsolete assumption while every isolated test passes. Pact illustrates a paired mechanism: consumer tests check the request a client sends and how it handles a supplied response; provider verification sends those expected interactions to the provider and compares the actual response. Passing those interactions does not establish complete workflow behavior or model-answer quality, and it does not imply that an arbitrary hosted model provider supports Pact.
Run checks according to the boundary that changed.
- Provider or SDK change — Exercise request serialization, streaming, response parsing, and documented error handling through the real adapter boundary.
- Persistence change — Exercise the real database’s constraints, transaction behavior, and conditional-update semantics with isolated test state.
- Application wiring change — Exercise a small complete path through the user-visible boundary, including the final persisted or rendered result.
- Model behavior change — Run task evaluations in addition to integration checks; protocol compatibility does not establish answer usefulness.
Higher-fidelity tests can detect configuration errors, stale assumptions, and missing connections, but they are slower, less isolated, and more dependent on shared infrastructure. Replit described visually plausible “painted doors”—for example, a button without a connected handler or a screen backed by mock rather than persisted data—and used programmatic and browser checks to expose them. The three pillars of autonomy talk is a concrete example of why code completion claims need outcome checks.
Reproduction and diagnosis
Replay the failed boundary
Investigating a model-containing failure involves two different reproduction goals. Regenerating asks the provider for another answer. Replaying supplies the saved historical answer to the downstream application. Replay can reproduce a parser, policy, or state-transition defect even when the provider cannot regenerate the original text.
Execution context for replay
Capture the inputs needed by the failed path, not merely the final error message.
- Dependency exchange — Relevant request and exact returned response, including the failure mode being investigated.
- Application state — The smallest initial state, clock values, and configuration required to reach the same boundary.
- Implementation identity — Code revision, instructions, schema, dependency versions, and relevant feature configuration.
- Model context — Model identifier and generation settings, retained as provenance rather than a guarantee of exact regeneration.
VCR.py provides a concrete HTTP record/replay mechanism: it records an interaction and later supplies the saved response for a matching request. Its none recording mode rejects unmatched requests instead of silently contacting the service. Applied to a model adapter, this makes the response a fixed fixture while preserving the normal downstream processing path. Database state, clocks, scheduling, and external mutations remain separate dependencies that must also be controlled.
A seed and fixed request parameters may improve repeatability but do not necessarily reconstruct a hosted response; an archived OpenAI example explicitly documented remaining variation even with a matching system fingerprint. Replayed fixtures can also contain private prompts, credentials, or user data. Filtering must be configured deliberately, must preserve the failure-inducing information, and does not by itself prove the fixture safe. Access should follow the sensitivity of the captured data. Observability covers collecting and connecting execution records in operating systems.
Test explanations of a failure
A symptom is an observation, not an explanation. Debugging proceeds by proposing a falsifiable hypothesis, choosing an intervention that distinguishes it from alternatives, and observing a result that could reject it. Recent changes help generate hypotheses, but temporal proximity does not prove causation. Diagnostic logging or resource changes can also alter latency and races, so the experiment itself may be a confounder.
A discriminating experiment
| Condition | Changed factor | Possible observation | Interpretation |
|---|---|---|---|
| Baseline replay | None | Failure repeats at the same boundary | The fixture reproduces the downstream symptom. |
| Replace only the suspected parser rule | Parser rule | Failure disappears and the expected state is produced | Supports the parser hypothesis, subject to regression checks. |
| Replace only the suspected parser rule | Parser rule | Failure remains unchanged | Contradicts the claim that this rule alone caused the symptom. |
| Call the live model again | Dependency response and possibly provider state | A different response succeeds | Does not discriminate between parser repair and changed input. |
Reduce without changing the failure
Failure reduction removes irrelevant input while preserving the same failure predicate. Andreas Zeller and Ralf Hildebrandt’s February 2002 paper, Simplifying and Isolating Failure-Inducing Input, describes ddmin, which repeatedly tests subsets until no single remaining element can be removed while retaining the failure. This yields a one-minimal case, not necessarily the globally smallest input and not an explanation of the defective code.
Once the defect is understood, preserve a focused regression test that fails before the repair and passes afterward. Then run broader relevant checks because a local fix can damage other behavior. A single improved model response still does not establish a model improvement; variable behavior requires repeated comparison under the evaluation design appropriate to the task.
Controlled change
Make changes reviewable
Version control records changes to files over time so particular versions can be compared and recovered. Refactoring changes internal structure while preserving observable behavior. A feature change deliberately changes behavior; a bug fix restores or establishes intended behavior. Separating substantial refactoring from behavior changes gives reviewers a clearer claim to examine.
A reviewable change is conceptually focused, not constrained by a universal line count. It includes the tests and usage context needed to understand its consequences while leaving the system working after integration. Review must inspect contracts, edge cases, concurrency, side effects, complexity, and test quality. Passing assertions are evidence only if they would fail for the broken behavior that matters.
| Change | Primary review question | Relevant checks |
|---|---|---|
| Domain rule | Does the new rule preserve stated invariants and failure behavior? | Focused unit and property-based tests |
| Provider interface or schema | Do both sides still agree on requests, responses, and failures? | Contract and adapter integration checks |
| Database transition | Are atomicity, compatibility, and concurrency assumptions valid? | Real database-boundary tests |
| Prompt or model | Does behavior improve without unacceptable regression on intended tasks? | Task-specific evaluations plus unchanged integration checks |
| Application wiring | Does the selected user path reach the visible or persisted outcome? | Complete-path test |
Continuous integration checks the combined revision frequently so interactions between changes are discovered while their causes remain recent. Fowler and Foemmel’s 2000 account required more than a daily build: developers integrated often, and failures were repaired or backed out. Matt Pocock’s software-fundamentals talk applies the same feedback principle to agent-assisted work: make a small change, check it, and reconsider the design before producing a large unverified batch. Test-first development is one useful practice, not a universal requirement.
Identify what was tested and released
A passing check describes the exact inputs and artifacts it exercised. A release artifact is a produced item used to run or configure the released application. Useful release identity therefore extends beyond a source commit to packaged code, configuration, instructions, schemas, dependency versions, and model selection. A moving label such as canary is not an immutable version.
| Input or artifact | Recorded identity | Evidence tied to it |
|---|---|---|
| Application source | Commit or immutable revision | Build, focused tests, and integrated checks |
| Packaged executable | Unique version and content hash | Release and system tests on that artifact |
| Prompt and policy configuration | Immutable configuration version | Behavioral evaluations and policy checks |
| Schemas and dependencies | Resolved versions | Compatibility and integration checks |
| Hosted model | Provider, model identifier, and platform | Task evaluations and availability record |
| Mutable runtime state | Current authoritative version | Compatibility and recovery checks; not part of the build artifact |
A reproducible build lets another party recreate identical specified artifacts from the same source, build environment, and instructions, commonly comparing bytes or hashes. Recording a source revision alone is insufficient because dependency versions, flags, and environment variables can affect the artifact. Identical artifacts still do not reproduce databases, clocks, remote services, or hosted inference.
Behavior-affecting configuration can change outside application code. A Braintrust workshop demonstrated changing managed model parameters without a code edit while still recommending version-control and synchronization discipline; see Shipping Complex AI Applications. The product details are historical, but the engineering point is durable: evidence must identify the configuration actually evaluated and released.
Exposure and recovery
Limit exposure and stop new work
A canary exposes a candidate change to a limited portion of use for a limited period before broader promotion. Candidate and control metrics should be examined separately because aggregate service metrics can hide a candidate failure. Population, duration, and load must exercise relevant conditions; there is no universal safe percentage or observation period.
A feature flag controls availability or selects a variant through runtime configuration. Define user-relevant promotion and stop conditions before exposure. A kill switch is a control intended to stop further selected behavior, but it works only at the decision points where the application evaluates it. Resolving a switch only when a session begins can leave an existing conversation unaffected until another session.
| Control | Decision affected | Required observation | Outside its reach |
|---|---|---|---|
| Canary allocation | Who receives the candidate | Candidate-versus-control outcomes under relevant conditions | Harm already caused to an exposed request |
| Variant flag | Which configured behavior runs | Behavior and failure measures by variant | Effects completed before the flag change |
| Kill switch | Whether selected future work may proceed | Activation, propagation, and decision-point enforcement | An already executing tool or completed external effect unless separately supported |
Runtime controls need drills and lifecycle ownership. Configuration changes can break a dormant kill switch; temporary rollout flags can become undocumented dependencies; interacting variants need combination testing based on valid configurations. Temporary rollout flags should have owners and removal conditions, while permanent emergency controls require continuing verification. The feature-flags talk supplies practitioner examples, but its suggested numeric thresholds are not universal limits.
Restore software without erasing effects
Rollback restores a previously identified software or configuration version. It requires that the earlier artifact still exists, its dependencies remain available, and it can operate safely on current persisted state. Amazon’s rollback-safety report illustrates a new writer producing data an original version cannot read: restoring that original executable would add failures rather than recover service.
Hosted dependencies create another limit. A recorded model identifier does not guarantee that the provider or every partner platform will continue serving it. Providers can deprecate and retire models on different schedules. A rollback plan that depends on an unavailable model is only a historical record, not an executable recovery path.
Recovery has separate responsibilities
Restoring software changes future execution; it does not erase a sent message, restore an overwritten record, or prove whether a timed-out payment occurred. Reconciliation determines the authoritative outcome and aligns local state with it. Compensation performs a new, business-specific action that counters completed work. Compensation is not database rollback: it can preserve intervening changes, incur fees, produce a different final state, and fail in turn. Irreversible effects need an acceptable remedy or human escalation rather than a claim that they were undone.
Rollback changes future execution, not history
An earlier release is usable only when it remains available and compatible with current state; completed effects need a separate recovery path.
Read the diagram as text
- Candidate release. The version whose behavior requires mitigation.
- Retained earlier release. Immutable artifact and configuration proposed for restoration.
- Available dependencies. Required runtimes, packages, and hosted models still exist.
- Current persisted state. Data as it exists after candidate execution.
- Restored serving. Earlier release running against current compatible state.
- Completed or uncertain effects. Messages, payments, writes, or other external outcomes already outside release control.
- Reconcile or remedy. Determine authoritative outcomes and compensate or escalate where supported.
- Candidate release → Current persisted state: left current data.
- Candidate release → Completed or uncertain effects: may have caused effects.
- Retained earlier release → Restored serving: condition: artifact retained.
- Available dependencies → Restored serving: condition: still available.
- Current persisted state → Restored serving: condition: state compatible.
- Completed or uncertain effects → Reconcile or remedy: corrective action.
- Restored serving → Reconcile or remedy: future work separated.
A justified recovery decision therefore combines three questions. Can further harmful work be stopped at the relevant decision boundary? Can a retained release run against today’s state and dependencies? Which completed or uncertain effects need reconciliation, compensation, or manual remedy? Treating these as separate obligations preserves the chapter’s central principle: software can make precise guarantees, but only within the boundaries those guarantees actually cover.
Open questions
How can teams specify semantic obligations precisely enough for strong automated checking without merely reproducing the same faulty implementation in the oracle? Progress would combine independently reviewable specifications, calibrated evaluators, and evidence that the checks reject realistic incorrect behavior.
How should test suites allocate effort between controlled substitutes, real provider boundaries, and complete application paths as providers and models evolve? The difficulty is balancing fidelity, cost, diagnostic clarity, and external-service instability; progress would produce risk-based strategies validated across real changes rather than a universal test ratio.
How can replay systems preserve enough failure-inducing context while minimizing sensitive data and preventing repeated external effects? Progress would provide tested minimization, access-control, and side-effect-isolation techniques that retain the original failure predicate.
How can rollback readiness be established when hosted models, mutable data formats, and irreversible external effects evolve independently? Progress would combine continuous compatibility checks, executable retained alternatives, and explicit reconciliation or remedy plans for effects software restoration cannot reverse.
