Contents
  1. Purpose and development
    1. Start with recurring engineering work
    2. Choose the service boundary
    3. Turning points in shared platforms
  2. Control and authority
    1. Connect configuration to running state
    2. Preserve tenant authority
  3. Model access and artifacts
    1. Specify usable model access
    2. Identify the release and its dependencies
    3. Promote and retire identified versions
  4. Deployment and capacity
    1. Control exposure and recovery
    2. Allocate usable capacity
  5. Reliable shared operation
    1. Promise outcomes at explicit boundaries
    2. Expose actionable diagnostics
  6. Supported development
    1. Make the supported path complete
    2. Measure usefulness and evolve the service
  7. Operating ownership
    1. Assign and exercise responsibility
  8. Check understanding
  9. Open questions
  10. Selected talks
  11. References
  12. Talk library
← All topics

AI Platform Engineering

An AI platform gives application teams a supported way to obtain model access, release identified software, and operate it. The central design problem is deciding which recurring work to share—and what the shared service must promise. A useful platform removes repeated integration work without hiding the limits or decisions that its users still need to understand.

Purpose and development

Start with recurring engineering work

AI platform engineering turns recurring engineering needs into supported shared capabilities. Its users are the people building and operating applications, including engineers working through agents. Start by identifying their tasks and constraints, then choose capabilities that help them complete that work. An inventory of infrastructure products is not yet a platform.

These applications contain models. Inference means applying a trained model to inputs, as explained in Machine Learning Fundamentals. Successfully executing that computation does not establish that its prediction or answer is useful. A platform can provide reliable execution and shared evaluation facilities while application owners determine whether the result meets their task requirements.

Separate recurring operational work from the decisions that make each application distinctive.
Recurring workPossible shared capabilityApplication responsibility
Obtain model accessApproved access paths and supported clientsSelect a suitable capability for the task
Release an identified versionArtifact registration and deployment machineryDefine behavior and accept task quality
Investigate failed workCorrelated operation records and diagnosticsExplain business-logic failures and user impact

Sharing replaces repeated implementation with a dependency on someone else's service. That exchange is worthwhile only if the service has an owner, usable interfaces, and support. Patrick Debois's platform-engineering account treats consuming teams as internal customers, with collaboration and assistance alongside APIs. Uber's Michelangelo shows how scope can grow from recurring needs. Development began in mid-2015 with model training and deployment, expanded into shared pipelines for preparing model inputs, and later emphasized developer productivity. A platform need not centralize every AI concern to be useful.

Choose the service boundary

A service boundary states what consumers can request and what the provider takes responsibility for. Specify supported users, interface behavior, owned state, operating limits, failure responses, and support. Not every reusable capability needs to become a remote service: shared code, project templates, and operated services solve different coordination problems.

The following comparison describes typical responsibility boundaries; a team's explicit support agreement can assign additional duties.
Delivery formUpgrades and stateEnforcement and incidents
Shared libraryMaintainer releases code; consumers adopt it and own application state.Checks run inside the consuming application. Its operators must ensure correct integration.
Project templateGenerator creates a starting project. Subsequent maintenance needs an explicit owner.Generated controls can be edited or become stale; scaffolding success is not operating assurance.
Operated serviceService owner maintains its implementation and service-owned state.Owner operates the promised enforcement boundary and handles service failures; consumers retain application duties.

Backstage Software Templates illustrates the template boundary. It creates projects from parameterized skeletons and exposes execution tasks and failure logs. Those facilities make creation inspectable; they do not establish ongoing deployment health or maintenance of the generated application.

A runtime manages when work executes and how its lifetime is controlled. The platform may offer and operate that runtime, but also supplies access, release, support, and ownership arrangements around it. Choose execution machinery from task lifetime and recovery requirements. Similarly, customer-specific implementation is not automatically a shared capability: Forward Deployed Engineering explains how to validate common needs before productizing a solution.

Buying an underlying component can transfer infrastructure work without transferring the whole consumer-facing promise. Managed services may operate machines and software while the customer retains data classification and permissions. Define that division explicitly rather than treating either self-hosting or managed hosting as a complete operating model.

Turning points in shared platforms

AI platforms combine several continuing traditions. Platform as a service supplies an operated environment for deploying applications. Machine-learning operations, or MLOps, encompasses developing, releasing, and operating machine-learning systems, including their data and model dependencies. Managed model APIs move some execution work outside the consuming organization. These approaches address different parts of the same delivery problem.

These developments widened the work a platform could support. Their dates identify particular systems and accounts, not a founding date for platform engineering.

Traditions assembled into AI platforms

  1. 2007SibylIntegrated ingestion, validation, training, analysis, and training-serving checks.Sources & context

    Contributors: Google; history documented by Konstantinos Katsiapis and colleagues.

    What changed: Combined modeling with production operations, but offered limited modeling flexibility. The retrospective describes TFX combining these workflows with TensorFlow’s more flexible programming model.

  2. April 28, 2010 — roadmap accountHerokuOrganized deployment, scaling, workers, and add-ons around complete developer tasks.Sources & context

    Contributors: Heroku; roadmap account by James Lindenbaum.

    What changed: External organizations could extend the platform through add-ons. This date identifies the account, not Heroku’s founding or initial launch.

  3. 2015Hidden Technical Debt in Machine Learning SystemsLocated production ML debt in dependencies, configuration, integration, and undeclared consumers.Sources & context

    Contributors: D. Sculley and Google colleagues.

    What changed: Explained why maintaining model code alone cannot make the surrounding application easy to change. A downstream consumer can break even when the model’s own tests pass.

  4. 2017TensorFlow Extended (TFX)Connected data processing, training, evaluation, and serving through reusable production components.Sources & context

    Contributors: Denis Baylor and Google colleagues; presented at KDD 2017.

    What changed: Replaced repeated custom integration and distinguished operational serving checks from prediction-quality checks. Internal launch occurred in 2017; public availability followed in early 2019.

  5. September 28, 2023 — general availabilityAmazon BedrockProvided access to multiple providers’ models without customer-operated serving infrastructure.Sources & context

    Contributors: AWS; general-availability announcement by Antje Barth.

    What changed: Shifted serving work to a managed service while discovery, access enablement, invocation, and application integration remained distinct responsibilities.

Expand for authors, primary sources, and context. These traditions continue alongside one another; spacing does not measure elapsed time.

The resulting platform can support trained models, hosted models, or both. What changes is the allocation of work, not the need to connect a release decision to a running application. That connection requires an explicit distinction between managing a service and executing its workload.

Control and authority

Connect configuration to running state

The control plane creates and changes resources and configuration. The data plane, also called the execution plane here, performs the service's primary work: processing requests or running jobs. These names classify operations, not necessarily separate products. Updating a gateway's configuration is management work; forwarding a model request is execution work.

Desired state is the accepted specification of what should run. Observed state records what the system currently reports. Reconciliation repeatedly compares the two and requests changes to close the gap. A controller performs this loop. In Kubernetes, a system for coordinating containerized workloads, the Job controller creates or removes Pod objects describing containers to run. Other components schedule and run those containers. Requesting the change and executing the work are separate responsibilities, so another observation is needed to determine whether the requested change took effect.

A deployment change can therefore be accepted before dependencies resolve, workers initialize, or the new version becomes eligible for traffic. Readiness is that eligibility, established through configured checks. Expose the operation's progress and failure reason instead of making callers infer completion from an administrative success response. Long-running operation guidance separates failures that prevent acceptance from failures during execution.

If initialization stalls, the accepted specification can name the candidate while existing traffic still uses an older release. Report the requested release, the release actually in use, and when that observation was made. This lets a user distinguish an unapplied change from a failure in the running application. Ray Serve's monitoring interface, documented here in a development version, separates goal configuration from observed application and deployment status, including the state of individual serving instances.

In this proposed architecture, accepted R2 is still loading while application traffic goes only to ready R1. Runtime and routing observations return to the controller for another comparison; they do not guarantee successful rollout or answer quality.

Preserve tenant authority

A tenant is a customer or organizational scope whose data, authority, and resource allowances require separation. A tenant may contain several applications and many human users. A workload identity identifies running software; it is not automatically the identity or authority of the human on whose behalf that software acts.

Authentication establishes identity; authorization determines permitted operations on particular resources. Derive tenant context from verified credentials or session state, not a model-proposed tenant identifier. Carry that context into resource lookup, storage, and caches. A cached answer remains a protected result: finding a matching key does not make disclosure permissible. Privacy and Data Governance develops the underlying authority rules.

Check current authority for model invocation, artifacts, execution records, caches, and diagnostics—including support, maintenance, and recovery. Platform entry or an administrator title grants no blanket access.

Uber's delegated workload-identity account preserves both human and agent actors through downstream requests and supplies supported clients to propagate that context. The platform lesson is to make attributable, scoped access the usable default rather than ask every team to recreate it.

Keep three isolation goals separate. Confidentiality concerns who can access data. Resource isolation concerns whether one workload consumes another's capacity. Failure independence concerns shared dependencies that can interrupt both. Kubernetes namespaces, access policies, and quotas address different pieces; quotas do not cover every resource, including network traffic. Dedicated compute can reduce resource sharing while leaving tenants dependent on the same management or artifact service.

Model access and artifacts

Specify usable model access

A model supplies learned behavior. A provider operates or supplies access to it. An endpoint is the concrete interface through which requests reach a serving implementation. A model gateway mediates those requests through supported interfaces and access policies. It can centralize authentication and attribution without choosing models according to answer quality.

Treat these as separate claims when making a model available.
ClaimWhat establishes it
DiscoverableA catalog identifies the capability and its supported use.
PermittedThe caller and intended processing satisfy applicable access policy.
AvailableThe actual service path can process the supported request.
SuitableApplication-specific evaluation supports the intended task.

Publish a capability record that joins model reference, provider, endpoint, accepted inputs, output forms, interface limitations, input and output limits, access scope, resource allowances, lifecycle status, and support owner. This is a proposed platform contract, not a universal catalog schema. It gives consumers enough information to request access and test the relevant path. A common JSON envelope can still contain model-specific parameters and behavior; preserve the request contract and check the concrete service path.

Managed inference transfers serving and capacity work according to the service contract; self-operated serving leaves more of that work with the platform team. Neither choice transfers application authorization or appropriate data use. Supplier review must also cover data handling, incident notification, continuity, and changes to the service. Internally hosted weights still have suppliers, dependencies, and provenance risks.

Uber's shared gateway illustrates centralized identity, policy middleware, and project-level request attribution. Aperture's documentation exposes another useful boundary: usage metrics can remain available without writing request bodies to disk, but shared bridge clients attribute several people's activity to one identity. Centralization makes controls reusable; it does not automatically make attribution complete.

Identify the release and its dependencies

An artifact is an identified item used or produced by a system: packaged code, model files, configuration, or an evaluation result. An artifact registry records identities, metadata, locations, and lifecycle information. It need not contain the large files themselves. Kubeflow's Model Registry, for example, records model versions and artifact locations while the files remain in their original storage.

A storage address says where to look, not which bytes will be found. A content digest is a value computed from content that can be checked against a trusted expected value. Provenance records origins and production history; dependency links identify other artifacts involved. Neither proves that an artifact is safe. The Update Framework combines authenticated metadata, hashes, version checks, and expiry because authenticity, content integrity, and freshness are separate properties.

A release manifest joins the dependencies needed to identify a release. Extend the software release identity with model artifacts or hosted references, input-processing configuration, runtime requirements, and evaluation records. At registration, check the required metadata, provenance, ownership, permitted use, storage access, and format before accepting the item for further validation. Some model-loading paths execute code; acquisition and inspection belong inside the artifact security boundary.

Version names require interpretation. MLflow distinguishes numbered versions from mutable aliases: changing an alias changes what a later alias-based load resolves, not necessarily what an existing process has loaded. Hosted identity is narrower still. Anthropic's versioning documentation distinguishes pinned weights and model configuration from serving infrastructure that can change observable behavior. Record provider assertions accurately without presenting them as deterministic replay guarantees.

A release is more than a model name

Example

Release dependencies, registry metadata, stored bytes, and supplier-controlled references establish different kinds of identity.

A proposed manifest structure. Model branches show two possible reference forms; a release may use either or both. Registry metadata references stored bytes. Hosted references rely on the supplier’s identity contract and do not guarantee deterministic replay. Assessment links record tested conditions, not every future execution.
Read the diagram as text
  • Release manifest. Identifies the assembled release and dependency references.
  • Packaged application. Exact code artifact and supported runtime requirements.
  • Input-processing configuration. Identified preprocessing and request-construction configuration.
  • Evaluation record. Tested release identities, conditions, and outcomes.
  • Registered model version. Metadata, expected digest, artifact location, and lifecycle record.
  • Model bytes in storage. Retrieved content checked against the trusted expected identity.
  • Hosted model reference. Provider, endpoint, and model ID; pinning has supplier-specific limits.
  • Release manifestPackaged application: Release dependency.
  • Release manifestInput-processing configuration: Release dependency.
  • Release manifestEvaluation record: Assessment reference.
  • Release manifestRegistered model version: Model reference: self-operated.
  • Registered model versionModel bytes in storage: Storage reference and expected digest.
  • Release manifestHosted model reference: Model reference: supplier-operated.

Dependencies also run downstream. A model's own tests can pass while an undeclared consumer depends on behavior the change removes—a central warning of the Hidden Technical Debt paper. Registration should therefore make releases discoverable to their consumers, while deployment records identify which versions those consumers actually use.

Promote and retire identified versions

Registration establishes a record, not permission for every use. Promotion makes an identified version eligible for a specified environment or purpose. The decision joins technical validation, application-owned quality acceptance, permitted-use conditions, and accountable approval. A model that loads within resource limits may still produce unsuitable predictions. Evals and Benchmarks explains the quality decision; the platform supplies the shared machinery that records and enforces it.

A lifecycle contract should distinguish these operations rather than use one status field as evidence for all of them.
OperationDecision or actionCompletion evidence
RegisterRegistry owner accepts identity and required metadata.Retrievable record and resolvable references
ValidateTechnical and application checks examine an identified version.Results tied to that version and test conditions
Approve and promoteAccountable owners permit a specified use.Scoped approval and eligibility in the target environment
DeprecateService owner announces a supported migration and retirement schedule.Known consumers, notice, and migration tracking
RevokeAuthorized owner withdraws permission for a use.Enforcement observed across affected consumers
DeleteRetention owner authorizes removal of specified records or copies.Confirmed disposition at each named storage boundary

Eligibility and deployment are independent dimensions. An approved artifact can be undeployed; a withdrawn artifact can still be loaded until enforcement reaches its consumers. SageMaker's approval workflow demonstrates the explicit connection: particular approval transitions trigger deployment pipelines when its documented project templates are used. Starting that pipeline is not proof that replacement finished.

Dependency lineage connects artifacts to the executions that used or produced them. ML Metadata records these relationships, enabling investigation beyond filenames. To retire a dependency, join such records with current deployments and unfinished work. Completeness depends on what integrations record. Provider retirement adds an external deadline: Anthropic documents that retired-model requests fail and recommends auditing remaining usage and testing replacements on application tasks.

Routine retirement allows migration planning; urgent withdrawal may require stopping use before a replacement exists. Retaining a recovery artifact does not authorize indefinite retention or future execution. Give retained copies a purpose, access restrictions, expiry or review condition, and disposal owner, following artifact lifetime policy. Removing a registry entry alone does not remove stored files or loaded copies.

Deployment and capacity

Control exposure and recovery

Deployment turns an eligible release into running service. Validate its dependencies and configuration, initialize it, and establish readiness before assigning traffic. A live process is not necessarily ready: it may still be loading model data. Readiness checks establish only their configured conditions, not application-answer quality.

A canary exposes a bounded portion of real use to a candidate while an existing version supplies a control. Compare version-specific results so fleet averages do not hide regressions. Define eligible traffic, observation duration, acceptable behavior, stop conditions, and the person or system authorized to promote. Small or unrepresentative samples can miss failures; there is no universal safe percentage.

Draining stops admission of new work while tracking work already accepted. Routing new requests elsewhere does not resolve requests already assigned to the deployment. Those requests may continue running; operators need to see whether they finish or require an explicit termination decision. Serving recovery distinguishes restored capacity from stream continuity; long-running version compatibility explains why unfinished work can require an older worker version.

Stopping admission leaves accepted work to resolve

Limited admission
Existing deployment

Control request → assigned here

Candidate deployment

C1 → admitted here

After candidate admission stops
Existing deployment

New permitted request → assigned here

Candidate deployment

C1 retained · ○ outcome pending

No new candidate admissions

The same candidate and C1 persist across both moments. The open endpoint marks a nonterminal request; it has not moved, completed, or been canceled. Spacing does not measure elapsed time.

C1 remains assigned to the candidate after new candidate admission stops. Track its outcome; stopping admission neither transfers it nor cancels or reverses its effects.
Document operational controls by their actual effect, not their button label.
ControlTarget and authorityEffect and confirmation
Stop new admissionNamed application, tenant, deployment, or service; authorized operatorBlocks new covered work; report pending and active work separately.
Disable agent or toolAgent or tool scope; authorized policy operatorTakes effect at supported decision points, including child agents; does not reverse completed actions.
Cancel operationIdentified operation; authorized callerRequests stopping under the runtime contract; inspect terminal status rather than assume acknowledgment means stopped.
Restore previous releaseAffected deployment or traffic assignment; release authorityChanges future execution after compatibility and eligibility checks; verify the effective release.

A control checked only at session creation leaves ongoing work outside later changes. Agents Need Feature Flags recommends checking kill switches at subsequent decisions and in spawned agents. Record who changed the control, its scope, when it changed, and where enforcement was observed. This makes the response inspectable without claiming interruption of an already executing tool.

Rollback restores an earlier software or configuration version, not an earlier world. Restoration requires compatibility with current state and dependencies, plus current permission. A withdrawn model may require forward repair; see rollback compatibility. Cancellation likewise does not undo committed remote effects. Preserve operation identities and investigate unknown outcomes separately. An already-delivered harmful utterance illustrates why some applications need stronger evidence before any live exposure.

Allocate usable capacity

Authorization permits work; it does not create resources. Admission control decides whether a workload may enter execution or a supported waiting state. The decision must account for the work's resource needs and the platform's commitments to other users. Request count alone can be misleading because requests differ in input size, output length, and downstream activity.

These limits and allocations answer different questions.
TermMeaningDoes not establish
QuotaAn allowance for a named resource and scopeThat hardware is idle or immediately available
Rate limitA bound on work admitted over timeA bound on all work still executing
Concurrency limitA bound on simultaneous workCompletion within a latency target
ReservationAn allocation held for work under a stated reservation contractPhysical placement or initialized workers
Ready capacityService resources able to execute the supported workload nowUnlimited throughput or suitable answers

Kubernetes Kueue provides a concrete distinction for run-to-completion workloads: quota reservation and configured admission checks precede execution, while scheduling places the actual workers. Physical-capacity checks depend on the configured mechanisms. Its batch-oriented contract should not be mistaken for a universal inference scheduler.

Classify work by waiting tolerance and occupancy. Interactive requests need prompt progress. Batch prediction handles accumulated inputs without an immediate-response requirement. Long-running jobs occupy resources or retain obligations across many steps. A noisy neighbor degrades another workload by consuming shared resources. Separate pools, quotas, or bounded queues can protect a class of work, but reserved separation also leaves less capacity freely available to others. Declare rejection and queue-expiry behavior instead of allowing indefinite waiting.

Scaling introduces delay before capacity becomes usable. Modal's cold-start documentation distinguishes waiting for a container from initialization, including model loading. Moving initialization earlier moves the wait rather than eliminating the work. Keeping containers warm can reduce startup exposure while consuming resources. A serving group is the cooperating set of workers needed for one complete model instance; a partially assembled group may add no executable capacity. See cooperating serving groups and paying for the required service.

Reserve readiness or share occupancy

Four identical slots; one non-preemptive job per slot. B1–B4 arrive at 0 and each need 8 time units. I1 arrives at 2 and needs 2. All work is authorized with sufficient quota.

Reserved slots finish initialization before 0; batch cannot borrow them. Shared slots start cold and initialize on demand for a waiting job, holding that slot until the job starts or expires; they then stay warm. I1 must start before 7; batch must start before 20. At a tie, completion and readiness update, waiting work expires, then eligible work starts: interactive first, then arrival order and B1–B4 order.

Same workload · observation window 0–24
OutcomeSelected: 1 reservedFully shared: 0 reserved
I1 waiting0 units · starts at 25 units · expires at 7
Batch completed by 244 / 44 / 4
Idle slot-time, 0–245352
Initialization slot-time, 0–24912
Prior warming slot-time30
Expired workNoneI1
Slot 1Interactive onlyI1 executing
Slot 2SharedInitializing for B1
Slot 3SharedInitializing for B2
Slot 4SharedInitializing for B3

Waiting: B1 (initializing), B2 (initializing), B3 (initializing), B4

Job starts and outcomes through 24
JobStartOutcome at 24
B13Completed at 11
B23Completed at 11
B33Completed at 11
B411Completed at 19
I12Completed at 4

One slot idle for one time unit is one unit of idle slot-time, not a currency estimate. Initialization occupies its slot even if the waiting job expires. Deadlines bound waiting only; started work can continue past 24.

Simplified capacity example, not a production scheduler or measured forecast. Reservations trade shared availability for interactive readiness; cold initialization and existing occupancy can consume the waiting allowance.

Reliable shared operation

Promise outcomes at explicit boundaries

A service-level indicator, or SLI, defines a measurement of service behavior. A service-level objective, or SLO, sets its target over a stated period. The error budget is the permitted shortfall, coupled to a policy for decisions such as slowing releases. Observability develops these measurements. Here the task is to choose boundaries that reflect what platform users need, rather than merely what individual components report.

These are proposed contract boundaries. Each requires an explicit target, aggregation period, supported conditions, and treatment of unresolved observations.
OperationEligible population and successful boundaryDependencies and owner
Provision serviceAccepted supported provisioning requests that become usable within the declared intervalConfiguration, artifacts, placement, initialization; platform service owner
Invoke modelEligible calls completing the supported response contract within the latency targetGateway, network, serving provider; model-access owner
Complete jobAccepted jobs producing the contracted result or explicit supported disposition by their deadlineRuntime, storage, downstream services; execution-service owner
Recover serviceDeclared failure scenarios restored to the agreed operating boundary within the recovery objectiveTrusted restoration assets and usable capacity; recovery owner

Keep unavailable observations visible and state exclusions before measuring. Server latency and client-observed completion can differ. Infrastructure availability, performance, answer quality, and end-user usefulness are separate outcomes: a successful model response can still be wrong, and healthy servers can sit behind a broken access path.

A failure domain groups components vulnerable to a common failure; its blast radius is the affected scope. Shared configuration, artifact storage, or provider accounts can couple otherwise separate applications. Inventory those dependencies and state which recovery actions need them. Static stability means continuing existing operation without management changes during a dependency failure. Pre-existing capacity and configuration can support continuity when provisioning is unavailable.

Outages can hide permission withdrawals. Declare authorization-freshness limits and stopping behavior; availability cannot justify indefinite stale permission. Recovery commitments should identify trusted restoration inputs, verification steps, acceptable loss of retained progress, and who confirms restored service.

Separate tenant compute retains shared initialization dependencies. Already loaded work can continue only with sufficient resources and the declared authorization-freshness contract; new initialization remains blocked while those dependencies are unavailable.

Expose actionable diagnostics

Diagnostics are part of the service interface. A trace connects records from one logical execution; trace instrumentation supplies the mechanics. Platform consumers need to determine which configuration took effect, where work is waiting, which dependency responded, and who can resolve the problem. A dashboard of aggregate traffic does not answer those questions for one operation.

A proposed operation record can expose the following fields without copying the entire request payload.
Field groupMeaning
Tenant and operation identityAuthorized scope and stable handle for investigation; not a grant of access
Requested and effective releaseWhat was accepted versus what the observed worker applied
Observation time and coverageWhen status was observed and which components remain unobserved
Phase and waiting reasonQueueing, initialization, execution, or an identified blocked dependency
Applied limit and dependency outcomeThe relevant policy decision, downstream response, or explicit unknown result
Support owner and restricted evidenceWhere to escalate and how authorized support obtains additional detail

Requested identity and returned identity should remain distinct. OpenTelemetry's developing generative-AI conventions provide separate requested-model and response-model attributes, along with prompt-version fields. They do not automatically identify application code, policies, retrieval indexes, or every workflow attempt; those require explicit instrumentation. A missing response-model field means identity is unavailable at that boundary, not that the requested model has been independently verified.

Likewise, a gateway can record a model's proposed tool call without proving the tool executed. The network-gateway demonstration shows inspection of traffic passing through that gateway; local actions and effect receipts require other observations. Distinguish platform failure, supplier failure, application-quality failure, and incomplete evidence before choosing an owner.

Use metadata-first diagnostic defaults. When sensitive inputs or outputs are necessary, define their purpose, readers, retention, and disposal separately. Support should receive an operation identity and scoped evidence, not unrestricted tenant access. Auditability is a reason to retain sufficient decision evidence, not permission to retain every payload.

Supported development

Make the supported path complete

A golden path, also called a paved road, is a supported way to complete recurring work with suitable defaults and integrated controls. It can span APIs, command-line tools, documentation, and human decisions; it does not require one portal. Its value comes from completing the task without making consumers assemble disconnected building blocks.

The workflow begins with discovery and access, continues through development, testing, registration, and deployment, and remains supported through inspection, stopping, and recovery. Automate repeatable handoffs, but preserve necessary approval decisions. Self-service removes dependence on a particular person's availability or unwritten knowledge; it does not grant every requested action.

Bloomberg's engineering-enablement account combines discovery with creation, deployment, runtime, and authentication support. Its directory helps teams find existing Model Context Protocol, or MCP, servers before building duplicates. MCP provides reusable capability exchange; the surrounding platform still owns the supported delivery and operating path.

Local checks establish behavior only under their tested conditions. Deployment changes identity, network access, secrets, persistent services, hardware, and concurrency. Exercise those integration boundaries rather than treating a generated project or passing local test as production readiness.

A supported extension path lets a team meet a need the default cannot serve while retaining mandatory authority boundaries. Record the extension interface, maintenance owner, support limits, and review requirements. Standardize operational obligations where possible, not every implementation choice. Amplifon's registry-and-blueprint design, still under development when presented, put authentication, cost tracking, and diagnostics into shared project templates. Teams could implement agent logic in different frameworks behind the templates' common interfaces. This separates freedom over implementation from responsibility for operating the service.

Measure usefulness and evolve the service

Access shows that someone can use a platform. Mandatory traffic shows policy compliance. Voluntary adoption suggests a choice; sustained use suggests the capability has entered recurring work. None alone establishes improved outcomes. Evaluate a bounded set of developer tasks against a credible prior workflow, combining instrumented use with structured feedback. The CNCF maturity model recommends both and warns that greater maturity consumes additional staff time and funding.

Define these measures for the pilot rather than treating them as a universal scorecard.
Intended improvementMeasure against the prior workflowKeep visible
Faster deliveryElapsed time per comparable task to a working deploymentTask complexity, approval waits, and failure rate
Less support dependenceWaiting time and support effort per completed taskHidden manual intervention and unresolved requests
Easier recoveryOperator effort and time to verified restorationIncident severity and unresolved work
Less repeated integrationWork needed to adopt a shared capability in another applicationPlatform maintenance and exception-support effort

Specify eligible tasks, observation periods, and team coverage before comparing results. Uninstrumented teams are unmeasured, not zero. A before-and-after improvement may reflect easier work, better staffing, or a provider change rather than the platform itself. Investigate support requests and exceptions: repetition can reveal a missing capability, a confusing interface, or an abstraction that excludes legitimate work.

Avoid rewarding activity independently of completed work. The Software Engineering + AI discussion reports anecdotes of engineers increasing token consumption to improve visible usage rankings. Tokens are the units models use to represent text; Tokenization explains why processing more of them measures usage, not accomplishment. More commits can similarly include repair work rather than useful functionality. Surveys can reveal satisfaction and friction, but self-perceived speed is not a substitute for measured delivery outcomes.

Evolution also changes obligations to existing consumers. Define stability levels, supported lifetimes, notice, migration assistance, and retirement procedures. Google's AIP-181 ties stable interfaces to support and deprecation commitments, distinguishing producer-initiated changes from consumer migration to a new version. It allows exceptional security or regulatory changes; it does not promise frozen model behavior. Use that distinction to communicate what remains stable and what users must retest.

Operating ownership

Assign and exercise responsibility

An operated capability needs people authorized and able to maintain it. Team Topologies distinguishes team responsibilities and interaction modes rather than prescribing one infrastructure hierarchy. Its service-discovery guidance describes close collaboration while needs and boundaries are uncertain, followed by service consumption through clearer interfaces. Those interfaces include response expectations, status information, support channels, and assistance. Facilitation helps teams learn to use the capability without transferring their application decisions.

Assign named owners to this responsibility pattern; one organization may combine roles, but the decisions remain distinct.
DecisionDecision authorityImplementation and confirmation
Technical release eligibilityPlatform release ownerPlatform operators validate dependencies and observe applied release and readiness.
Task-quality acceptanceApplication ownerApplication team supplies task-specific evaluation and accepts behavior.
Permitted use and withdrawalDesignated governance or resource ownerAccess administrators enforce the decision and report affected consumers.
Capacity commitmentService owner with spending authorityInfrastructure team or supplier provisions; operators verify usable capacity.
Service restorationIncident lead and affected service ownersPlatform and supplier operators repair their boundaries; owners confirm recovery.
User communicationNamed incident or service communications ownerSupport reports impact, restrictions, progress, and confirmed restoration.

A runbook describes the conditions, actions, expected results, and escalation for an operational procedure. A production-readiness review establishes agreement that the service can meet its operating obligations. Neither is complete merely because a document exists. Practice realistic failures with actual tools, inspect what failed, and update the procedure. Google's incident-response guidance emphasizes prepared command, communications, and operations roles, exercises, and confirmation from affected service owners.

A managed supplier can own the underlying infrastructure while the platform team owns the service offered internally. Keep supplier escalation and change obligations alongside application and governance responsibilities. The consumer should not have to reconstruct the organization's outsourcing arrangements during an incident.

A bounded platform service contract brings the chapter's decisions together.

  • Users and workName the consumers, recurring tasks, and application decisions the platform does not take over.
  • Interface and stateSpecify supported requests, owned records, release identity, and observable completion boundaries.
  • Authority and limitsDeclare tenant scope, protected access paths, resource allowances, readiness, and overload behavior.
  • Change and recoveryDefine promotion, exposure, draining, withdrawal, compatibility, retention, and unresolved-outcome handling.
  • Operation and improvementProvide diagnostics, support, accountable owners, exercised recovery, and measurements of completed useful work.

Open questions

  1. Management-outage contracts still need testing: permitted operating duration, authority freshness, and stopping when those conditions fail.

  2. A pinned hosted-model reference does not identify every serving behavior that influenced an answer. More useful reproducibility would combine provider change disclosure, observable serving identity, and application regression evidence without claiming access to hidden implementation state.

  3. Dependency records become less useful when consumers appear through dynamic integrations or bypass the registration path. Progress would make missing coverage and stale relationships visible enough to support retirement decisions, rather than merely drawing a complete-looking catalog.

  4. Shared capacity must accommodate unpredictable job duration without sacrificing interactive service or wasting reserved resources. Useful progress is workload-specific evidence of completed work, waiting, fairness, and operating expense under realistic bursts—not a universal utilization target.

  5. Platform usefulness is hard to attribute when adoption coincides with changing models, staffing, and tasks. Stronger assessments would compare defined work across credible alternatives while including support effort, recovery, and unmeasured populations.

Follow the curated reading path through the speakers and demonstrations behind this entry.

28 min

AI Engineer World's Fair 2024 · 2024

AI Platform Engineering

Patrick Debois

Cited in this entry

Establishes the internal-customer relationship and the division between shared capabilities and application-team work.

Watch talk
19 min

AI Engineer World's Fair 2026 · 2026

Agents Need Feature Flags

Sachin Gupta

Cited in this entry

Explains why runtime controls must reach ongoing decisions and child agents rather than operate only at session creation.

Watch talk

Explore more talks

The rest of the library, beyond the curated path. Cited talks support this entry; reviewed transcripts were processed in full. Metadata candidates have not been reviewed as sources or verified as topic members.

26 matching talks

Every catalogued talk on this subject: Infrastructure and deployment

TalkSpeakerEventYear
Christopher Lovejoy, Saul HowardAI Engineer World's Fair 20262026
Nishant GuptaAI Engineer World's Fair 20262026
DottaAI Engineer Europe 20262026
Remy GuercioAI Engineer Europe 20262026
Harshil AgrawalAI Engineer Europe 20262026
Lovina DmelloAI Engineer World's Fair 20262026
AI SDK v6

Transcript reviewed

Nico AlbaneseAI Engineer Europe 20262026
Gabriel Jorge MenezesAI Engineer World's Fair 20262026
Conquering Agent Chaos

Transcript reviewed

Rick BlalockAI Engineer World's Fair 20252025
Building security around ML

Transcript reviewed

Dr. Andrew DavisAI Engineer World's Fair 20242024
Carter Abdallah, Vincent Weisser, Lucas Atkins, Chris AlexiukAI Engineer World's Fair 20262026
Andrew ThompsonAI Engineer World's Fair 20252025
Sandipan BhaumikAI Engineer Europe 20262026
Anju KambadurAI Engineer Summit 20252025
Jia WuAI Engineer World's Fair 20262026
Fryderyk WiatrowskiAI Engineer Europe 20262026
Omar KhattabAI Engineer World's Fair 20252025
Kyle MisteleAI Engineer World's Fair 20262026
Sharmila Chokalingam, ShubhiAI Engineer World's Fair 20242024
Charles FryeAI Engineer Summit 20232023
Jared JoselowitzAI Engineer World's Fair 20262026
Dylan PatelAI Engineer World's Fair 20242024
AI Engineer Summit 20252025
Gergely Orosz, swyxAI Engineer Europe 20262026
Yegor Denisov-BlanchAI Engineer World's Fair 20252025
Lei ZhangAI Engineer Code 20252025

References

Coverage and source review
Processed transcripts
31 processed in full · 5 in the curated path
Automated source review
Passed
Metadata candidates
0 unreviewed; not verified topic membership
Corpus version
1bd8e407b26a07b33815594e1b2db5f41827119a2b3cb6fbf240f9fc571fc767

Automated review checks source support; it is not publication approval.

A synthesis of selected conference talks and technical references. Citations link to the source material; they do not imply that every talk on this subject is included.

  1. CNCF Platforms White Paper

    A platform presents integrated capabilities around its users' needs. CNCF recommends prioritizing common use cases, providing consistent self-service interfaces and documenting supported workflows. A golden path combines a reusable workflow with templates and documentation. Platforms should be composable, allowing teams to use selected capabilities and operate additional capabilities when necessary. Platform teams own interfaces and user experience but need not operate every underlying service; existing internal infrastructure and managed providers can supply implementations. User research and feedback guide capability selection and evolution.

  2. TFX: A TensorFlow-Based Production-Scale Machine Learning Platform

    Denis Baylor and Google colleagues presented TFX at KDD 2017 to replace repeated, custom production integration with shared components for data analysis, transformation, validation, training, model evaluation and serving. The paper distinguishes whether a model is safe to serve—such as loading successfully within resource constraints—from whether its predictions meet the application's quality requirements. It reports reducing time to production from months to weeks and describes deployment in Google Play. Shared lifecycle machinery therefore addressed operational work beyond choosing a learning algorithm.

  3. AI Platform Engineering

    Centralize company-appropriate model access, reusable model repositories, and shared data connectors; a vector database alone does not provide a usable RAG platform.

  4. AI Platform Engineering

    Shared observability should capture multi-step prompt activity and run ongoing production evaluations, rather than relying solely on endpoint availability.

  5. Google SRE: The Evolving SRE Engagement Model

    Google describes a shared-responsibility model in which SRE supports common platform infrastructure while development teams remain on call for application functionality and business-logic defects. Shared infrastructure includes traffic management, overload protection, logging and monitoring. Accepting production responsibility involves agreement on service objectives, staffing, necessary improvements and training rather than simply handing over code.

  6. AI Platform Engineering

    Use Team Topologies to distinguish service delivery, collaboration, and facilitation instead of treating the platform solely as an infrastructure handoff.

  7. Meet Michelangelo: Uber’s Machine Learning Platform

    Mike Del Balso and Jeremy Hermann's September 5, 2017 account dates Michelangelo's development to mid-2015. Uber previously lacked a standard experiment store and production deployment path; teams built bespoke serving containers. Work began with scalable training and deployment, expanded to shared feature pipelines, and then emphasized developer productivity. The resulting platform supported managing data, training, evaluation, deployment, prediction and monitoring. UberEATS delivery-time prediction illustrates actual use: application services requested predictions from deployed models using historical and recent restaurant features. The account explicitly connects its motivation to Sculley and colleagues' ML anti-patterns.

  8. Modern Software Delivery — Team Topologies mini-book

    Team Topologies describes close collaboration between platform and product teams while discovering a service's needs and boundaries, followed by less coordination once a clear interface allows consumption as a service. Its guidance makes support part of that interface: response times, status information, communication channels, on-call assistance and office hours. It also distinguishes installing Kubernetes from providing an internal platform; recommended practices, defaults and missing developer-facing capabilities still need definition.

  9. Backstage Software Templates

    Backstage Software Templates load code skeletons, substitute user-provided variables and publish results to locations such as GitHub or GitLab. Users review inputs before execution; each execution has a task identity and failed-step logs. Starting over creates a new execution with reusable parameters. Cancellation prevents subsequent steps, but cancellation of the current step depends on that step's support. This is a concrete self-service project-creation workflow with visible execution and failure states.

  10. How Forward Deployed Engineering is done at Cognition

    Use customer problems as a field-derived evaluation set, and distinguish recurring enterprise needs from individual exceptions before promoting workarounds into features.

  11. AWS: Shared Responsibility Model

    Operational responsibility depends on the service abstraction. AWS documents that customers operating virtual machines manage guest operating systems, patches and installed applications, while abstracted services transfer more infrastructure and platform operation to the provider. Customers still manage their data classification and permissions. AI deployment inference: self-hosting assigns the operator more responsibility for inference software, capacity, patching and availability; managed inference transfers some of that work under service-specific terms without transferring responsibility for application authorization or appropriate data use.

  12. Update and Roadmap: Huge Growth and Future Plans

    In an April 28, 2010 account, Heroku's James Lindenbaum describes a platform roadmap organized around complete developer use cases. Git-based application deployment came first, followed by caching and scaling, background workers, and add-ons integrating external services. Add-ons let other organizations extend the platform beyond what its own team could implement. This provides an early concrete example of self-service delivery and incremental platform scope preceding contemporary AI platforms.

  13. Towards ML Engineering: A Brief History Of TensorFlow Extended (TFX)

    Google's Konstantinos Katsiapis and colleagues trace Sibyl to 2007. It integrated data ingestion, validation, training, model analysis and training-serving skew detection, but its modeling flexibility was limited. TensorFlow supplied a more flexible programming model without a complete production workflow. TFX combined these concerns, launching internally in 2017; the public offering followed in early 2019. The authors report that most Sibyl workloads migrated to TFX before Sibyl's retirement. This history distinguishes a modeling framework from the surrounding platform that makes models operable.

  14. Amazon Bedrock Is Now Generally Available – Build and Scale Generative AI Applications with Foundation Models

    Antje Barth announced Amazon Bedrock's general availability on September 28, 2023, following an April announcement. The service exposed multiple providers' foundation models without customers operating the serving infrastructure, with integrations for usage monitoring and API activity records. Its examples separated model discovery and access enablement from runtime invocation. A shared invocation envelope still contained model-specific prompt formats and parameters. Agents and knowledge bases remained in preview at this announcement. This illustrates a platform responsibility shift toward managing access and application integration around externally operated models.

  15. AWS Advanced Multi-AZ Resilience Patterns: Control planes and data planes

    A control plane creates, modifies and deletes resources and propagates configuration changes. A data plane performs the resources' primary work, such as running an instance or reading and writing database items. AWS distinguishes recovery actions performed through the data plane from subsequent control-plane reconfiguration.

  16. Kubernetes: Controllers

    Controllers observe current state and make or request changes to bring it toward the desired state recorded in an object's specification. The Job controller creates or removes Pod objects through the API; other components schedule and run them. The controller does not execute those containers itself. This separation makes a recorded request for work distinct from execution of that work.

  17. Google AIP-151: Long-running operations

    Long-running APIs return an operation that clients can inspect later. Metadata can report progress and partial failures. Errors preventing execution are returned immediately, while execution failures belong in the operation's error field. A resource being created can appear in reads while explicitly remaining unusable. Accepting multiple operations does not require executing them concurrently; a service may queue them. Completed operation records may eventually expire.

  18. Kubernetes: Liveness, Readiness, and Startup Probes

    Startup, liveness and readiness probes answer different questions. Startup probes allow initialization before other probes run; liveness failures can trigger restart; readiness controls eligibility for service traffic. Readiness can include loading data or checking required dependencies. Deleting a Pod marks its endpoint unready for regular traffic. Incorrect liveness checks can restart overloaded containers and amplify failures.

  19. Monitor Your Application — Ray Serve

    Ray Serve distinguishes the latest received configuration from observed operation. The serve config command returns the application's goal state; serve status reports application and deployment status, replica-state counts and contextual messages. The dashboard links replicas, proxies and controllers to their logs. Documented proxy states distinguish draining, which rejects new requests while pending work remains, from drained, which has no pending requests. These interfaces let developers investigate whether requested changes progressed to running service and where failures occurred.

  20. Kubernetes: Multi-tenancy

    A tenant can represent an internal team or an external customer; its meaning depends on the deployment. Kubernetes separates access controls from resource-sharing controls. Namespace-scoped authorization restricts access, while quotas constrain resource consumption and object creation. A noisy neighbor is a tenant whose activity degrades other tenants' workloads. Quotas reduce this risk but do not cover every shared resource, including network traffic. Containers exceeding resource limits can be throttled or killed depending on the resource.

  21. Uber Engineering: Solving the Identity Crisis for AI Agents

    Uber describes binding registered agents to SPIRE-backed workload identities. A security token service exchanges credentials at each hop for short-lived tokens addressed to the next recipient while preserving the human and agent actor chain. Recipients verify signatures and audiences; the MCP gateway applies tool-access policies. A standardized client automates token exchange and identity propagation, and existing integrations are migrated with support and testing guidance.

  22. OWASP Access Control

    Authentication establishes identity; authorization decides which actions that identity may perform on particular resources. A user allowed to initiate a transfer must still be authorized for the source account. Least privilege limits the authority of running code and service accounts, while centralized checks reduce inconsistent enforcement. In an AI application, tool availability and a model-produced argument are therefore insufficient grounds to execute a business operation; the application must apply resource- and action-level policy.

  23. OWASP Multi-Tenant Security: tenant context and resource access

    Tenant context should originate in verified authentication claims or session state, not an unvalidated header or model-proposed tenant ID. Validate tenant status, propagate that context through application layers, and reset request-local context after use. Resource lookup combines tenant identity with resource identity and enforces authorization at the data-access layer. Row-level database policies and tenant-scoped repositories can prevent a valid caller from selecting another tenant's record merely by changing its ID. Cache keys and storage access need the same separation.

  24. The Protection of Information in Computer Systems: Basic Principles

    Least privilege limits each user and program to permissions needed for its job. Complete mediation requires authority checks on every access to every object, including initialization, recovery, shutdown, and maintenance. It requires reliable identification of request sources and care with cached authorization when permissions change. Fail-safe defaults base access on explicit permission. Applied to a harness, these principles imply that protected operations must pass through an enforcement mechanism that the requesting program cannot bypass or modify; a prompt instructing the model to behave is not that mechanism.

  25. AWS Fault Isolation Boundaries: Static stability

    Static stability means continuing operation without changing state when dependencies fail. AWS describes data planes retaining existing state during control-plane impairment and recommends avoiding control-plane dependencies in recovery paths. Pre-provisioned spare capacity can support recovery without requiring new instances to be created during the incident.

  26. Tailscale: How Aperture works

    Aperture mediates model requests by identifying the caller, looking up the requested model's provider and forwarding the request with provider authentication headers. Its documentation distinguishes supported API formats rather than promising one universal protocol. Shared bridge instances attribute multiple people's activity to one identity. Request and response capture can be configured so bodies are never written to disk while usage metrics remain available.

  27. OpenRouter: Provider Routing

    OpenRouter documents that providers can ignore unsupported request parameters under default routing. Setting require_parameters restricts routing to providers supporting all supplied parameters. Provider ordering alone can still permit fallback to other providers; disabling fallbacks changes that behavior. Separate routing controls restrict providers according to data-collection policy. Provider base names can match multiple endpoint variants or regions.

  28. One Registry to Rule them All - Sonny Merla, Mauro Luchetti, & Mattia Redaelli, Quantyca

    Amplifon's private MCP registry extends the community registry with curated entries and enterprise metadata for accountability and impact analysis.

  29. NIST Generative AI Profile: Third-Party Risk

    NIST recommends inventorying third parties with access to organizational content, assessing suppliers and specifying security, ownership, usage and provenance expectations in contracts. Examine secondary data use, incident notification, service continuity and responsibility for system changes. These concerns apply to proprietary and open-source components, fine-tuned models and embedded tools. Hosting inference internally does not remove risks from acquired weights, training data, libraries or external integrations. Evaluation, access governance, monitoring and incident ownership remain organizational responsibilities.

  30. Agentic SDLC at Uber - Building Blocks for Uber’s Software Factory

    Uber places request attribution, policy middleware, and audit-session capture in a shared Model Gateway.

  31. Model Registry — Kubeflow SDK

    Kubeflow's Model Registry stores model metadata and artifact locations, while model files remain in their original storage. A registered model is a named entity with versions; each version has metadata and an associated artifact URI. Clients retrieve the location by model name and version, and model-format metadata helps KServe select a serving runtime. The published example registers an S3 location rather than uploading weights into the metadata registry.

  32. PROV-DM: The PROV Data Model

    Provenance records the entities, activities and responsible agents involved in producing or delivering information. Entities include files and document versions; activities use and generate entities; derivation connects an output to an earlier entity through transformation, update or construction. These relationships provide a concrete representation for source-to-derived-data lineage. PROV distinguishes a particular document version from a resource identifying its changing latest version and warns that provenance descriptions must remain valid as resource state changes.

  33. The Update Framework: authenticated update workflow

    TUF starts from trusted root metadata specifying authorized keys and signature thresholds. Clients verify signed timestamp, snapshot, and targets metadata, then verify downloaded targets against authenticated hashes and lengths. Signatures and target hashes resist unauthorized substitution. Persisted metadata versions allow rejection of older metadata, resisting rollback; cross-role version checks also prevent inconsistent combinations. Expiration checks against the fixed update-start time detect attempts to keep clients indefinitely on stale metadata. These mechanisms complement one another: a valid signature alone does not prove freshness.

  34. MLflow: Model Registry Workflows

    MLflow registers logged model artifacts as numbered model versions with metadata including their source run, signature and creation time. Clients can load an explicit version or resolve a mutable alias. Reassigning an alias independently of application code changes which version the next alias-based load obtains. Separate registered models and access controls can represent development and production environments; tags can record validation status.

  35. PyTorch Security Policy: using models securely

    PyTorch treats models as programs and warns that running an untrusted model can be equivalent to running untrusted code. It recommends separating weights from Python code, checking provenance, and using an isolated environment. Serialization format changes the attack surface: restricted weight formats support fewer behaviors, while flexible loading has more ways to execute or process unsafe content. Even a safer format does not eliminate downstream input-validation risks, and TorchScript inspection utilities may execute code. Model acquisition and inspection therefore belong in the software supply-chain threat model.

  36. Claude Platform: Model IDs and versioning

    Anthropic distinguishes pinned model IDs from convenience aliases. Its documentation states that an existing model ID retains its weights and model configuration, while earlier convenience aliases can resolve to newer dated snapshots. The surrounding serving infrastructure—including request routing, safety classifiers and sampling logic—can change and produce observable differences even when the model ID and weights remain unchanged.

  37. Hidden Technical Debt in Machine Learning Systems

    D. Sculley and Google colleagues argued in their 2015 paper that production ML accumulates system-level debt through data dependencies, configuration, glue code and undeclared consumers of model outputs. A downstream consumer can make a model change hazardous even when the model's own tests pass. The authors recommend mechanisms for identifying dependencies, managing configuration and reducing package-specific integration. Their architecture figure places model code inside a much larger surrounding system, explaining why maintaining a trained model is not equivalent to operating its application.

  38. Update the Approval Status of a Model

    SageMaker supports manual model approval and approval through pipeline conditions after evaluating performance against requirements. With SageMaker-provided project templates, changing a pending or rejected model to Approved starts a deployment pipeline. Changing an approved model to Rejected starts a pipeline that deploys the latest approved version. Other transitions, such as pending to rejected, do not initiate deployment. Approval metadata thus affects operation through an explicitly configured integration.

  39. Claude Platform: Model deprecations

    Anthropic documents that requests to retired models fail and recommends testing replacement models on application tasks before retirement. Usage exports identify consumption by API key and model to help locate remaining dependencies. Partner-operated hosting platforms can have different retirement schedules.

  40. NIST Privacy Framework 1.0: lifecycle and minimized audit evidence

    The framework inventories data elements, processing purposes, actions, owners and flows. Policies define permitted uses and retention periods; the data lifecycle aligns with system development and operations. Authorizations must be maintained and revocable, access limited by least privilege, and deletion and destruction performed under policy. Audit records themselves must incorporate data minimization. Engineering application: define the decision evidence needed for review, its purpose, authorized readers, retention trigger and disposal method before logging. Retain the necessary decision, model and policy versions and relevant evidence without indiscriminately copying personal data into logs, prompts or backups. Where review requires sensitive evidence, constrain fields, access and retention rather than treating auditability as permission to keep everything. Assess removal and disclosure across downstream copies and service providers.

  41. ML Metadata

    ML Metadata records artifacts, executions and their relationships. An execution represents a workflow step or run with parameters; events record which artifacts it used or produced. Following these relationships supports questions about the data, configuration and upstream steps behind a model. Contexts group records into projects, pipeline runs or experiments. TFX uses this library, but it can also be used independently. It supplies a concrete mechanism for retaining model-production lineage rather than treating a filename as the complete model record.

  42. The Site Reliability Workbook — Canarying Releases

    A canary exposes a bounded portion of production traffic to a release candidate while the existing version serves as control. Version-separated metrics make candidate regressions visible rather than hiding them inside fleet averages. A useful canary requires limited deployment, an evaluation process, and integration of its result into rollout. Define acceptable behavior for relevant indicators such as errors, latency and resource use, gather representative traffic for sufficient time, and use the result to advance or roll back the candidate. This tests live behavior after deployment begins, beyond offline eligibility checks.

  43. Agents Need Feature Flags

    Ship agent-wide and per-tool kill switches first, and ensure in-flight work checks them at the next decision point.

  44. Cloud Speech-to-Text: operations.cancel

    The cancellation method initiates a best-effort asynchronous cancellation. Its successful HTTP response contains an empty object; callers must inspect operation status to determine whether cancellation succeeded or the operation completed despite the request. Successful cancellation retains the operation record with a CANCELLED error rather than deleting it. Unsupported cancellation can return UNIMPLEMENTED.

  45. Continuous deployment — AWS Prescriptive Guidance

    AWS recommends staged model validation using offline tests, defined promotion metrics and runbooks, and the ability to switch between versioned models. It defines rollback as reverting to a previous deployment version when errors or unexpected behavior arise. Shadow evaluation runs a candidate alongside the existing model while the earlier model continues supplying production outputs.

  46. Agents Need Feature Flags

    Track mitigation effectiveness and record flag changes with enough context to reconstruct an incident.

  47. gRPC lifecycle: cancellation is not rollback

    Client and server can disagree about an RPC's success: the server may finish while its response arrives after the client's deadline. gRPC explicitly warns that cancellation does not roll back changes already made. Application implication: cancellation during a mutation may leave an unknown outcome. Retain an operation identifier, query authoritative status or reconcile the resulting state, and use an idempotent retry contract before resubmitting. If an effect must be reversed, that requires a separate supported compensating operation rather than assuming cancellation undid it.

  48. Shipping AI to a Million Patients Without an A/B Test

    An already-delivered clinical utterance cannot be undone, so reactive rollout monitoring cannot substitute for evidence gathered before exposure.

  49. Google SRE: Handling Overload

    Admission controls and per-customer quotas limit resource consumption so one workload does not exhaust shared capacity. Resource usage can be a better capacity signal than requests per second because requests vary in cost. Graceful degradation reduces work by returning less complete results or using cheaper, potentially stale cached data. Client-side throttling can prevent rejected requests from consuming backend resources. Under extreme overload, even degraded computation may be impossible and explicit errors are necessary.

  50. OWASP LLM10:2025 Unbounded Consumption

    Uncontrolled inference can consume shared capacity or money without crossing a content-policy boundary. Long inputs, repeated requests, and expensive operations can make request count a poor proxy for work. OWASP recommends input limits, per-user quotas, resource management, timeouts, throttling, graceful degradation, and bounds on queued and total actions. For an agent, the engineering implication is to account for the whole task, including generated tokens and downstream calls, and enforce limits before repeated work expands beyond its budget.

  51. Kueue: Concepts

    Kueue treats a run-to-completion workload as its admission unit. A LocalQueue groups related workloads belonging to one tenant. Work waits until admission, competing for available quota. Quota reservation locks the workload's required allocation but is distinct from Pod scheduling. Admission requires reservation and any configured admission checks; physical node capacity is explicitly checked when topology-aware scheduling is used. Cohorts permit borrowing unused quota, and preemption can evict admitted work to accommodate another workload.

  52. Cold start performance — Modal

    Modal distinguishes waiting for a warm container from initialization performed on its first invocation. Loading model weights can contribute to either delay depending on where initialization occurs. Moving initialization into a container-entry method delays readiness until it completes; it moves the work rather than eliminating it. Retaining idle containers, maintaining a minimum count or provisioning an active-workload buffer can reduce cold-start exposure while consuming additional resources. This makes startup behavior an explicit service choice rather than a hidden property of an inference endpoint.

  53. Google Codelabs: Prototype to Production—Getting predictions from custom trained models

    The tutorial distinguishes asynchronous batch prediction for accumulated data without an immediate-response requirement from online prediction for low-latency requests. Registering a model enables its batch-prediction example; online prediction additionally deploys the model to an endpoint that associates artifacts with compute resources. The deployment example specifies machine type, replica bounds and traffic allocation.

  54. Building Deterministic Infrastructure for Non-Deterministic AI Agents

    Variable agent execution makes inference resemble a cluster scheduling problem.

  55. Implementing SLOs — Google SRE Workbook

    A service-level indicator measures a defined aspect of service, often as good events divided by eligible events. A service-level objective sets a target for that indicator over a specified period; the remaining allowed fraction of bad events is the error budget. The indicator must reflect the customer experience, and the organization needs an explicit policy for how budget consumption changes decisions. Measuring a number without defining its population, success criterion, period, and owner does not create a useful reliability contract.

  56. Google SRE: Service Level Objectives

    A service-level indicator is a defined quantitative measurement of service behavior; a service-level objective sets its target value or range. Measurements are aggregated over a specified window. Client-observed latency can differ from server-observed latency. Increasing demand can increase latency and eventually expose a performance cliff. Targets therefore need measurement boundaries and operating conditions, not just a number.

  57. NIST SP 800-61r3: incident response and verified recovery

    Triage validates an incident report and estimates severity and urgency; response priority considers impact, scope, and available resources. Containment limits expansion, while eradication removes persistence mechanisms and entry points. Investigators preserve the integrity and provenance of evidence and action records, with restricted access and defined retention. Recovery can begin during response under explicit criteria. Verify restoration assets before use, check restored systems for compromise, remediate root causes, and verify restoration before production use. Confirm service restoration with owners and monitor its adequacy. For an AI deployment-state example, specify affected components, containment, preserved evidence, authorized recovery actions, trusted restoration inputs, verification criteria, and who confirms resumed service; these are application choices implementing the guidance.

  58. Building Deterministic Infrastructure for Non-Deterministic AI Agents

    Agent debugging needs traces of the decisions and state changes that led to an outcome.

  59. OpenTelemetry GenAI semantic conventions: spans

    gen_ai.request.model records the requested model; gen_ai.response.model records the actual model when available. gen_ai.request.* includes temperature, top_p, top_k, seed, max_tokens and stop_sequences. gen_ai.conversation.id correlates an existing conversation; do not invent one from a trace ID. Opt-in gen_ai.input.messages records ordered input history, gen_ai.system_instructions separately supplied instructions, and gen_ai.output.messages responses. Tool execution uses gen_ai.tool.call.id, arguments and result. Content may instead be stored externally; reference representation remains application-defined. gen_ai.prompt.name/version support prompt provenance. Code revision, policy revision, retrieval corpus/index version and workflow definition/run/attempt identity still require application instrumentation and explicit conventions; invoke_workflow alone is insufficient.

  60. What if the network was the sandbox?

    An LLM gateway can extract model-visible tool calls and associate request history with users or workload tags without depending on instrumentation inside the agent container.

  61. Platforms for Humans and Machines: Engineering for the Age of Agents — Juan Herreros Elorza

    Self-service should expose an understandable, automated path through the complete task, without requiring a particular person's intervention.

  62. What We Learned Deploying AI within Bloomberg’s Engineering Organization

    Bloomberg combines an MCP discovery hub, shared creation and deployment infrastructure, and stronger production quality controls in a paved path, also described as a golden path.

  63. One Registry to Rule them All - Sonny Merla, Mauro Luchetti, & Mattia Redaelli, Quantyca

    The MCP and A2A template repositories standardize operational concerns while the A2A blueprint leaves agent implementation behind shared interfaces and ports.

  64. One Registry to Rule them All - Sonny Merla, Mauro Luchetti, & Mattia Redaelli, Quantyca

    The talk demonstrates a platform under development with sample catalog data, rather than measured production operation.

  65. Platform Engineering Maturity Model, version 1

    The model recommends collecting structured user feedback alongside instrumented usage and measuring outcomes such as adoption and user time saved. Its examples show how telemetry covering only one implementation can omit other teams. It also warns that higher maturity requires additional funding and staff time, so reaching the highest level is not an objective by itself.

  66. Platforms for Humans and Machines: Engineering for the Age of Agents — Juan Herreros Elorza

    Choose measures that match the intended outcome and compare them before and after platform changes.

  67. Software Engineering + AI = ?

    Token maxing illustrates Goodhart's law: when usage becomes a perceived performance target, engineers can optimize token consumption instead of useful work.

  68. Does AI Actually Boost Developer Productivity? (Stanford / 100k Devs Study)

    More activity can reflect smaller tasks or repairs to AI-generated defects rather than additional useful functionality.

  69. Does AI Actually Boost Developer Productivity? (Stanford / 100k Devs Study)

    The speaker recommends using surveys for morale and satisfaction rather than treating self-rated productivity as measured output.

  70. AIP-181: Stability levels

    Google's API guidance ties stability labels to obligations. Stable components are supported for the major version's lifetime, and their retirement process and notice period should be defined when stability is declared. Breaking changes generally require a new major version or a formal deprecation process. The guidance separately assigns mitigation of producer-initiated changes to existing traffic to the API producer, and migration to a different API version to the consumer. Exceptional security or regulatory changes can override normal notice expectations.

  71. AI Platform Engineering

    Use centrally managed generic rules with additional rules owned by each product team, while treating guardrails as incomplete defenses.

  72. The Site Reliability Workbook: Incident Response

    The workbook recommends defining incident-command, communications and operations responsibilities before an emergency, preparing contacts and channels, and practicing with realistic exercises. Google's described drills use controlled emergencies; smaller exercises can replay scenarios from postmortems and use actual troubleshooting tools in a test environment. Reviewing what worked and what failed turns practice into concrete improvements. A PagerDuty case in the chapter ends with the incident commander confirming recovery with each affected service's on-call engineer before declaring completion.

  73. Agents Need Feature Flags

    Make model selection and fallback runtime routing decisions.