Continual Learning

Extend a model over time

Continual learning develops capabilities from sequential experience rather than assuming that all training data remain available together. Here, the focus is successive parameter updates. Parameters are the model's learned numerical state; a checkpoint saves a particular model state. Each update inherits both useful capabilities and obligations to preserve behavior that still matters. [1]

Continuing from a capable checkpoint can reuse what earlier training established instead of rebuilding it from initialization. The mechanics of an individual update belong to Pretraining and Midtraining and Post-training and Alignment. This chapter adds the obligations created by repetition: preserving earlier capabilities, choosing what history to retain, and controlling what each successor inherits. [2]

Experience can also persist outside the model. Agent Memory stores information that later requests can supply to an unchanged model. This can improve behavior without demonstrating retention through parameter updates. The initial Continual Learning Bench (CL-Bench), for example, evaluated retained conversations, shortened conversation histories, retrieved memories, and notepads—not weight-updating methods. The distinction matters when interpreting a reported learning gain: what state was allowed to change? [3]

A concrete parameter-learning setting is TiC-CLIP, an ICLR 2024 study of image–text models updated with successive yearly data batches. Its checkpoints are assessed on earlier and later periods as well as static benchmarks. That arrangement makes the central obligation visible: learning from the next period should not be judged only on that period. TiC-CLIP paper [4]

Specify the learning sequence

A task specifies an input-to-output problem. A distribution describes the mix of cases encountered. Before choosing a method, state what changes between experiences and what information the model receives at evaluation. The three incremental-learning settings distinguish these assumptions. [1]

Learning cats versus dogs, then cows versus horses, does not alone establish the ability to distinguish cats from cows. A supplied task identifier removes that cross-episode choice; a unified classifier must make it. [1]

Online learning updates from arriving examples or small batches; it does not simply mean serving requests over a network. Continual learning can instead use periodic batches. Nor must each experience announce a new task. In the CLOC photographic-geolocation study, the model learns to predict where photographs were taken from images arriving in chronological order. Conditions change naturally over time, without explicit task boundaries telling the learner when to switch strategies. [5][6]

Complete the specification with historical-data access, recurring conditions, and feedback timing. A label is the target or outcome used for supervision; it may arrive well after its input. An evaluation that supplies it immediately gives the learner information it would not yet possess. Also distinguish a changed population from a changed label definition or broken input format, as developed in Maintain fitness through change. [7]

Separate acquisition from interference

Acquisition is improvement on a newly targeted capability; retention is preservation of earlier useful behavior. Both can depend on the same parameters. Training changes those parameters to reduce a loss, a numerical penalty for prediction errors. A change that reduces loss on new cases can increase it on earlier ones. Catastrophic forgetting names severe deterioration following subsequent learning, not every decline observed later. Specialization can interfere with retention develops this shared-parameter mechanism. [8]

Stability protects established behavior; plasticity permits new acquisition. The stability–plasticity dilemma is that suppressing change can protect yesterday's solution while preventing today's learning. Because each stage changes the starting point for the next, the order of experiences and the accumulated changes matter—not just the size of the latest update. [9][10]

Separate the changed variables

To investigate deterioration, cross checkpoint identity with assessment population. The following design contains no measured results. Keep preprocessing, target meanings, graders, and execution conditions fixed wherever a comparison claims to isolate model change.

Compare downward within a column to examine a checkpoint change on matched cases. Compare across a row to examine one checkpoint under different populations. Neither comparison alone explains the cause. If an answer became obsolete because a policy changed, preserving that answer is not retention of useful capability; the target definition needs a new version. Paired comparisons establish the model-change comparison, while data maintenance establishes what changed in the work.

Retention includes behavior beyond task accuracy. Qi and colleagues' October 2023 study found worse harmful-response assessments after benign fine-tuning of GPT-3.5 Turbo and Llama-2-7b-Chat under tested recipes. These were historical, single-adaptation experiments, not a demonstration of cumulative forgetting. They nevertheless show why refusal behavior needs its own assessment rather than being inferred from utility gains. Study; behavioral dimensions. [11]

Origins of stability and rehearsal

The problem predates language models. Earlier research asked whether learning systems could establish representations, acquire exceptions, and continue learning without repeatedly receiving everything together. Three contributions clarify why preservation and rehearsal remain central.

ART matches inputs to learned categories and uses reset-and-search when a match is inadequate. This protects established categories from inappropriate recoding within that architecture; it is not a guarantee for gradient-trained language models. [9]

The arithmetic network first learned addition facts involving one, then facts involving two. The second phase severely disrupted earlier answers although they remained valid. Retention tests made no weight updates, exposing interference in shared connections rather than a need to revise obsolete facts. [10]

Reinstating earlier experience lets integration encounter old and new material together. The biological interpretation is a proposed account, not a claim that engineered replay reproduces human memory. [12]

These are complementary problem formulations, not successive replacements. An architecture can protect established computations, a training objective can discourage interference, and replay can change which experience participates in learning. Before choosing among them, the sequence needs measurements that reveal what is acquired and what is lost.

Read capabilities across checkpoints

To see whether capabilities accumulate, test each checkpoint on earlier capabilities as well as the newest one, using held-out cases that were not used for training. Following Gradient Episodic Memory (GEM), let $R_{i,j}$ denote performance on capability $j$ after learning stage $i$. Each row is a checkpoint; each column is a fixed assessment. The initial row records what the model could already do before the sequence began. [8]

Read the table within columns. Stage 2 acquires 30 percentage points on B: 75 minus 45. It also improves A by 5 points. Stage 3 acquires 45 points on C while A loses 15 points from its preceding score and B gains 3. The three-column average rises from 65% to 76%, concealing A's regression. The missing C measurement after stage 1 is not zero and cannot support a stage-1 transfer calculation.

Choose the reference

Riemannian Walk instead measures forgetting from the best earlier held-out score. In the example, A has lost 15 points from its maximum of 85%, but only 10 from its acquisition score of 80%. These references answer different questions. An average across tasks should therefore identify its reference rule and preserve important per-capability results. [13]

Forward transfer measures performance before direct training on a later task relative to a baseline. GEM uses random initialization; a pretrained starting checkpoint requires an explicitly adapted baseline. Faster learning after exposure is a different measurement. [14]

Capability columns need not all be accuracy benchmarks. In Samuel Denton's workflow-adaptation experiment, submission-tool use, passing the task's tests, and satisfying both were measured separately. An agent could learn to submit without learning to solve, or solve without submitting. That separation also belongs in a continual learner's retention suite, alongside instruction following and consequential safety behaviors. Use matched-work comparisons for uncertainty and paired analysis. [15]

Rehearse selected experience

Replay trains again on selected earlier experience alongside new examples. A replay buffer is the bounded training collection retained for that purpose. Including older examples gives an update pressure to remain useful on them rather than optimizing only the arriving cases. [16]

The 1995 complementary-learning-systems simulation makes the mechanism concrete. Focused learning of penguins as birds that swim but cannot fly disrupted other concepts. Interleaving those facts with earlier material reduced interference but slowed acquisition. Total exposure differed, so this was not a matched-compute comparison. The lesson is about competing learning pressure, not a universal replay ratio. [12]

Availability, exposure, and influence

Buffer management contains three separate decisions. Admission and eviction determine what remains available. Sampling determines which retained examples actually enter an update. Loss weighting determines their relative influence once selected. Replay implementations can configure current-data and memory batch sizes separately. The collection's size therefore does not specify its training exposure. [17]

iCaRL, by Rebuffi, Kolesnikov, Sperl, and Lampert, combines replay with output preservation while adding image classes. Its selected examples, or exemplars, receive roughly $K/t$ slots per class for capacity $K$ and $t$ classes. Within each class, examples are ordered so that retained prefixes approximate the class's average learned feature vector. As more classes arrive, eviction removes each list's tail. Images are stored rather than fixed vectors because training changes how the model represents them. The complete method also classifies using exemplar means, so its results cannot be attributed to replay alone. Paper [18]

Treat dilution as an explicit design consequence: fixed capacity divided among more capabilities leaves fewer examples for each. Then inspect exposure separately. A retained rare case offers little protection if it is almost never sampled; repeatedly sampling it does not create broader coverage. Tiny buffers did improve generalization in studied settings, so repetition is not automatically overfitting—but its effect must be measured. Tiny episodic memories study [16]

Check whether an older target remains valid and permitted before replaying it. A policy revision can turn previously correct supervision into a stale target. Keep record identity and eligibility attached to retained examples, following dataset lineage. Replay should preserve useful capabilities, not mechanically reinforce every historical answer. [19]

When original examples are unavailable

Generative replay replaces stored examples with generated inputs and reference targets. In the 2017 Deep Generative Replay study, a previous generator produced inputs and its paired solver supplied targets for subsequent training. Reproducing historical output frequencies on meaningless noise inputs did not preserve digit discrimination. Input coverage mattered, and effectiveness depended on generator quality. Generated examples are therefore neither complete historical coverage nor automatically private; Synthetic Data develops their correlated-error limits. [20]

Constrain parameter movement

Retention regularization adds a penalty intended to discourage harmful changes. Kirkpatrick and colleagues' Elastic Weight Consolidation (EWC), introduced in a 2016 preprint, anchors parameters to an earlier solution and weights departures by estimated importance. Its diagonal Fisher estimate assigns each parameter an importance value based on local sensitivity of predictions on earlier-task data. “Diagonal” means it treats parameters separately, omitting their interactions. Importance estimates guide resistance to change; they do not identify which facts a parameter stores. EWC paper [21]

Unlike shrinkage toward zero, this penalty protects a learned reference. Larger importance resists movement more strongly. The local diagonal approximation omits parameter interactions; it does not guarantee behavioral retention. [21]

A local compromise

Consider an illustrative two-parameter example centered on the reference. Let displacements be $x$ and $y$, with importance 1 and 4 and $\lambda=1$. The preservation penalty is $P=(x^2+4y^2)/2$. Moving 0.5 along $x$ costs 0.125; the same distance along $y$ costs 0.5. Equal movement need not mean equal resistance.

The compromise moves less in the more strongly protected direction. These are objective minima, not a simulated optimizer trajectory. In particular, the preservation gradient is zero exactly at the reference, so it would be wrong to depict the first gradient step there as already deflected by that penalty.

Carry protection across stages

Successive stages also require a reference-update rule. Huszár's analysis of quadratic EWC penalties explains how penalties centered at successive learned optima can double-count earlier evidence. Under its recursive approximation, one penalty centered at the latest solution can carry accumulated importance. Specifying only “use EWC” therefore leaves an important multi-stage choice unresolved. [22]

Progress & Compress, introduced by Schwarz and colleagues at ICML 2018, separates acquisition from consolidation. An active network learns new skills; a fixed-size knowledge-base network then learns to reproduce its behavior while online EWC protects earlier learning. Online EWC retains one reference parameter vector and updates importance as $F_{\mathrm{new}}=\gamma F_{\mathrm{previous}}+F_{\mathrm{current}}$, where $\gamma$ discounts older importance. Discounting limits accumulated resistance, but strong preservation still impaired acquisition in some Atari experiments. Fixed-size retained state is not unlimited learning capacity. Paper [23]

Constrain earlier behavior

Behavioral regularization constrains predictions rather than parameter movement. It uses distillation—training a student against a teacher's outputs—to preserve earlier behavior during adaptation. See Distillation for the general mechanism. [24]

Li and Hoiem's Learning without Forgetting (2016) caches old-task probabilities on new-task images, then combines agreement with those targets and new-label supervision. Parameters can change without running the reference at every update. It avoids old training images, not data altogether. Paper [24]

Input coverage limits protection. Agreement on new-task images need not preserve behavior in absent old-input regions; the authors identify distribution differences as a limitation. Increasing the agreement weight favors preservation over acquisition. [24]

Choose what deserves preservation

Matching a reference and solving the task are separate outcomes. A distillation study explicitly separated predictive agreement from independently labeled accuracy; in its self-distillation experiments, more agreement could accompany lower accuracy. A reference's mistakes can be reproduced faithfully. Does Knowledge Distillation Really Work? [25]

Replay can supply historical inputs on which agreement matters: iCaRL combines historical exemplars with pre-update predictions. Conversely, if an old answer is obsolete, preserving it conflicts with the intended correction. The preservation population and target validity must be chosen deliberately. [18][19]

The teacher is also continuing state. Avalanche's versioned LearningWithoutForgetting implementation copies the model after an experience to become the next reference and records encountered classes. Rejecting candidate weights without considering that reference can leave the next update protecting the wrong behavior. This is a recovery implication of the lifecycle, not an automatic rollback feature. [26]

Separate trainable capacity

Parameter isolation assigns new learning to separate capacity. Rusu and colleagues' Progressive Neural Networks (June 2016) freeze earlier task-specific computation paths, called columns, and train a new column using their features through lateral connections. Storage grows, and inference requires selecting the appropriate task column. Paper [27]

The preservation condition is precise: the same unchanged computation under unchanged input and execution conditions retains its behavior. It does not establish that a selector will choose that path correctly or that separate paths form a unified classifier. This is why the task-identity assumption matters.

Expand ⤢

Low-Rank Adaptation, or LoRA, freezes a base matrix and trains a smaller added update. Freezing the base does not freeze the adapted system's outputs. The study LoRA Learns Less and Forgets Less measured specialization separately from retained commonsense and science performance; tradeoffs varied with domain, duration, and configuration. A small trainable component is a restriction on change, not a retention certificate. [28]

Separate triggers from permission to train

An update trigger is a rule that starts investigation or a candidate-training process. Schedules, new-data availability, deteriorating performance, and changed statistics are common triggers. Google's continuous-training guidance separates these from data validation and candidate-model validation. A signal to begin work is not evidence that the resulting checkpoint should be deployed. [29]

ADWIN, by Albert Bifet and Ricard Gavaldà, provides a concrete change detector. Published in the SDM 2007 proceedings, it compares averages within an adaptive observation window and discards older observations when subwindow differences indicate change. It can monitor prediction errors without fixing one historical window length. Its theoretical setup assumes independent, bounded observations, and its false-alarm bound is per time step—not a lifetime promise. An alarm neither identifies the cause nor proves that neural retraining will help. Paper [30]

Diagnosis determines the repair layer. A stale-policy answer may require refreshed information; a missed escalation may require a workflow gate. Soheil Feizi's repair-layer discussion recommends the smallest durable change at the layer that explains the failure. Training is appropriate only when the intended learned behavior—not merely a tool, input, or execution rule—needs to change. [31]

Use data-change diagnosis to distinguish outages, representation defects, target revisions, and population shifts. Use context maintenance when current information is missing from the request. For an actual learning intervention, record the target capability, trustworthy supervision, retained behaviors, resource limits, and owner authorized to approve the candidate. Delayed labels may postpone that decision even while unlabeled inputs accumulate.

Balance cadence and learning intensity

Cadence controls when updates begin. Learning intensity describes the work and exposure within each update: optimization steps, learning rates, replay frequency, and stopping conditions. These are independent controls. Ten short updates can repeatedly train on the same retained examples and accumulate substantial parameter change. The within-run mechanisms are developed in Training schedules.

Small-batch updates can incorporate available supervision promptly; periodic batches wait for more data before training. Broader consolidation revisits a wider mixture. None removes feedback delay or computation time. A real-time continual-learning protocol keeps predicting with the latest available model while training proceeds; slower methods consequently miss more potential training batches under that protocol's scheduling rule. Skipping batches is a design choice, not an inherent requirement. Real-time evaluation study [5]

Consider an illustrative eight-hour interval. Feedback for an observation arrives after two hours, training ends at hour four, and candidate validation finishes at hour five. The earlier checkpoint supplies predictions until then. Starting training sooner would help only if valid supervision and resources were already available; finishing training would still not itself authorize promotion.

Optimization choices can change the acquisition–retention balance. In controlled language-model continuation experiments, learning-rate rewarming—raising the rate again for continuation—accelerated adaptation but increased forgetting. Replay comparisons held continuation compute constant. The experiments covered particular English and German shifts and model sizes; they do not prescribe a universal restart schedule. Continuation study [2]

CLOC likewise found different schedule rankings for different goals: cosine decay gave the strongest final historical performance but the weakest online fit among compared schedules. A larger replay buffer could also hinder adaptation. These are reasons to measure both arriving-case performance and retained capability, not to infer either from the other. CLOC study [6]

Compare schedules on the same arrival stream, with the same label-availability rules, and disclose their training work. Record distinct examples separately from repeated exposures. Stop an update according to acquisition and retention criteria, then assess how that stopping rule behaves across many updates. The best interval is a property of the workload and learning policy, not of the word “continual.”

Respect information arrival

Prequential evaluation, also called test-then-train, records a prediction before learning from that observation. MOA distinguishes this from periodic held-out testing. The stream score combines predictions made by successive model states; it does not remeasure the current checkpoint on earlier cases. A rising stream score can therefore coexist with historical forgetting. [32]

With delayed outcomes, preserve the original prediction until its label arrives. River's progressive evaluator separates prediction and learning events using an explicit delay. The later label scores that recorded prediction and may supervise a later update; it cannot retroactively become information available to the earlier model. [7]

Keep historical and current views

Keep two assessment views. Fixed historical suites compare checkpoints on stable capabilities. Fresh chronological cohorts measure performance under later conditions. Version their cases, labels, graders, and execution settings. If a target definition changes, preserve the old assessment's meaning rather than silently substituting new labels and calling the difference forgetting.

Test-before-train protects one boundary, not every boundary. Cases inspected to trigger an update or select its replay policy have already informed development. They remain valuable regression tests, but repeated selection against them does not provide an independent final assessment. Reserve later or otherwise protected cases and preserve related-record separation, following Protect the independent assessment.

Report outcome availability beside performance. A recent interval with unresolved labels is incomplete, not successful. If only escalated cases receive review, the observed labels describe a selected population. Incomplete feedback explains that distinction. Show uncertainty and coverage by meaningful time window and capability slice, especially when conditions recur and an aggregate would hide a temporary failure.

Compare complete learning policies

A learning policy specifies what history is retained, how it is sampled, what changes are constrained, when updates run, and how candidates are selected. Compare complete policies from matched starting checkpoints and stream access. Report training work, historical-data access, auxiliary storage, tuning effort, and adaptation delay separately from capability outcomes.

TiC-CLIP compares continuation with replay, new-data-only continuation, equal-budget restarting, and more expensive pooled-data training. Replay improved historical and static-task performance over new-data-only continuation in reported comparisons. Bounded buffers remained competitive with full-history retention under tested budgets. Its period-specific accuracies are not GEM-style transfer differences, and unequal-budget references must remain labeled as such. [4]

Measure the ability to keep learning

Retention alone can reward a learner that barely changes. Loss of plasticity is deterioration in the ability to acquire subsequent tasks. Dohare and colleagues' August 2024 Nature study evaluated long sequences of two-category image-classification tasks under fixed per-task training budgets. Some networks learned well initially but later approached or fell below a linear baseline. Their continual-backpropagation method selectively reinitialized low-utility units—resetting selected internal components so they could learn afresh—and maintained acquisition over the tested sequences. The protocol announced task boundaries and reset each task's output head, the part producing its category predictions. It chiefly demonstrated sustained acquisition, not joint long-horizon retention for language models. [33]

Test enough stages to expose cumulative effects, including relevant orderings and returns to earlier conditions. An unchanged checkpoint is a useful baseline but fails a requirement to acquire genuinely new capability. A highly plastic learner can fail retention requirements. Select among policies that meet the explicit obligations; then compare their resource use and delay. Missing measurements remain unavailable rather than being converted into favorable scores.

Preserve the learning state

Inference and continued learning need different state. Weights plus compatible architecture and preprocessing may be enough to produce predictions. A resumable checkpoint also identifies how optimization continues. Continual learning adds dependencies that determine what the next update will rehearse and protect.

Match the saved boundary

Checkpoint APIs have specific recovery boundaries. Avalanche's checkpoint implementation serializes a learning strategy, random-number manager, and completed-experience counter. Its documented save operation runs outside training and evaluation loops. Excluding attributes changes what can be restored. This supports experience-boundary recovery, not arbitrary mid-update transactions. [34]

TorchData's StatefulDataLoader separately records progress through inputs. Its default state tracks yielded batches, and custom samplers or datasets can supply their own state. Worker state is aggregated, but distributed-rank coordination is not supplied by that mechanism. A yielded batch is not necessarily a completed optimizer update; the application must coordinate those boundaries. [35]

Lineage records artifact–execution dependencies. ML Metadata distinguishes datasets and models from executions that consume and produce them. Use that model to trace a candidate to its parent, inputs, and retained learning state, extending dataset lineage. A recorded dataset dependency does not prove every record was consumed, and provenance alone performs no recovery. [36]

Keep the served checkpoint separate from the experimental learner. When rejecting or branching an update, either restore a compatible set of continuing state or deliberately define a new starting policy. Do not accidentally pair old weights with a new teacher or unrelated replay snapshot. Even logical restoration is not bitwise reproducibility: PyTorch does not guarantee identical execution across releases, platforms, or CPU and GPU implementations. [37]

Promote only supported improvements

Candidate acceptance should connect directly to the capability matrix: improvement on intended work, acceptable changes on retained capabilities, and compatible operation. Feizi's verifiable continual-learning proposal combines an executable failure case, measured before-and-after improvement, and regression tests. Passing those tests supports preservation within their coverage, not a promise that nothing can break. [38]

Include retention during candidate development rather than selecting solely for the newest task and inspecting regressions only afterward. That makes preservation part of the search criterion. It does not remove the need for independent assessment, nor does it supply an algorithm that can check an indefinitely growing history at constant cost. [39]

Shadow evaluation assesses a candidate alongside the incumbent while the incumbent still supplies production outputs. A canary allows a bounded population to receive candidate outputs. Their evidence differs: shadow results cannot establish the consequences of acting on the candidate. Keep side effects isolated and use the experimental guidance in Choose the live experiment. AWS's deployment guidance likewise separates validation, promotion criteria, and switching between versioned models. [40]

GPT-4o's April 2025 sycophancy incident shows the importance of a missing behavioral check. OpenAI's May 2 retrospective reports favorable offline and small-scale preference results before a release that excessively agreed with users. Dedicated sycophancy deployment evaluations were missing. The preliminary explanation implicated interacting training changes, including an added reward from user feedback. A prompt mitigation preceded restoration of the previous model. This was a release regression and serving recovery, not demonstrated cumulative forgetting. Incident account [41]

Recover before learning again

Serving rollback changes which checkpoint supplies predictions. Learning recovery establishes which inputs and state may produce its successor. Those are separate responsibilities: a serving pointer does not encompass the learner's buffer, teacher, optimizer, or data cursor.

The separation follows from concrete state mutations. ReplayPlugin updates its stored experience after training, while a previous-experience teacher can also advance. Returning to earlier weights does not reverse those changes. Recovery therefore needs an explicit decision about auxiliary state, not just the weight file. [17][26]

First contain exposure and pause affected learning or promotion. Trace the rejected candidate's inputs and descendants. Then choose between restoring a compatible earlier learning bundle and deliberately rebuilding one from an appropriate checkpoint and eligible data. Reassess the restored or rebuilt learner before it can produce an accepted successor. This is an engineering recovery design; the component libraries do not collectively provide an automatic transaction implementing it.

Expand ⤢

An older served version must still be compatible with current application state. Restoring it changes future execution but does not reverse messages, database writes, or other completed effects. Those require the separate reconciliation or compensation described in Restore software without erasing effects.

Removing a bad example from replay prevents that route of future exposure; it does not establish that already-trained parameters no longer reflect it. Cao and Yang's 2015 machine-unlearning paper distinguishes source deletion from reversing learned influence. The response to that stronger requirement belongs to Address learned influence, not to a claim that buffer deletion undoes training. [42]

Post-release outcomes become candidate learning inputs, not automatic ground truth. A data flywheel collects interactions and feedback, curates them, and trains and evaluates candidates; that loop alone supplies no forgetting protection. NVIDIA's data-flywheel discussion illustrates the operational cycle. Close it with the same disciplines used at the start: trustworthy supervision, retained-capability requirements, identifiable learning state, and permission to reject the next update. [43]

Open questions

1. 01

   Bounded memory must protect rare, recurring capabilities while remaining useful for new conditions. Uniform sampling, class balance, and recency each prioritize different coverage. Progress would show sustained acquisition and slice-level retention under fixed storage and training budgets, not merely a favorable final average. [16][6]

2. 02

   Preserving earlier behavior and preserving the ability to learn are distinct objectives. Methods that maintain acquisition over long sequences still need joint tests of historical retention, especially without announced task boundaries. Progress would establish both properties under the same continuing protocol. [33]

3. 03

   Realistic personalization evaluation needs feedback that reflects changing human preferences rather than only a simulator's regularities. This matters because success in easily verified environments may not transfer to long-term user adaptation. Progress would connect simulated learning gains to independently observed user outcomes. [46]

4. 04

   Growing retention suites make repeated candidate selection increasingly expensive. Selective testing must avoid systematically omitting the very capabilities an update threatens. Progress would quantify missed-regression risk alongside saved evaluation work across long learning sequences. [39]

5. 05

   Recovery must coordinate model state with the replay, references, and data progress that shape future updates. Component checkpoints provide only particular boundaries. Progress would demonstrate failure-tested restoration of a compatible learner and verify its next update, rather than demonstrating only that old weights can serve again. [34][35]