Purpose and foundations
Memory across interactions
An agent is software that chooses actions using observations; Agent Engineering develops that decision loop. Agent memory is application-retained information made available to later interactions. A saved decision can spare someone another explanation. A retained investigation can reveal an approach already tried. Neither benefit follows from persistence alone: the system must recover the relevant information and interpret it appropriately.
Saving information does not make it visible to the model. Model context is the input supplied to one model call. To use a saved preference or prior decision, the application must retrieve it and include it in that input; the full store may be much larger than any one call can accept. State beyond the prompt explains this separation. Persistence also needs the right scope. A checkpoint saves execution state so interrupted work can resume, but does not by itself make that experience available to a new task. In the implementation described in How We Solved Context Management in Agents, stored material was recoverable within a session but unavailable to new chats. Agent Runtimes and Harness Engineering covers execution recovery; memory design determines what later interactions can reuse.
External memory changes what information a model receives, not its parameters—the numerical values learned during training. Updating those values over time belongs to Continual Learning. Memory records also differ from a key-value cache, which reuses intermediate numerical representations during model execution. This chapter concerns the retained information itself: what is worth keeping, how to recover it, and when to revise or remove it because its meaning or permitted use has changed. More storage expands that responsibility as well as the available history.
Episodes and reusable claims
Episodic memory records particular experiences with their circumstances. Semantic memory represents reusable knowledge or claims separately from one occurrence. The Cognitive Architectures for Language Agents (CoALA) framework uses this distinction to separate interaction histories from knowledge about the world or the agent. These borrowed cognitive terms describe software records, not evidence of human-like cognition. Semantic memory is also distinct from semantic search: one describes what is stored, the other a way of finding material by meaning.
| Representation | Main use | Meaning that needs preservation |
|---|---|---|
| Conversation transcript | Recover what was said and in what order | Speaker, surrounding exchange, tentative alternatives |
| Structured episode | Recover a particular attempt or decision | Task, circumstances, action, observed outcome |
| Reusable claim | Apply a fact, rule, or preference later | Scope, source, qualifications, stated or inferred status |
An episode recording a request for bullet points on one report does not, by itself, justify a standing preference for bullet points everywhere. A reusable claim needs its own support and scope. It might be an explicit preference, a sourced fact, or a labeled inference; its storage category does not certify its truth. Operational experience follows the same distinction. A previously reviewed case can guide a similar case before anyone has articulated a general rule. A later expert explanation may supply that rule while the original episode remains useful for understanding exceptions. Why Your Agent Disagrees With Itself develops this complementary use of cases and domain knowledge.
Origins and turning points
Retaining experience predates language-model assistants. Earlier systems already faced distinct problems: representing an individual rather than an average user, finding a relevant event in a growing history, and revising a solution before reusing it. Language models added flexible interpretation and generation of memory records, but did not remove those older responsibilities.
| Year | Contribution | Problem it made explicit |
|---|---|---|
| 1972 | Tulving’s episodic–semantic distinction | Endel Tulving distinguished personal events and their temporal-spatial relationships from organized knowledge. His later account acknowledges antecedents and evolving definitions. |
| 1979 | GRUNDY — persistent user models | Elaine Rich’s book-recommendation system separated general knowledge, persistent individual information, and dialogue-specific information. |
| 1983 | CYRUS — reconstructive retrieval | Janet Kolodner’s published model organized events and used contextual categories, time, and earlier question context to retrieve relevant experience. This is the paper’s date, not an established invention date. |
| 1994 | Case-based reasoning — revision before retention | Agnar Aamodt and Enric Plaza’s framework connected retrieving a case, reusing it, testing and revising a solution, and retaining useful experience. |
| 2023 | Generative Agents — observations and reflections | Joon Sung Park and colleagues’ simulation architecture made observations retrievable and stored higher-level reflections as additional memories. |
| 2023 | MemGPT — controlled external-memory access | Charles Packer and colleagues’ system used model-directed functions to manage bounded working context and retrieve external records. Storage extended recoverable information, not the model’s physical context window. |
These contributions describe complementary choices, not a replacement sequence. Explicit profiles remain useful when known preferences should be available predictably. Episodes preserve details that a generalized claim may omit. Model-generated reflections can make experience easier to reuse while introducing another interpretation to check. A contemporary system can combine all three without reproducing any historical architecture wholesale.
Forming accountable records
Identity, scope, and provenance
A remembered sentence needs enough surrounding information to remain interpretable after its original conversation disappears. Provenance records its origin and derivation: which source supplied it, what transformation produced it, and who or what was responsible. W3C PROV represents these relationships using entities, activities, and responsible agents. Such records support inspection; they do not prove that a source is truthful or an inference correct. Origins and dependencies develops the broader lineage contract.
Attribution also requires several identities. Consider an example in which Mira tells a project assistant that Leon prefers email for project updates. Mira is the speaker; Leon is the described subject; the assistant is the recording actor. The workspace may own the record, while only selected members may read it. Nothing in the sentence establishes those ownership or access decisions. Collapsing these roles can turn a third-party report into a preference attributed to the account holder. Distinguishing people and actors explains the identity boundary.
| Record property | Purpose |
|---|---|
| Stable record ID and revision | Identify the claim and the particular version read or changed. |
| Content and claim status | Preserve the assertion, including whether it was stated, observed, or inferred. |
| Speaker, subject, and recorder | Separate reporting, being described, and creating the record. |
| Scope, owner, and access policy | Describe where the claim applies and who may use it. |
| Source and derivation references | Recover support, qualifications, and transformation history. |
| Relevant times and lifecycle status | Distinguish historical applicability, recording, supersession, expiry, and restriction. |
Use stable identities rather than display names where the system has them. For authenticated accounts, OpenID Connect specifies issuer plus subject as the stable identifying combination; email addresses and usernames do not offer the same guarantee. That identifies an account, not every person mentioned in its conversations, and grants no memory access by itself. Likewise, a namespace organizes records but needs an authorization mechanism. Keep uncertainty explicit: “Mira reports Leon’s project preference” preserves information that an unexplained confidence value would erase.
Choose what deserves retention
Memory formation is the selection and conversion of observations into retained records. Its first decision is whether to retain anything. Start from a plausible later use, then consider stability, scope, sensitivity, verification effort, and the consequences of remembering incorrectly. An explicit request to remember supplies intent; automatic extraction proposes useful content; an inferred preference adds an interpretation. None should silently establish unrestricted reuse. A small qualitative study, Users’ Expectations and Practices with Agent Memory, found task-specific expectations, including keeping unrelated personal information out of other projects.
A retention policy specifies purpose, duration, review, access, and disposition—the action taken when retention ends. Useful lifetime and permitted retention duration are different constraints. A fact may become obsolete before its record must be deleted; a still-useful record may no longer be permitted. Set artifact lifetimes supplies the governing framework. NARA’s disposition guidance illustrates precise clock-start events such as project completion, replacement, or cancellation, rather than an indefinite instruction to keep something until no longer needed. It does not prescribe durations for agent applications.
| Information | Smallest useful representation | Reuse or end condition |
|---|---|---|
| Explicit project preference | Narrow claim with the original instruction | Use within the project; revise when the user changes it. |
| Failed technical approach | Episode with attempted change, result, and relevant versions | Consult for similar work; recheck assumptions against current code. |
| Rapidly changing operational state | Source reference, with a dated snapshot only if needed | Read the authoritative source for a current-state decision. |
| Incidental sensitive detail unrelated to the purpose | No long-term memory record | Exclude from memory formation under the application policy. |
Admission instructions make these choices operational but are not enforcement on their own. Mem0’s custom extraction instructions distinguish personal preferences from operational lessons and recommend inspecting extracted records on actual conversations. A prompt asking a model not to save sensitive content still requires checks at the storage boundary. For episodic reuse, Polygraph’s documented debrief workflow connects sessions to code artifacts and explicitly rechecks old assumptions. Retain enough to support that rechecking, rather than treating a past conclusion as permanently applicable.
Accept meaning, not just structure
Once retention has a purpose, extraction must preserve what the source actually says. The write path should identify a candidate, resolve its subject and scope, preserve qualifications, apply retention rules, and compare existing records before accepting a change. Acceptance may instead reject the proposal or request clarification. This is an application contract: a model can propose a record, but schema validity checks its shape rather than its truth. Similarly, an agent’s statement that an operation succeeded is not a substitute for the observed outcome.
The Mem0 team’s April 2025 paper makes candidate extraction and memory updating separate phases. New conversation material proposes facts; comparison with existing memories leads to add, update, delete, or no-change operations. Those model-selected operations still need semantic and permission checks. The older case-based reasoning framework likewise placed testing and revision before useful retention: storing an experience and accepting its solution are different decisions.
Small linguistic changes can create large memory errors. Preserve negation, conditions, temporary exceptions, and whether an event is planned or completed. In Lessons from Studying Every Memory System, discussion of Thailand and Turkey as alternative destinations was reportedly condensed into a profile claiming visits to both, although only the Thailand trip occurred. The defect was not simply old information: the stored assertion changed consideration into completion. A date field or a fluent summary would not repair that change of meaning.
Choose extraction methods according to the uncertainty that remains. Deterministic mappings fit explicit fields with known meanings. Model proposals can cover less structured language, but need source comparison. Targeted confirmation is useful when an unresolved interpretation would materially change future behavior. Formation during the request can make information available sooner while adding latency and competing with the task. Background formation separates that work and may inspect more accumulated context, but another interaction can arrive before it finishes. LangChain’s memory overview describes this latency–freshness tradeoff. A pending proposal should not be presented as an accepted memory.
Organizing and recalling memory
Consolidate without manufacturing support
Consolidation reorganizes or combines retained records into representations that are easier to reuse. Removing duplicate copies, merging compatible statements, and inferring a broader lesson are different transformations. The last produces a new claim. Unlike compaction, which reduces the current input, consolidation changes what later interactions may retrieve. It therefore needs to preserve dependencies and exceptions, not merely produce shorter prose.
A reflection is an interpretation of experience retained to guide later behavior. Generative Agents synthesized such interpretations from observations and made them retrievable. Reflexion, introduced by Noah Shinn and colleagues in 2023, organized the process around three roles: an actor attempted the task, an evaluator assessed the attempt, and a self-reflection component converted the feedback into stored guidance for the next attempt. These systems changed the information supplied to later model calls, not model parameters. The guidance remained an inference: a mistaken evaluation could produce a mistaken lesson, and repeatedly retrieving that lesson would not add independent support.
Preserve whether each source supports, contradicts, or qualifies a derived claim. Copies share an origin; they are not independent observations. For example, two successful export runs might support using a format by default, while a legacy reader’s compatibility requirement limits where that default applies. Copying one run’s report adds a record, not another successful run. Keep the derived default linked to both outcomes and the exception so a source change can trigger reconsideration. Links themselves also need interpretation: Zep’s episode associations can record extraction, reaffirmation, or invalidation. Counting all associations as support would erase those differences.
Copies do not add independent support
ExampleA derived default has fewer independent origins than records, and its exception must survive consolidation.
Read the diagram as text
- Run A: format accepted. A recorded export outcome.
- Copy of Run A report. Same origin, another record.
- Run B: format accepted. A separate recorded outcome.
- Legacy reader needs another format. A compatibility requirement.
- Default except for legacy readers. Derived guidance, linked to its origins.
- Run A: format accepted → Copy of Run A report: copied into.
- Run A: format accepted → Default except for legacy readers: supports.
- Copy of Run A report → Default except for legacy readers: repeats existing support.
- Run B: format accepted → Default except for legacy readers: independently supports.
- Legacy reader needs another format → Default except for legacy readers: limits applicability.
Keep substantial consolidation reviewable. Anthropic’s Dreams documentation describes processing an existing memory store and session transcripts into a separate output store, leaving the input unchanged. The output identifier can exist before processing finishes, and failed or canceled work may leave partial results. Completion, review, and adoption are separate decisions. This gives developers a reversible maintenance workflow, not a guarantee that synthesized corrections are right. Supporting sources must also remain accessible only under their own access and retention rules.
Records and searchable views
Choose a retained representation by the questions it must answer and the unit that may need correction. A compact profile is convenient to supply wholesale, but combines many assertions in one update surface. Individually addressable claims allow narrower changes while requiring more retrieval and consolidation. Episodes preserve circumstances; source-linked text preserves detail that extraction may have discarded. These forms can coexist rather than compete for one universal memory format.
| Form | Useful retrieval | Correction unit |
|---|---|---|
| Compact profile | A small set of recurring defaults | One field or the profile, with assertion-level support retained separately |
| Individual claim records | Facts filtered by subject, scope, and applicability | An identified claim and its dependent representations |
| Episodic log | A prior attempt, decision, or sequence | An attributed correction linked to the historical episode |
| Source-linked text | Exact wording and surrounding circumstances | A source revision and the records derived from it |
An index is a derived structure for finding records. It need not be their authoritative home. Exact-key lookup or keyword search can provide useful memory without vector retrieval. An embedding is a learned numerical representation that can support similarity-based candidates; Embeddings and Representation Learning explains its limits. OpenClaw’s built-in engine documents memory files separately from a SQLite search index, with keyword retrieval available without embeddings. Rebuilds prepare a replacement index before publishing it, preserving the previous index if rebuilding fails.
A knowledge graph adds explicit entities and relationships when repeated identity-aware or connected queries justify the maintenance cost; Knowledge Graphs develops that choice. Whatever the representation, declare what a successful write promises. Accepted storage, completed extraction, and searchable visibility can be separate. Zep’s ingestion-status documentation explicitly separates processing from search readiness; a helper returning any result does not prove that the intended new record is retrievable. If the next interaction requires the new version, wait for record-specific visibility or use an authorized direct read. Do not silently rely on an older search view.
Recall what applies now
Recall selects retained information for the current interaction. Its starting point is the task, subject, time, and scope—not a similarity query alone. The retrieval unit might be a preference, a complete episode, or a source passage. Small scoped profiles can be supplied routinely when their contents are predictably relevant. On-demand recall avoids loading unrelated history but introduces a discovery decision and access latency. MemGPT’s model-directed searches illustrate that external persistence still requires bounded results to enter the working context.
Separate eligibility, which determines what may be used, from ranking, which chooses useful records among eligible candidates. Subject, current authorization, lifecycle status, and applicability are constraints, not small penalties a strong similarity score can overcome. Enforce them on every path before information reaches the model, including source expansion. Search filters also need their actual semantics checked: Mem0’s documented entity-scoped search constrains only explicitly supplied dimensions, so filtering by user alone can span applications or runs.
| Candidate | Eligibility decision | Role in selection |
|---|---|---|
| Nearly identical preference for another person | Exclude from this person-specific use | Similarity cannot repair the subject mismatch. |
| Old project default explicitly superseded | Exclude as current guidance | May be relevant only to an authorized historical inquiry. |
| Current scoped default with different wording | Eligible | Evaluate usefulness for the task. |
| Eligible claim whose exception is unclear | Retrieve supporting context before applying | The compact record is insufficient by itself. |
Use exact lookup for a known record or identity, temporal filtering for a dated question, and similarity search to discover candidates expressed differently. Recency and importance can complement relevance, as they did in Generative Agents, but do not establish truth. When a claim omits necessary circumstances, retrieve its episode or neighboring messages. An ActiveGraph experiment used matching log messages plus adjacent messages without first extracting facts, illustrating a simple baseline for preserving conversational context.
No matching eligible record, incomplete search, and unavailable storage are different results. A failed lookup cannot establish that nothing was remembered; degraded coverage explains that boundary. Return selected records with their identities, scope, relevant times, source links, and qualifications to the context assembler. The assembler decides how to supply them; memory is responsible for what they represent and whether they remain eligible.
Use and change
Personalize without granting authority
Personalization adapts behavior using information associated with a person or context. A stated standing preference supplies a default; a one-time request supplies a local instruction; an inferred tendency remains a hypothesis. Keeping these distinct lets memory reduce repetition without overriding the present task. GRUNDY already handled this distinction in 1979: dialogue-specific information could take precedence over a persistent individual model without erasing the general preference.
| Present task | Appropriate use of that memory |
|---|---|
| Prepare a project update with no format specified | Use the scoped default. |
| Prepare this update as a detailed narrative | Honor the explicit exception; preserve the standing default. |
| Send the update to an external recipient | The formatting preference supplies no authorization to send. |
When an uncertain preference would materially alter an outcome, expose the uncertainty or request confirmation. Repeated acceptance or silence does not validate an inference; incomplete feedback explains why. Make targeted correction possible by exposing the remembered claim, its scope, and its justification. Judy Kay and Robert Cook’s Justified User Model, presented in 1994, let users inspect supporting and opposing evidence and the rules behind inferences. User edits added explicit evidence instead of silently erasing the justification history. Modern interfaces still need to connect a visible correction to the actual stored record and future behavior; repairing the affected boundary develops that interaction.
Remembered permission is historical information, not current authorization. Apply current authority at the protected operation. Storage also does not make external text a trusted instruction. Memory poisoning contaminates retained records so later retrieval influences behavior. The 2024 AgentPoison paper studied attackers able to insert records into a retrieval store and use an embedder during attack construction; later queries could retrieve malicious demonstrations without any weight change. This is a persistent data-to-behavior risk under that threat model. AI Security explains why relevance labels and delimiters do not replace enforceable boundaries.
Correct, supersede, or qualify
New information can disagree with memory for different reasons. The old extraction may have been wrong, the world may have changed, or the new statement may concern another scope. Start by classifying the disagreement. Neither newest-wins nor highest-confidence-wins distinguishes these cases. A newer report may be less authoritative, and a confident interpretation may omit a condition. When sources remain in conflict, preserve their attribution instead of silently selecting one; Stop Babysitting Your Agents describes the practical problems caused by hiding conflicting code and organizational guidance.
| Disagreement | Disposition | What remains distinguishable |
|---|---|---|
| Source was extracted incorrectly | Correct the claim | The source statement versus the faulty interpretation |
| A fact genuinely changed | Supersede its former current value | Earlier and later applicability |
| A request creates a temporary exception | Qualify scope | The default and its bounded exception |
| Sources give incompatible reports | Retain attributed disagreement | Who supports each claim and under what conditions |
| An inference lacks adequate support | Suspend use pending clarification | Unresolved interpretation versus accepted fact |
Effective time, also called valid time, describes when a claim applies in the modeled world. Recorded time, or system time, describes when its version entered database history. Suppose a transfer from team Alpha to team Beta takes effect Monday 1 June 2026 and is recorded Wednesday 3 June. A current account of Monday can include the correction; reconstructing what the system knew on Tuesday cannot. XTDB’s temporal model supports these separate views. Neither clock proves the assertion true. Preserve historical versions only when their purpose and retention rules justify doing so.
One effective date, two knowledge cutoffs
Transfer: Alpha → Beta, effective Monday 1 June 2026; recorded Wednesday 3 June. Each date means 00:00 UTC.
| Knowledge cutoff ↓ | Sun 31 May | Mon 1 Jun | Tue 2 Jun |
|---|---|---|---|
| Tue 2 Jun | Team Alpha | Team Alpha ✓ | Team Alpha |
| Wed 3 Jun | Team Alpha | Team Beta | Team Beta |
| Version / team | Effective interval | Recorded interval |
|---|---|---|
| Original account · Alpha | (−∞, ∞) | [05-30, 06-03) |
| Corrected · old team · Alpha | (−∞, 06-01) | [06-03, ∞) |
| Corrected · new team · Beta | [06-01, ∞) | [06-03, ∞) |
Starts are included; ends are excluded. The original account is the database’s view from May 30 until June 3. Wednesday’s correction closes its recorded interval, retains that historical version, and adds two versions: Alpha before Monday, Beta from Monday onward. The transfer date stays fixed.
A correction must reach dependent claims and retrieval surfaces. A profile summary or indexed passage can continue serving the old assertion after the authoritative record changes. Source links identify which derivatives need reconsideration. In Citation Needed, graph mutations preserve source links through entity merges and attach the evidence responsible for invalidating a fact. This records why the change happened rather than replacing one unexplained value with another. Consider record M corrected to revision 8: a workaround applies to “Release A only.” An authorized direct read returns that restriction, while ordinary recall still supplies a summary based on revision 7 saying “All releases.” The release-B task and model are unchanged; derivative refresh remains pending.
Protect the accepted correction from delayed writers. An extraction job may have read revision 7 before a user creates revision 8. Its later proposal should not overwrite revision 8 as though nothing changed. A version precondition requires the stored revision still to match the proposal’s basis; a mismatch triggers reconsideration. HTTP’s If-Match provides a concrete server-enforced form of this rule. Preserve state at acceptance explains the invariant. The check prevents a stale overwrite; it does not decide which factual claim is correct.
Expire and forget without reappearance
Information can stop influencing work in several ways. Lower ranking makes it less likely to be retrieved. Expired applicability makes it unsuitable as current guidance. Restriction excludes it from particular uses. Physical deletion removes a retained representation. These are not interchangeable: dropping text from a prompt does not delete its source, and recency decay does not implement retention.
Time to live, or TTL, is a storage expiry mechanism whose behavior depends on the implementation. LangGraph’s store interface documents adapter-dependent expiry, normally refreshed by access, with best-effort deletion. Frequently read content can therefore remain retained even when its facts are old. Use separate applicability checks and explicit retention end conditions. A project closing, a source being replaced, and a user requesting forgetting can require different changes even when no TTL has elapsed.
Forgetting must follow derivation links. Zep’s deletion documentation preserves shared artifacts with remaining episode associations, but warns that shared names and summaries are not regenerated and can retain information from a deleted episode. Support-based deletion therefore does not establish complete semantic removal. OpenClaw’s tracked-origin cleanup takes a different approach: it removes an affected merged entry and checks forgotten-origin status before publishing later derivatives. Its contract excludes original transcripts, untracked paraphrases, exports, external backups, and other plugins.
The general requirement is to record exclusion before cleanup and keep it effective through delayed processing and restoration. A tombstone is a retained marker that a record or origin must no longer be republished. Deleting today’s index entry is insufficient if yesterday’s pending job can recreate it tomorrow. Before publishing, the resumed writer checks the current exclusion registry; an excluded origin causes publication to be rejected. Propagation of correction and deletion covers coordinating these controls and verifying each affected surface.
A delayed writer must check current exclusion
Retained source material can also regenerate memory through ordinary processing. OpenAI’s Memory FAQ states that disabling memory and deleting memories do not delete old chats; re-enabling memory can create memories from retained chats. Its current memory summary is not an exhaustive inventory. These controls should not be confused with the separately documented legacy saved-memory store. Report completion narrowly: exclusion from future use, cleanup of tracked derivatives, and removal of source material are separate claims. Any retained exception needs an owner and permitted purpose. External-record deletion cannot retract delivered outputs or undo parameter training; learned influence requires separate treatment.
Evidence of useful memory
Test the memory responsibilities
A memory evaluation case specifies earlier interactions, what may be retained, intervening changes, a later task, and expected behavior. The later input must not already reveal the information whose retention is being tested. Include correct memories, wrong subjects, unrelated projects, stale claims, explicit corrections, and requests not to retain information. Use isolated stores or controlled snapshots so one case cannot teach another its answer. Preserve chronological cutoffs: a later correction must not enter a replay of what was known earlier.
LoCoMo, introduced in the 2024 conversational-memory study, links answer evidence to conversation turns and includes temporal and unanswerable cases. LongMemEval, introduced in 2024, evaluates extraction, cross-session reasoning, temporal reasoning, updates, and abstention—declining to supply an unsupported answer. Its indexing, retrieval, and reading decomposition makes failure localization explicit. Both use constructed histories and bounded assessment protocols; neither a high score nor correct answers verify storage deletion or access isolation.
Newer protocols expose additional responsibilities. The April 2026 Memora preprint specifies both facts responses must include and obsolete facts they must exclude. The August 2026 AuthMem-Bench preprint holds a claim and later task fixed while changing whether its source had the relevant authority, testing whether consolidation improperly promotes reports into decisions. These are useful test designs, not universal permission policies or proofs of physical erasure.
| Responsibility | Observable evidence | Focused test |
|---|---|---|
| Formation | Accepted record against its source | Check attribution, negation, scope, and appropriate non-retention. |
| Retrieval | Eligible expected records versus delivered identities | Hold the stored snapshot and later task fixed. |
| Use | Delivered record versus answer or action | Supply the correct memory directly without changing the task. |
| Correction propagation | Accepted revision versus searched revision | Correct the record, then inspect ordinary recall and direct retrieval. |
| Forgetting | Exclusion state, active derivatives, and publication results | Pause a writer, forget its origin, then resume it and check rejection. |
Oracle-memory injection supplies the known relevant record directly as an experimental condition; it is not a deployable retrieval policy. If injection repairs an answer, investigate the ordinary formation and delivery path. If it does not, the model may be misusing the information, or the task may require something else. Memory Harnesses for Long-Running Research Agents compared recall policies with a fixed model and reported failures even when correct memory was supplied. Capture record versions, selection decisions, delivered context, and observed behavior through privacy-controlled boundary records, rather than attributing every wrong answer to retrieval.
Measure whether remembering helps
Correct memory behavior is necessary but not sufficient for useful memory. A baseline is a credible alternative for the same work: no cross-interaction memory, a small explicit profile, recent-history access, or a richer retrieval system. A paired comparison evaluates alternatives on matched tasks. Hold the model, task, and relevant settings fixed when isolating a memory component. If a whole-system comparison changes models, budgets, or tools, report those changes instead of assigning the entire difference to memory.
One operational measure is the difference between a system’s stateful and stateless task performance: retain information across earlier instances in one condition and reset it in the other. This separates some benefit of prior experience from initial model strength. Beyond Static Intelligence argues for keeping this gain alongside absolute performance and cost. A weak system can improve substantially and remain inadequate; a strong one can perform well without benefiting much from memory.
| Model | Task success: no memory → memory | Preference-enforcement utterances per session |
|---|---|---|
| Llama-3.3-70B-Instruct | 41.78% → 46.38% | 2.98 → 1.96 |
| Qwen-2.5-7B | 36.21% → 35.18% | Not reported here |
Those simulators withheld progress when preferences were violated; fewer preference reminders are not measured time savings. A separate 19-participant, three-session study using assigned preferences found improved preference-related ratings but weaker cross-domain transfer, with some participants preferring explicit restatement. Neither result establishes months-long productivity or includes a complete memory-maintenance cost comparison.
Measure the outcomes the application actually values: correct continuation, repeated clarification, repeated mistakes, and successful use of corrections. Keep harms separate—false personalization, stale assumptions, cross-person disclosure, and inappropriate cross-project reuse. Add user correction effort, write and read latency, consolidation compute, and ongoing maintenance. Include tasks on which memory should have no effect. This detects negative transfer, where retained information makes later work worse, rather than rewarding recall regardless of consequence.
In live comparisons, assign and isolate the relevant user or workspace history. Switching policies independently per request can let one condition inherit memories created by the other; live experiment design explains this interference. Account for missing feedback rather than treating silence as success. Adopt the smallest memory policy that improves the intended work while satisfying scope, correction, and forgetting requirements. Better continuation does not compensate for unauthorized reuse, and a larger archive is not evidence that the system has learned anything useful.
Open questions
Reliable generalization remains difficult: episodes can support useful defaults without justifying universal rules. Progress would preserve scope and exceptions while showing better later-task performance, rather than simply producing more persuasive reflections.
Selective forgetting becomes harder after multiple sources have been merged into one claim. Removing the whole derivative loses useful information; retaining it can preserve unwanted contributions. Progress would establish bounded, verifiable cleanup across derivations and delayed publication without claiming that source-link removal proves semantic erasure.
Long-term net value remains uncertain when memory changes both agent behavior and user behavior. Matched field studies need to measure useful continuation, harmful carryover, correction effort, and operating cost over sustained use; recall accuracy and short-session preference ratings answer narrower questions.
































































































































