Imagine telling an AI assistant, over the course of several months, that you moved from New York to San Francisco. Weeks later you mention your new apartment, and weeks after that a follow-up conversation happens without any of the original messages present — the assistant only has whatever it chose to keep. Ask it, at that point, where you live. This sounds like a trivial question. It is, in fact, one of the harder open problems in AI agent design, and two of the most visible commercial memory systems — Mem0 and Letta — have made opposite bets on how to answer it.

The problem underneath the problem

The obvious failure mode is an agent that simply forgets the move and says “New York.” That’s a memory problem in the ordinary sense: information fell out of a bounded context window. But there’s a subtler failure that persists even when the fact is never forgotten. If the assistant stored both “lives in New York” and “lives in San Francisco” — which is exactly what happens if it never deletes or overwrites anything — it now has two contradictory facts sitting in storage. The question is no longer “did it remember,” but “does it know which one is current.” Call this the current-value problem: not retention, but resolution.

A recent paper, Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents,1 isolated this specific failure on real conversational data and found it to be surprisingly stubborn. Feeding a frontier model (GPT-5.4) the full, unbounded conversation history got 92% accuracy on knowledge-update questions from the LongMemEval benchmark. Forcing the same model to work from a bounded, self-maintained memory of a few hundred characters — the realistic setting any deployed agent actually operates under — dropped that to 77%, a gap the authors report as statistically significant (p < 0.005) and stable across model scale. Scaling the memory budget 24x, from roughly 300 to 7,150 characters, produced no improvement at all: 28% before, 28% after. The only intervention that moved the needle was training: fine-tuning a small open model (Qwen2.5-3B) with reinforcement learning, rewarding it specifically for reporting the current value and penalizing stale answers, nearly doubled its accuracy on a held-out real-world evaluation set, from 9.0% to 16.7%.2 Even after training, the ceiling was 16.7% — far from solved, but evidence that the bottleneck is a policy problem, not a capacity problem. Bigger memory doesn’t help. A model trained specifically to do this task does, a little.

Bet one: don’t resolve it at write time, rank it away at read time

Mem0 rebuilt its memory pipeline in 2026 around a genuinely different design than its earlier “extract, then consolidate” approach. The new version is single-pass and, notably, ADD-only: when a new fact arrives, older related facts are not deleted or merged, they are simply kept alongside it.3 Conflict resolution is deferred entirely to retrieval time, where a fused ranking signal — semantic similarity, keyword match, entity match, and temporal recency — decides which memory surfaces for a given query. According to Mem0’s own documentation, this is an explicit design choice rather than an oversight: issues describing the semantic-conflict behavior of the ADD-only pipeline have reportedly been closed as “not planned.”

The payoff, on paper, looks strong. Mem0 reports 94.4% on the 500-question LongMemEval suite (up from a 93.4% baseline), and a temporal-reasoning subcategory improving from 93.2% to 97.0% after adding timestamp metadata (when an event happened, whether it’s ongoing, how precise the timing is) to each stored memory.4 All of this while keeping retrieval under roughly 7,000 tokens, versus 25,000+ for full-context approaches.

Bet two: let the agent decide, in the moment, what’s still true

Letta (the successor to the MemGPT research project) took the opposite position: keep an explicit, OS-inspired hierarchy — core memory that lives permanently in the context window, recall memory that is searchable conversation history, and archival memory as long-term cold storage — and let the agent itself decide, mid-reasoning, what belongs where and when something needs to be rewritten.5 There’s no dedicated current-value mechanism and no ranking algorithm doing the resolving; the agent calls a memory-editing function when it judges a fact worth updating. The current-value problem is handed not to infrastructure but to the model’s own judgment, exercised fresh in every turn.

Do these actually contradict the negative result?

At first glance, Mem0’s 97% on “temporal reasoning” looks like a rebuttal to Supersede’s claim that ranking-based fixes don’t work. I don’t think it is, and the reason is worth sitting with, because it’s a more general lesson about how easily benchmark numbers get compared across incompatible conditions.

Mem0’s gain comes from temporal ordering — knowing an event happened after another event, or that a plan is future rather than past. Supersede’s adversarial setting is narrower and meaner: after up to 48 sessions, with only a few hundred characters of memory retained and the raw conversation gone for good, answer what the single current value of one specific fact is, scored by an exact, judge-free string matcher. These are related skills but not obviously the same skill, and “LongMemEval” as a benchmark name covers both without forcing anyone to specify which subset, which version, or which scoring protocol they mean. It’s entirely possible for a system to get much better at ordering events in time while remaining just as fragile at the specific test Supersede constructed. If Mem0’s own bug tracker really does treat certain conflict cases as “not planned,” that reads less like confidence the problem is solved and more like an implicit admission that ranking is a best-effort heuristic, not a guarantee — one that will look great on a typical distribution of user conversations and can still fail on the adversarial tail Supersede was built to probe.

That leaves an open, checkable question rather than a settled one: has anyone run Mem0’s actual pipeline through Supersede’s exact protocol — 48 sessions, ~300-character bound, no judge model — and reported the number? As far as I could find, no. Until someone does, the fair reading is that Mem0’s benchmark strength and Supersede’s negative result are not in conflict; they’re measuring different points on the same underlying problem, and nobody yet knows how badly the gap opens up under truly adversarial conditions.

Letta’s bet has its own vulnerability, and it’s a more direct one. Handing current-value judgment to the model’s own reasoning only works as well as that reasoning does, and there’s independent evidence that base LLMs are inconsistent at exactly this kind of belief maintenance — preserving unrelated facts while updating the one that actually changed, rather than either clinging to stale information or overcorrecting and wiping out things that were still true. That’s not a guarantee failing gracefully; it’s a guarantee that was never actually engineered into the system, dressed up as one because it sounds more agentic to say “the model decides.”

What I take from this

Neither architecture actually solves the current-value problem in the sense Supersede defines it — reliably, under adversarial multi-session drift, without a judge model checking the answer. Mem0 defers the decision to a ranking function tuned for the common case. Letta defers it to the model’s in-context judgment, which is adaptive but has no floor under it. Both are reasonable engineering trade-offs given real constraints (write-time consolidation is expensive; explicit current-value pointers are a real infrastructure commitment), but it’s worth being honest that “reasonable trade-off” and “solved problem” are different claims, and public benchmark numbers have a way of collapsing that distinction if you don’t look closely at what was actually measured.

The open question I’d want answered next: is there a system that treats “what is the current value of this fact” as a first-class, actively-maintained pointer — something closer to a database’s notion of the latest row version — rather than either a ranking signal or a judgment call? That would be a third bet, distinct from both of the ones currently live in production, and it’s the one Supersede’s negative results seem to be pointing toward.

  1. Vedant Patel. “Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents.” arXiv:2606.27472. Accessed 2026-07-30. ↩

  2. Vedant Patel. “Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents.” arXiv:2606.27472v1 (HTML). Accessed 2026-07-30. ↩

  3. Mem0. “Introducing The Token-Efficient Memory Algorithm.” Mem0 Blog. Accessed 2026-07-30. ↩

  4. Mem0. “The Token-Efficient Memory Algorithm Now Has Temporal Reasoning.” Mem0 Blog. Accessed 2026-07-30. ↩

  5. Letta. “Agent Memory: How to Build Agents That Learn and Remember.” Letta Blog. Accessed 2026-07-30. ↩