WikiTrace Links Agent Memories to Their Evidence and Cuts Memory-Induced Failures to 4.5%
KAIST AI's WikiTrace links every stored agent-memory lesson to the trajectory it came from, so agents can check lessons before acting. Memory-induced regression falls from 6.5% with unchecked lessons to 4.5%, while scores rise from 87.0 to 90.2.
Every team running coding agents with memory has seen the failure mode: the agent performs worse on a task after a memory layer is switched on. A paper published November 10 by KAIST AI researchers — currently the #1 paper of the day on Hugging Face Daily Papers — names that failure, measures it, and ships a working fix. The name matters for operators: memory-induced degradation is when an agent fails a task it would otherwise solve because it follows a lesson that is wrong, out of scope, or misread.
The problem: lessons without evidence
Agent memory systems typically distill past trajectories into concise lessons, labeled only with whether the past task succeeded or failed. Those lessons are then handed to the agent without the trajectories they came from. The agent has no way to check them.
The project page walks through a concrete ALFWorld example. An agent’s experience notes claim that a lamp task completes when the object is at the lamp’s location and the lamp is on. Guided by that note, the agent positions a vase next to a desklamp, switches it on and declares the task complete — without ever picking up the vase. The correct rule required holding the object. The task fails, and it fails because of memory, not despite it.
The scale is not trivial. The paper’s discussion thread notes that simply rerunning the agent without memory already flips 9.7% of solved tasks — a number that puts many published memory gains in perspective.
WikiTrace: keep the path back to the evidence
The proposed system, WikiTrace, stores each lesson as three linked layers:
| Layer | Contents |
|---|---|
| Insight | One concise guideline: what to do, when it applies, which run it came from |
| Compilation | The compiler’s diagnosis of why the source run succeeded or failed |
| Raw trajectory | The run itself, unchanged: task, actions, observations, outcome |
On a new task, the agent starts from all insights in its first message. Whenever an insight lacks the detail needed to act, conflicts with another insight, or comes from a failed run, the agent descends through those links with terminal tools and reads the source run before acting.
The results, averaged over eight suites from five benchmarks (ALFWorld, ScienceWorld, PlanCraft, τ³-bench Retail and AppWorld):
| Method | Score | Regression |
|---|---|---|
| No Memory | 79.5 | 9.7% |
| Insights Only (same lessons, no links) | 87.0 | 6.5% |
| WikiTrace | 90.2 | 4.5% |
The same lessons with links score 3.2 points higher than without them on average, and the repository reports +8.5 points over the best prior memory method with the lowest Regression at 4.5%. Regression here means the rate at which memory turns would-be successes into failures; for the No Memory baseline it comes from rerunning the agent without memory.
Why the links matter most when things go wrong
The most operationally interesting finding is when the descent mechanism pays off. In the community discussion, descents after a blocked attempt lead Insights Only by 17.3 points, while routine pre-checks add only 3.4. The evidence links matter most exactly when something has already gone wrong — the moment a generic memory system would keep steering the agent into the same wall.
The authors also candidly report a failed alternative: in an early version, the agent received only indexes and decided what to open. It often acted after reading summaries, or skipped memory entirely. Front-loading insights — then letting the agent drill down on demand — is the design that worked.
What coding-agent builders can do on Monday
The code is public under Apache-2.0, and the pattern does not require adopting the whole harness:
- Store provenance with every memory item. When your harness writes a lesson to memory, persist the source transcript (or a pointer to it) alongside the distilled rule. This is the same write-provenance discipline covered in Memory Is an Input: Write-Provenance Rules for Shared Agent Memory.
- Give the agent read access to the source. Mount memory read-only and make the raw trajectory reachable with a tool call, so checking a lesson costs no environment steps.
- Trigger checks on doubt, not on every task. Descend when an insight lacks detail, conflicts with another, or came from a failed run. Unconditional checking added little; targeted checking added a lot.
- Measure Regression, not just Gain. The repository’s
analyzestep reports both: Gain is the share of tasks No Memory fails that your method solves; Regression is the share it breaks. If you only track the first, silent degradation is invisible.
The repository reproduces the pipeline in three commands — build compiles memory-free training runs into linked insights, run evaluates a method with that memory, analyze compares against the No Memory baseline — requiring Python 3.11+, uv, Node 20 and Docker, with five benchmark installers.
Limits
The benchmarks are text-world and tool-use environments, not large real-world codebases; every insight currently goes into the first message, and the authors acknowledge that similarity-based retrieval will be needed as memory grows. The results are also self-reported means over three trials. Even so, the core finding — that unchecked distilled lessons actively break working agents, and that linking them back to evidence measurably reverses the damage — is directly actionable for anyone operating agents with persistent memory, a topic we have covered in Cross-CLI Memory That Survives the Session and in Beacon’s cross-harness memory layer.
Don’t just trust your agent’s memory. Give it the receipts.
