How agent memory gets written: hot path versus background

A memory can be written at one of two moments: during the turn that produced it, or after that turn has finished. LangChain names the pair — in the hot path, or in the background — and picking one is a trade of latency against availability, with no third option that escapes both costs.

The choice is usually made once, early, by whoever wires up the storage call, and then it quietly sets the ceiling on what the system can do for the rest of its life. An agent that writes in the background cannot answer a follow-up about something it was told forty seconds ago. An agent that writes in the hot path makes every user wait for a save none of them asked for.

Writing during the turn

Writing in the hot path means the save happens before the user sees a reply. LangChain’s description: “the agent system explicitly decides to remember facts (usually via tool calling) before responding.” The agent has to make a judgement mid-turn, with only the turn in front of it, about whether something is worth keeping — and it makes that judgement with less information than it would have had five turns later, when it might be clear whether the detail mattered.

What the approach buys is immediacy. The memory exists before the reply is composed, so it is available to the rest of that same reply and to any question that arrives a second later. Nothing about the store’s freshness has to be explained to the user, because there is no lag to explain.

What it costs is stated plainly in the same source: “introducing some extra latency before any response is delivered.” The user is paying, in wall-clock time, for a durability guarantee they never requested and cannot see. A tool call, a similarity search over existing memories and a write all land between the question and the first token of the answer.

In the hot path, the write is in the user’s critical path.

Writing afterwards

Writing in the background moves the save out of the turn entirely. LangChain again: “a background process runs either during or after the conversation to update memory,” and the stated drawback is the obvious one — “the memory is not updated immediately.”

“Not immediately” is worth making concrete, because it does not feel like a storage detail from the outside. A user says call me Sam rather than Samuel, asks something else twenty seconds later, and gets an answer addressed to Samuel. The extraction job has not run yet. From the user’s side that is indistinguishable from not having been listened to, which is a reliable way for a memory system to lose a user’s confidence while working exactly as designed.

The compensating advantage is real. A background process reads the finished conversation rather than a turn in progress, so it can weigh a detail against what came after it, batch several facts into one pass, and take as long as it needs without anybody watching a cursor blink. Careful extraction is easier when nothing is waiting on it.

In the background, the write is off the critical path and the freshness guarantee goes with it.

What each choice hands you

Of the four problems below, the first is the only one I could find a published vendor answer for. The other three follow from the mechanism rather than from any source that documents them, which makes them a reading rather than a demonstration — worth saying, because a reasoned consequence and an observed one are not the same evidence.

Contradiction. A new fact arrives that disagrees with a stored one, and something has to decide whether it replaces the old fact, sits beside it, or is discarded. mem0 documents one answer, in a preprint written by the vendor’s own team: the update phase retrieves the most semantically similar existing memories, hands them to the model together with the candidate fact, and lets the model pick an operation — “ADD for creation of new memories when no semantically equivalent memory exists; UPDATE for augmentation of existing memories with complementary information; DELETE for removal of memories contradicted by new information; and NOOP when the candidate fact requires no modification to the knowledge base.” That resolves contradiction by making every write a model decision, which also means contradiction handling inherits whatever error rate the model has on that task.

Failure mid-write. A turn can succeed while its write fails, and the user has no way to tell the difference: they got a good answer and they reasonably believe the thing they said was recorded. In the hot path the failure at least happens inside the turn’s own control flow, where it can be caught and surfaced. In the background it happens in a job nobody is looking at, and the first evidence of it is a system that has forgotten something a user is certain it was told.

Observability. Neither approach tells a user what was written about them, and by default neither tells an operator either. A memory store that cannot be inspected turns every complaint into an investigation, because “it forgot” and “it never saved that” and “it saved the wrong thing” all present identically from the outside.

Ordering. Two writes about the same fact, produced in one order and resolved in another, give a final state nobody chose. Background workers make this easy to arrange by accident: two extraction jobs from two turns of the same conversation, running concurrently, each looking at a store that the other is about to change.

The techniques next door

A separate family of techniques manages what is in the window during a long task rather than what gets written out of it, and the two are easy to confuse because both are described as memory work. Anthropic documents three. Compaction is “taking a conversation nearing the context window limit, summarizing its contents, and reinitiating a new context window with the summary.” Structured note-taking is the technique “where the agent regularly writes notes persisted to memory outside of the context window. These notes get pulled back into the context window at later times.” Sub-agent architectures hand a task to a specialist that “might explore extensively, using tens of thousands of tokens or more, but returns only a condensed, distilled summary of its work (often 1,000-2,000 tokens).”

Structured note-taking is the one of the three that crosses the boundary, because a note persisted outside the window is a write in the sense this piece means, and it has the same two moments available to it. Compaction and sub-agent summarisation stay inside a single task’s lifetime. Why that boundary matters at all is the subject of CM-005, What agent memory actually is, and why the context window isn’t it.

What to ask about an implementation

Four questions, each answerable by reading a system’s own documentation rather than by testing it.

When does the write happen — before the reply is sent, or after? If the documentation does not say, assume background, because background is the default that costs a vendor nothing to demonstrate.

What happens when a new fact contradicts a stored one, and what decides? A specific answer — a model call, a timestamp rule, an append-only log — is a good sign. Silence usually means append-only, which is a decision even when it is not described as one.

If a write fails, does anything anywhere say so? Ask for the failure path, not the success path.

Can a user see what has been stored about them, and remove one item? A system that cannot show a user the store cannot show an operator the store either, and every argument about what it remembered will be conducted without evidence.