The three types of agent memory: semantic, episodic, procedural

Semantic memory holds facts. Episodic memory holds experiences. Procedural memory holds how-to. The three words come up in almost every discussion of what an agent should store, and they are worth knowing precisely, because they are used loosely everywhere.

They are also not native to AI. The split is borrowed from decades of human-memory research, applied to language agents in a 2023 paper, and carried into practice mostly as vocabulary rather than as a specification. Borrowed vocabulary behaves differently from a standard: two public sources already disagree about what belongs in the third category, and one vendor’s documentation declares three values and implements one. Anyone putting these words into a schema is putting terms the field is still interpreting into a schema, which is a reasonable thing to do and a better thing to do knowingly.

Where the split comes from

The attribution most of the field makes is to CoALA — Cognitive Architectures for Language Agents, by Theodore R. Sumers, Shunyu Yao, Karthik Narasimhan and Thomas L. Griffiths, submitted to arXiv on 5 September 2023 and last revised on 15 March 2024. I read the paper rather than a summary of it, because the origin of a taxonomy is the one thing a definitional piece cannot afford to get wrong. It holds. The breakdown is in §4.1, headed “Memory”, and it is stated directly.

What the paper actually enumerates is worth quoting rather than summarising: “Under the CoALA framework, language agents explicitly organize information (mainly textural, but other modalities also allowed) into multiple memory modules, each containing a different form of information. These include short-term working memory and several long-term memories: episodic, semantic, and procedural.” Working memory leads the paper’s list, and the three that travel as a set are the long-term ones. The field’s three-item shorthand is a fair summary of the paper’s long-term group and an incomplete summary of its memory section — the transient module it leaves out is, in a running agent, the context window.

The paper does not claim the taxonomy. It credits it onward, twice: “Building on psychological theories, Soar uses several types of memory to track the agent’s state (Atkinson and Shiffrin, 1968)”, and it attaches working memory to Baddeley and Hitch, 1974. So the borrowing runs from 1960s and 1970s psychology, through Soar and the symbolic cognitive-architecture tradition, into a 2023 framework for language agents.

The attribution survives a direct read of the paper. The dates are worth carrying alongside it: a framework submitted in September 2023 is old relative to the systems being built on it, and it was written before most of the products that now quote it existed.

Semantic memory: facts

Semantic memory is the store for things that are true independently of when they were learned. CoALA’s definition is one sentence: “Semantic memory stores an agent’s knowledge about the world and itself.” LangChain, writing for practitioners, calls it “a repository of facts about the world.”

In an implementation, a semantic memory looks like a short assertion with no timestamp doing any work: the user prefers metric units, the production database is Postgres 16, invoices go to accounts@example.com. It is written once, it is expected to stay true, and it is retrieved because something in the current request resembles it — usually by embedding the request, searching a vector index, and pasting the top few results into the call.

The failure mode that follows from the definition is staleness rather than loss. A fact stored without an expiry stays retrievable long after it stops being true, and nothing in a similarity search can tell the difference between a fact that is still correct and a fact that was correct in March. Note also what CoALA includes and LangChain’s shorter gloss does not: knowledge about the agent itself, which is where a system’s own configuration and capabilities sit if they are stored as memory at all.

One consequence of retrieval-by-similarity is worth stating even though neither source states it, because it follows from the mechanism rather than from any published finding. If a fact is fetched because the incoming request resembles it, then how the fact is worded determines when it comes back. The user prefers metric units will surface on a question about units and may not surface on a question about shipping weights, even though it is the relevant fact in both cases. Teams generally discover this by watching a correct memory sit in the store, unretrieved, while the agent gets the answer wrong — which is a retrieval problem wearing a memory problem’s clothes.

Episodic memory: events

Episodic memory stores what happened. CoALA is specific about whose experience is being stored: “Episodic memory stores experience from earlier decision cycles. This can consist of training input-output pairs, history event flows, game trajectories from previous episodes, or other representations of the agent’s experiences.” LangChain’s version is shorter and matches — “storing sequences of the agent’s past actions.”

Both definitions are written from the agent’s side rather than the user’s, which surprises people who arrive expecting episodic memory to mean the conversation we had last Tuesday. It does cover that, because a logged conversation is one of the agent’s experiences, and CoALA’s “or other representations of the agent’s experiences” is written broadly enough to include it. The emphasis is worth noticing anyway: an episode in this vocabulary is a decision cycle with a trajectory, not simply a transcript.

What distinguishes an episode from a fact is that an episode is indexed by having happened. The user prefers metric units is a fact. On 14 August the user asked for imperial units and then corrected themselves is an episode, and it stays true forever regardless of what the preference becomes, because the event does not stop having occurred. In practice this is implemented as an append-only log of turns or tool calls, retrieved by recency, by similarity, or by both.

An append-only store grows without limit, which forces a design decision that no definition makes for you: what to keep. Recency-ranked retrieval quietly answers it by making old episodes unreachable rather than absent, and similarity-ranked retrieval answers it by making rare episodes unreachable instead. That trade is reasoning from the mechanism rather than something CoALA or LangChain documents, and it is the point at which an episodic store stops being a logging decision and becomes a product decision.

Episodes accumulate and facts get replaced, which is why the two behave differently under retrieval even when they sit in the same database.

Procedural memory: how-to, and where the field disagrees

Procedural memory is the one to be careful with, because CoALA and LangChain both place it somewhere that common practitioner usage does not.

CoALA §4.1: “Language agents contain two forms of procedural memory: implicit knowledge stored in the LLM weights, and explicit knowledge written in the agent’s code.” LangChain says the same — procedural memory is “the combination of LLM weights and agent code, which fundamentally determine how the agent works.”

The common practitioner reading is different: procedural memory as skill files, instruction documents, a rewritten system prompt, a playbook the agent updates after a failed tool call. That reading is everywhere in tooling and in how teams talk about their own systems, and it does not match either source above. A skill file is not model weights, and whether it counts as “the agent’s code” is exactly the question neither source answers.

Both readings are in circulation. Only one of them is in the paper the term is usually attributed to, and I am going to take that as far as the evidence goes and no further: two public sources place procedural memory in weights and code, a widespread practitioner usage places it in written instructions, and no source found here adjudicates between them.

CoALA does add two consequences that hold under either reading, and they are the most practically useful sentences in the section. First, procedural memory cannot start empty: “Unlike episodic or semantic memory that may be initially empty or even absent, procedural memory must be initialized by the designer with proper code to bootstrap the agent.” Second, writing to it is the dangerous one — “significantly riskier than writing to episodic or semantic memory, as it can easily introduce bugs or allow an agent to subvert its designers’ intentions.” A system that lets an agent edit its own procedures is doing something categorically different from letting it save a fact, and the paper says so plainly.

Both of those cut the same way for anyone building. An agent whose semantic and episodic stores are empty still runs; it simply knows nothing yet, and it fills up as it goes. An agent whose procedural memory is empty does not run at all, because on either reading of where that memory lives — weights and code, or written instructions — there is nothing there to tell it how to act. Procedural memory is therefore the one of the three that has to be authored before first use and audited after every change to it.

What the vendors actually ship

One vendor, checked against its own documentation, because a declared vocabulary and an implemented one are separate things and only one of them runs.

mem0’s documentation page on memory types lists three values and marks exactly one of them as implemented. procedural_memory is “Implemented”, Python open-source only. semantic_memory and episodic_memory are “Not implemented”, and the page is explicit about what happens if you ask for them: “Only procedural_memory is a real, working value. Calling memory.add(messages, memory_type="semantic_memory") (or episodic_memory) is rejected and tells you to pass procedural_memory instead.” The page describes the underlying enum in the same terms — three values defined, one wired up, and no documented roadmap for the other two.

The interesting detail is which one survived. The type implemented is the one the two cited sources place in weights and code, and it is implemented as something a developer passes to an add() call — which is closer to the practitioner reading than to either source’s definition. The vocabulary and the shipped behaviour are pulling in different directions inside a single product’s documentation.

Two caveats belong on that, both of them about dates. The page carries no publication date, so there is nothing to stamp it with beyond the day it was read, and a validation error is exactly the kind of behaviour a release can change without announcing it. Anyone relying on this should re-read the page rather than cite this one. That is not a hedge peculiar to mem0 — an undated vendor documentation page is a claim with no shelf life attached, and there are a great many of them.

mem0’s documentation is one vendor’s documentation, and nothing here is a statement about anybody else. I have not audited the category and this piece does not try to.

A borrowing, not a standard

Two things established above are worth putting next to each other. Two public sources — one of them the paper the taxonomy is attributed to — place procedural memory in model weights and agent code, while a widespread practitioner usage places it in written instructions. And one vendor’s own documentation declares three memory values while implementing one.

Neither of those is a failure of anybody’s. Both are what a borrowed vocabulary looks like while it is still being interpreted. Atkinson and Shiffrin were modelling human recall in 1968, and Baddeley and Hitch were modelling working memory in 1974; neither project was trying to decide what a piece of software should write to disk. Soar borrowed from them for a symbolic architecture, CoALA borrowed from Soar for language agents in 2023, and the current tooling borrowed from CoALA. Each step was a reasonable move and each one carried the terms further from the experiments that produced them.

The practical consequence is small and specific. These three words are useful for talking about retrieval patterns you have to design anyway — something that stays true, something that happened, something that determines behaviour — and they are unreliable as an interface contract. If a system advertises support for a memory type, the thing to read is what its documentation says happens when you ask for it. And if you are choosing where any of it lives, the prior question is which parts of it live outside the model at all, which is the subject of CM-005, What agent memory actually is, and why the context window isn’t it.