Context, memory, and state: three words for three different things
Context is the set of tokens a model can see on one call. Memory is whatever survives that call, held outside the model and put back in front of it later. State is the agent’s current working situation — the goal, the plan, the half-finished tool call, the variables it is holding — which lives outside the model too, and gets rendered into context each turn. The context window is none of those three: it is a number, the maximum tokens one call may contain.
Almost every explanation of agent memory spends its first three paragraphs re-deriving those four terms before it can say anything, because nobody has fixed them. Fixed once, here, so the derivation does not have to happen again.
Why the four words fight
They arrived from three different places and were never reconciled.
State comes from systems programming, where it means the contents of a running process at a moment in time, and where “stateless” is a property you design for on purpose. Memory comes from cognitive science by way of the symbolic cognitive-architecture tradition — semantic, episodic and procedural are borrowed terms with a fifty-year history behind them, which CM-006, The three types of agent memory, traces. Context is LLM plumbing: it names the input buffer, and it acquired a conceptual meaning afterwards, because the buffer turned out to be where all the interesting decisions land.
Three vocabularies, one system, no translation layer. So the same sentence means different things to the person who wrote the retry loop, the person who designed the store, and the person who wrote the prompt — and all three are using the words correctly for their own tradition.
The collision is visible in the field’s own reference texts. Anthropic’s definition of context engineering describes the practice as “considering the holistic state available to the LLM at any given time”, and elsewhere in the same piece calls the system prompt, tools, external data and message history “the entire context state”. That is state used as a synonym for what is currently in the window, in the document that most people take their definition of context from.
The words are not being used sloppily. They are being used precisely, by people with different training, about overlapping things.
Context: what the model can see on this call
The narrow definition is the useful one, and Anthropic states it plainly: “Context refers to the set of tokens included when sampling from a large-language model.” That is all it is. Everything the model will consider — the system prompt, the tool schemas, the message history, the retrieved documents, the user’s actual question — is in that set, and nothing outside it exists as far as this call is concerned.
Two consequences follow immediately. Context is assembled, by your code, before every call: it is an output of your system, not an input to it. And context is per-call, so it has no history — the model cannot tell whether a fact reached it because a user typed it, because a retrieval step fetched it, or because a summariser wrote it three hours ago. Provenance is something your application knows and the model does not.
Context is a set, not a store. It is rebuilt every call.
Memory: what survives the call
Memory is what persists between calls, and it persists somewhere that is not the model. Models are stateless in the systems sense — CoALA, the 2023 paper most of this vocabulary comes from, opens its memory section with exactly that: “Language models are stateless: they do not persist information across calls.”
So memory is machinery: a place to put things, a step that writes to it, and a step that fetches from it and drops the result into context. A stored memory that nothing retrieves changes no answer the system gives. CM-005, What agent memory actually is, covers the store-plus-fetch shape; CM-007, How agent memory gets written, covers the write half.
The relationship to context is one-directional and worth stating explicitly: memory only affects behaviour by becoming context. Retrieval is the verb that converts one into the other. Everything in the store that retrieval does not select is, for the duration of that call, indistinguishable from data you never collected.
State: what the agent is in the middle of
State is the awkward one, because it genuinely overlaps both of the others, and because two communities use it for different scopes.
In the narrow systems sense, an agent’s state is its current working situation: the goal it is pursuing, the plan it has committed to, the tool call it is waiting on, the loop counter, the variables it parsed out of the last model response. This is not memory in the “what did the user tell me last month” sense, and it is not context either — it is a data structure your runtime owns, which happens to be projected into context each turn.
CoALA names this directly and is careful about the distinction. Its working memory “maintains active and readily available information as symbolic variables for the current decision cycle”, and the paper explicitly separates that from the common practitioner shorthand of treating the model’s own context as the working memory: “CoALA’s notion of working memory is more general: it is a data structure that persists across LLM calls. On each LLM call, the LLM input is synthesized from a subset of working memory.”
That sentence is the cleanest available statement of how the three relate. State is a structure that outlives the call; context is the slice of it, plus whatever retrieval added, that the model actually sees this time.
State overlaps memory because both persist across calls, and overlaps context because it is rendered into every one. What separates it is scope: state is about the task in progress, memory is about what remains true after the task ends.
The context window is a container, not a concept
The context window is the model’s maximum token count for a single call. It is a property of the model you chose, it is measured in tokens, and it belongs in the same mental category as a disk quota. It is not a design idea and it holds nothing between calls.
This matters because “we need a bigger context window” and “the agent doesn’t know enough” are answers to different questions, and the first is a purchasing decision while the second is an architecture one. Anthropic’s framing of the limit is not really about the number anyway: context “must be treated as a finite resource with diminishing marginal returns”, because a model’s recall degrades as the token count rises — a degradation their piece names context rot. Room and reliability are not the same axis.
The four, side by side
| Term | Lifetime | Where it lives | “More of it” means |
|---|---|---|---|
| Context | One call | Assembled in the request | More tokens sent, for better or worse |
| Memory | Indefinite | A store outside the model | A bigger corpus to retrieve from |
| State | The current task | Your runtime’s data structures | A more complicated agent |
| Context window | N/A — it is a limit | A number in the model’s spec | A different model |
Using the distinction
The distinction earns its keep in design conversations, where it turns one vague complaint into four separable questions. When somebody says an agent “forgot”, ask which of these failed. Was the fact never written to memory? Was it written and not retrieved? Was it retrieved and then squeezed out of the assembled context by something else? Or did the runtime drop the task state on a restart, so there was nothing to forget?
Those are four different bugs with four different owners, and the reason teams debug them slowly is that all four present to a user as the same sentence.