The stateless machine
Every conversation looks continuous. It isn’t. Each turn re-sends everything.
A useful mental model:
output = f(training, prompt, tools / RAG, reasoning)
Four inputs shape every response: what the model was trained on, what’s in the prompt, what it retrieved via tools, and how much reasoning effort it spent. The model has no memory between turns. What feels like memory is your harness re-submitting the prior conversation, instructions, and any retrieved context on every request.
What actually crosses the wire
On each turn, Claude Code builds a fresh request containing:
- The model’s training (baked into the weights: not sent over the wire, but it determines what the model “knows” before you say anything)
- The system prompt (Claude Code’s instructions about how to operate and what tools exist)
- Your conversation so far (every message from this session, re-transmitted)
- Any tool-call results from earlier turns (files read, bash output, web fetches)
- The current user message
The model receives that whole payload, produces a response, and returns nothing stateful. The next turn starts the same construction from scratch.
Why the four inputs matter
- Training is fixed. The model knows what it knew at the end of its training. You can’t change this from a session.
- Prompt is everything you control on this turn: your messages, the
AGENTS.md, any Skills the harness loads. This is the most expressive surface you have. - Tools / RAG is everything the model pulls in agentically (file reads, grep, web search, MCP tool calls): agentic retrieval. (MCP is the Model Context Protocol, the standard way Claude Code connects to external tools and data sources; Modes and abilities covers it.)
- Reasoning is how hard the model thinks before answering, controlled via reasoning effort and how you phrase the ask.
Each input is independent. Improving any of them improves the output. Most quality gains, in practice, come from sharpening the middle two: what’s in the prompt and what the model retrieved.
“Memory” is just re-submission
“Memory” features in LLM products (including anything that lets Claude “remember” a fact across sessions) are syntactic sugar over re-submission. Behind the scenes, they store text somewhere (a file, a database, a session log) and inject it into the prompt on future turns. There’s no persistent state inside the model.
This is why rewinding works the way it does (see Talking to Claude). There’s nothing inside the model to undo; rewinding just re-builds the prompt without the bad exchange.
Implications
- Long sessions cost more. Each turn re-sends everything that came before. By turn 50 of a session, you’re sending far more tokens per turn than you were at turn 1.
- Long sessions also degrade. The model’s attention is spread across more stuff, including potentially-stale stuff. See the context budget.
- Starting fresh is cheap and often correct. If a session has accumulated noise, start a new one with a tight prompt.
Resources
- Karpathy: Deep dive into LLMs, the canonical walk through inference and training.
- 3Blue1Brown: Attention in transformers, why the context window shape matters.
- Anthropic: Effective context engineering for AI agents