Lesson 4 / 25

State and Memory

Distinguish short-term message history from long-term memory stored outside the model.

The model remembers nothing

Each API call is independent. "Memory" is the messages list you resend every turn. Long histories cost tokens and can exceed the window, so real systems trim or summarise old turns and store lasting facts (user preferences, past results) in a database or file that is loaded when relevant.

A notebook

A person with no memory keeps a notebook and rereads the relevant page at the start of each day. The notebook is storage; the page they read is the context.

Trim, do not truncate blindly

When history grows, summarise old turns but keep the original task, key decisions and any tool results still needed. Cutting the oldest messages blindly can drop the very instruction the task depends on.

Quick check: Why does a conversation seem to have memory?

  • The model stores it permanently in its weights
  • The app resends earlier messages each turn
  • The server never restarts
  • Browsers cache the model
Answer

The app resends earlier messages each turn — The history is part of every request, so the model sees it again each time.