Lesson 17 / 29

Managing Long Histories: Trim and Summarise

Keep the message list within the model's context window.

A growing list is a growing bill

Message history grows with every turn and tool call until it exceeds the context window and the bill. Strategies: trim to the last N messages or a token budget (keeping the system message and any pinned facts); summarise older messages into one short message that you keep instead; filter bulky tool outputs (store the raw data elsewhere and keep a short result in the messages); and move durable facts to long-term memory. Do the trimming in a dedicated node or a message-preparation step before the model call, so the stored history stays complete for audit while the model only sees what it needs.

Trim for the model, keep the full log

Pass a trimmed copy to the model call but keep the complete history in the checkpoint for debugging and audits.

Quick check: Why summarise old messages?

To disable tools
To delete the thread
To keep context short and cheap while preserving key information
To increase the recursion limit

Answer

To keep context short and cheap while preserving key information — Summaries compress history so it fits the window.