Offloading to Files and Memory

Keep a pointer, not the data

Instead of keeping a 20,000-token document in the conversation, save it to a file and keep only its path and a two-line description in context. The agent reads the file, or a part of it, only when needed. The same idea works for a notes file where the agent records findings so they survive trimming and restarts.

A tool that returns a reference

The big content goes to disk. The model sees a short handle and can call read_chunk for a specific slice.

def fetch_report(url: str) -> str:
    path = save_to_workspace(download(url))
    return f"Saved to {path} ({size_kb(path)} KB). Use read_chunk(path, start, length)."

Quick check: What stays in context when you offload a large document to a file?

• The whole document
• Only a short reference such as the path and description
• Nothing at all
• A copy for every turn