Defining "Done"

Verifiable beats claimed

A model saying "I'm done" is a claim, not proof. Where possible define done as something code can verify: tests pass, a file exists, a schema validates, a record shows the new status. Then the loop ends on evidence, and a false "done" is caught.

Four ways a loop ends

A loop can finish successfully, hit a hard limit, detect no progress, or be stopped from outside.

Check before accepting

If the check fails, feed the failure back as the next message instead of returning. tests_pass runs your real test command.

if reply.stop_reason == "end_turn":
    ok, report = tests_pass()
    if ok:
        return final_text(reply)
    messages.append({"role": "user",
                     "content": f"Tests still fail:\n{report}\nPlease fix them."})

Bound the retries too

"Feed the failure back" is itself a loop. Count how many times verification failed and give up after a small number, such as three, with a clear report.

Quick check: Which is the strongest "done" condition?

• The model says it is finished
• The loop ran 10 times
• The test suite passes
• The reply was long