Retry With Feedback

Fail, explain, try again

When validation fails, do not just resend the same prompt. Append a short, specific error message ("your last reply was not valid JSON: expected a value; reply with JSON only") and call again, with a small maximum number of attempts (2 or 3). Log every failure and the final fallback (a safe default, a human queue, or an error to the user). Count how often retries happen: a high rate means the base prompt needs fixing, not more retries. Retries cost tokens and latency, so also set timeouts.

Retry loop, run

I ran this plain-Python (standard library only) example. A small function stands in for the model so the result is repeatable. The fake model returns broken JSON on the first call and valid JSON on the second. The loop succeeds on attempt 2 after appending the error message to the prompt.

import json

calls = {"n": 0}
def fake_model(prompt):
    calls["n"] += 1
    if calls["n"] == 1:
        return "Here you go: {category: billing}"
    return '{"category": "billing"}'

def ask_with_retry(prompt, max_tries=3):
    for attempt in range(1, max_tries + 1):
        raw = fake_model(prompt)
        try:
            return attempt, json.loads(raw)
        except json.JSONDecodeError as e:
            prompt += f"\nYour last reply was invalid JSON ({e.msg}). Reply with JSON only."
    return max_tries, None

print(ask_with_retry("Classify: I was charged twice."))

Output:

(2, {'category': 'billing'})

Track the retry rate

If 20% of calls need a retry, improve the prompt or use schema-constrained output instead of paying for retries forever.

Quick check: What should you add to a retry prompt?

• A specific description of what was wrong with the last reply
• Nothing
• A longer persona
• A random seed only