Structured Output and Validating Replies

Ask, then verify

When code consumes the reply, ask for JSON with exact keys and allowed values, give an example, and validate what comes back. Both providers offer features that make valid output much more likely: tool/function calling with a schema, and structured output / JSON-schema modes on supported models. Even then check types, ranges and allowed values, and handle failures: models sometimes wrap JSON in prose or code fences, get truncated at max_tokens, or choose a value outside your list. Extract the JSON object, parse it with json.loads, validate against your rules, and retry with the error message or fall back safely. Never pass model output to eval, a shell or SQL without strict checks.

Extracting and validating JSON, run

I ran this plain-Python (standard library only) example. Clean JSON passes; JSON inside prose and a code fence is extracted and passes; a category outside the allowed set is rejected; a reply with no JSON is reported. These strings stand in for model replies.

import json

def parse_json_reply(text):
    """Models sometimes wrap JSON in prose or code fences; extract and validate it."""
    start, end = text.find("{"), text.rfind("}")
    if start == -1 or end == -1:
        return None, "no JSON object found"
    try:
        data = json.loads(text[start:end + 1])
    except json.JSONDecodeError as e:
        return None, f"invalid JSON: {e.msg}"
    if data.get("category") not in {"billing", "technical", "other"}:
        return None, "category not allowed"
    return data, None

for reply in ['{"category": "billing"}',
              'Sure! Here you go:\n```json\n{"category": "technical"}\n```',
              '{"category": "refunds"}',
              'I cannot help with that.']:
    print(parse_json_reply(reply))

Output:

({'category': 'billing'}, None)
({'category': 'technical'}, None)
(None, 'category not allowed')
(None, 'no JSON object found')

Show one example of the exact JSON

A single example output in the prompt usually fixes key names and formatting more reliably than a long description.

Quick check: Why validate JSON even when using a structured-output feature?

• It is forbidden to trust models
• JSON cannot be parsed
• Values can still be wrong or outside your business rules, and failures happen
• It makes replies shorter