# Case Study: A Support-Reply Drafting Endpoint — Claude API / OpenAI API Basics

Source: https://www.geekswithgeeks.com/en/llm-apis/z-case

> Design a backend endpoint that drafts replies safely and cheaply.

## The design

Goal: agents click "Draft reply" in a support tool. **Request path**: the browser calls **your backend** (authenticated); the backend loads the ticket, **pseudonymises** personal data, builds the prompt (stable policy text first so it can be **cached**, ticket last), and calls the provider with `max_tokens` set, a **timeout**, and SDK **retries with backoff**. **Output**: the model returns JSON `{category, reply}`; code **extracts and validates** it (allowed categories, length limit), restores placeholders, **escapes** the text for display, and on failure retries once with the error message, then falls back to "could not draft, please write manually". **Tool use** (optional): a read-only `get_order_status` tool with the user id taken from the session, a 3-step loop cap and logging. **Streaming** relays text to the UI and cancels upstream when the agent closes the panel. **Cost/safety**: usage recorded per team and feature, daily token budget, keys in a secrets manager, redacted logs, **human reviews every draft** before sending. **Testing**: fake-server tests for retries, 400/429/5xx, streaming, tool loop and validation; a 100-ticket evaluation set run before each prompt or model change.

## A dependable LLM feature

Good prompts, validated output, retries, budgets, security and tests combine into a feature you can ship.

![Four habits: validate, retry, measure, protect.](assets/figures/llm-apis/section-8-map.svg) — Figure 8.1 — Validate, retry, measure and protect.

## The design on one page

Each line maps to a section of this course.

```text
Path        browser -> YOUR backend (auth) -> provider; key never in the browser          (Sec 1)
Prompt      stable policy first (cacheable), pseudonymised ticket last, max_tokens set     (Sec 2, 6)
Output      extract JSON -> validate -> restore placeholders -> escape -> display          (Sec 2, 7)
Reliability SDK retries+backoff, timeout, 1 retry on invalid JSON, manual fallback         (Sec 5)
Tools       read-only get_order_status, user id from session, 3-step cap, logs             (Sec 4)
Streaming   relay text; cancel upstream when the panel closes                              (Sec 3)
Cost/safety usage per team, daily budget, secrets manager, redacted logs, human review     (Sec 6, 7)
Testing     fake-server tests + 100-ticket eval set before each change                     (Sec 7)
```

## Ship behind a feature flag

Roll out to a small group first and watch cost, errors and agent feedback before widening.

**Quiz:** Why does the backend, not the browser, call the provider?

- [x] So the API key stays secret and users are authenticated by you
- [ ] Browsers cannot send JSON
- [ ] It makes tokens free
- [ ] Backends are faster than browsers

*Answer:* So the API key stays secret and users are authenticated by you. Anything in the browser can be extracted, including keys.