# Streaming in Applications: UI, Backends and Cancellation — Claude API / OpenAI API Basics

Source: https://www.geekswithgeeks.com/en/llm-apis/s-ui

> Pass streams through your backend to the browser and handle problems mid-stream.

## Your backend is the middleman

Browsers must not hold your API key, so the browser calls **your backend**, which calls the provider with streaming and **forwards the pieces** to the browser (for example over Server-Sent Events or WebSockets). Design for the unhappy paths: the **user closes the page or presses stop** (cancel the upstream request so you stop paying for tokens), the **stream fails midway** (show what arrived and offer retry; do not silently treat partial text as complete), a **proxy buffers** the stream (disable buffering or the text arrives in one lump), and **moderation or validation** that needs the full reply (validate after the stream ends, or stream only after a safety check). Record usage from the final event for billing.

## A streaming relay (illustrative)

Sketch of a backend endpoint in the style of a web framework; function names are placeholders. Not run here.

```python
def chat_stream(request):
    user = authenticate(request)                       # your auth, not the provider key
    check_rate_limit(user)
    def events():
        try:
            with client.messages.stream(model=MODEL, max_tokens=500,
                                         messages=load_history(user, request)) as stream:
                for piece in stream.text_stream:
                    if request.is_disconnected():          # user pressed stop / closed the page
                        return                              # leaving the block closes the upstream request
                    yield f"data: {piece}\n\n"
                final = stream.get_final_message()
                record_usage(user, final.usage)             # bill from the final event
        except Exception:
            yield "event: error\ndata: stream failed, please retry\n\n"
    return EventStreamResponse(events())
```

**Quiz:** Why cancel the upstream request when the user presses stop?

- [x] So you stop paying for tokens nobody will read
- [ ] To delete the key
- [ ] To speed up the model
- [ ] It is required by HTTP

*Answer:* So you stop paying for tokens nobody will read. Generation continues and bills until the request is closed.