Designing Tools the Model Can Use

The description is the user interface

A model chooses tools from their name, description and schema alone, so these are your UI. Good practice: one clear job per tool, a verb-noun name (get_order_status), a description that says when to use it and what is returned, constrained arguments (enums, patterns, min/max) and concise results (not megabytes). Prefer task-oriented tools over thin wrappers around every API endpoint: "find the latest unpaid invoice for a customer" beats five low-level calls the model must chain. Return structured data when the client will process it, and keep the total number of tools small, since every tool definition is added to the context on each request.

The cost of many tools, run

I ran this with plain Python 3 (standard library only). Three tools cost about 210 tokens of descriptions per request. Sixty similar tools cost 4,500 tokens on every request, which is 45 million tokens over 10,000 requests, before any real work. The per-tool token counts are illustrative.

# Every tool description is sent to the model on every request: it costs tokens.
tools = {"get_order_status": 60, "propose_refund": 80, "search_docs": 70}
print("3 tools:", sum(tools.values()), "tokens of descriptions per request")

many = {f"tool_{i}": 75 for i in range(60)}
per_request = sum(many.values())
print("60 tools:", per_request, "tokens per request;", per_request * 10_000, "tokens per 10,000 requests")

Output:

3 tools: 210 tokens of descriptions per request
60 tools: 4500 tokens per request; 45000000 tokens per 10,000 requests

Say when NOT to use the tool

One sentence such as "Do not use for refunds; use propose_refund" prevents many wrong calls between similar tools.

Quick check: Why keep the number of tools small?

• Small numbers are required by JSON
• Tools expire quickly
• The protocol allows only five
• Every tool definition consumes context tokens and can cause wrong choices