Prompts, Models and Settings as Versioned Artefacts

Behaviour = prompt + model + settings + data

The behaviour of an LLM feature is determined by a bundle: the prompt templates, the model name and version, sampling settings (temperature, max tokens), tool definitions, retrieval configuration (chunking, embedding model, k) and any fine-tuned adapters. Change any piece and behaviour changes. So treat the bundle as a versioned artefact: keep prompts in files under version control (not scattered strings), review changes in pull requests, give each release a version ID, and record that ID with every request in your logs. Pin model versions where the provider allows dated identifiers, because silent provider updates can change results; plan for deprecations by re-running your evaluation set against the replacement before switching. Being able to answer "which prompt and model produced this answer last Tuesday?" is the foundation for debugging, audits and rollbacks.

Version it, gate it, constrain it

Treat prompts, models and settings as versioned artefacts with release gates, and constrain outputs so code can trust them.

A release manifest

One file that pins everything that defines behaviour. Illustrative; names are placeholders.

{
  "release": "support-assistant-2026.10.2",
  "prompt": { "file": "prompts/support_v14.md", "sha256": "<hash of the file>" },
  "model": { "provider": "<provider>", "name": "<dated-model-id>", "temperature": 0.2, "max_tokens": 500 },
  "retrieval": { "embedding_model": "<embedding-id>", "chunk_tokens": 300, "top_k": 5, "reranker": "<reranker-id>" },
  "tools": ["get_order", "propose_refund"],
  "eval": { "set": "evals/support_v7.jsonl", "pass_rate": 0.93, "date": "2026-10-01" }
}

Pin dated model identifiers

Where available, use an identifier that names a fixed model version, not a moving alias.

Quick check: Why record a release version ID with every request?

• So you can trace any answer to the exact prompt, model and settings
• To make requests larger
• Because APIs require it
• It has no use