Lesson 18 / 26
Third-Party and Vendor Governance
Assess AI suppliers on security, data use, evaluation evidence, change control and exit terms.
You inherit your vendor's risks
Most organisations use AI from suppliers. Before buying, ask: How is our data used, stored and retained, and is it used to train models? What security certifications and controls exist? What evaluation evidence and known limitations can they share? How are model changes communicated (a silent model update can change behaviour)? Which sub-processors and locations are involved? What are the audit rights, incident-notification duties and exit terms? Your obligations to customers and regulators do not disappear because a vendor built the model.
A vendor questionnaire
Send it before procurement and keep the answers in the inventory entry.
Data Is our data used to train your models? Retention? Region of processing?
Security Certifications (e.g. ISO 27001, SOC 2)? Penetration test summary?
Quality Evaluation results and known limitations for our use case?
Change Notice period for model/version changes? Can we pin a version?
Incidents Notification time after a breach or harmful-output incident?
Supply Sub-processors and their locations?
Exit Data export and deletion on termination? Audit rights?Quick check: Why ask whether you can pin a model version?
- A silent model update can change behaviour and break your tests
- Pinning makes the model cheaper
- Versions do not exist
- It is illegal to update
Answer
A silent model update can change behaviour and break your tests — Version control lets you re-run your evaluation before accepting a change.