Draft for Tamir's review. Not published.
Your data and what the vendor may derive from it
Ask for the exact clauses on training, retention and derived data. Require no training on your data, or an opt-out confirmed in writing. Most vendors offer enterprise terms without training use when asked. Check that subcontractors are bound by the same terms.
Match the terms to the data each vendor will see. If the service is shared between customers, ask for evidence of isolation on every access path, including export, debugging, administrator access and restored backups. A promise of isolation is not evidence of it. Write down what you own at the end, including labels, guidelines, configuration and any fine-tuned model.
Performance, wrong answers and model changes
Define quality on your own test set, with thresholds, monitoring you can see and the right to suspend if they fail. Require logs of inputs and outputs and the right to audit them. Set times for correcting errors and a process for affected customers.
Models degrade as data changes. Put the retraining schedule and its cost in the fee, so a drop in performance is never a paid change outside the contract. Keep the right to pause or replace the model, with a fallback such as rules or a challenger model.
Depending on your seat
If you're on the board and the model will influence decisions a regulator holds you responsible for, require what the regulator will ask you for. That means model documentation and versions, explanations per decision you can give customers, and the right to challenge the model. Ask compliance to confirm the arrangement.
If you're the CEO and a department wants a fast signature for a discount, ask three things first. What is the tool approved for, where does the data go, and what evidence shows it works on cases like yours? A discount deadline is not a reason to skip any of them. For a program with vague outcomes, require named deliverables per phase and a stop point after each.
What to check before you decide
- Read the clauses on training, retention and derived data, and require no training use or a confirmed opt-out.
- Classify the data each vendor will see, and check that subcontractors carry the same terms.
- Define quality measures on your own test set, with monitoring you can see and the right to suspend.
- Require logs of inputs and outputs, error correction times and the right to audit.
- Put retraining and its schedule inside the fee, with performance thresholds that trigger it.
- Keep the right to pause or replace the model, with a fallback ready.
- Write down what you own at exit: data, labels, guidelines, configuration and deliverables.
Questions people ask
Board approving an AI vendor contract for credit decisions, what should we require given the regulator holds us responsible?
Require access to model documentation, inputs and versions, per-decision explanations the bank can give customers, performance monitoring the bank can see, the right to pause or replace the model, and a fallback. The contract must give the bank what the regulator will ask it for. It depends on the regulator's expectations for automated credit decisions and on whether the model decides or advises.
AI vendor contracts allow use of our data to improve their models, should the board approve?
Approve only with no training on the company's data or with explicit opt-out confirmed in writing, data retention limits and clarity on what the vendor may derive. Most vendors offer enterprise terms without training use when asked. It depends on the sensitivity of the data the vendors will see and on what the contracts say in detail.
A department head is pushing me to sign for an AI tool quickly with a vendor discount, what do I need to know before signing?
Three things before any signature: what the tool is cleared to do and whether that matches how the department will use it, what it does with patient data, and what evidence shows it works on patients like yours. A discount deadline is not a reason to skip any of them. It depends on whether the tool informs or decides, and on your privacy and clinical governance approvals.
A consultancy offers a fixed fee AI program with outcomes but no defined deliverables, what should I require before signing?
Require named deliverables per phase, a decision point after each phase where you can stop, and clarity on who does the work. A program you cannot stop at a defined point is a commitment to the whole fee. It depends on which problems you want solved and on whether the firm's role is advice, delivery or both.
AI assistant vendor excludes liability for wrong answers to our customers, what can we realistically get in the contract?
Vendors rarely accept liability for the model's answers, so get what they do accept: defined quality measures, the right to restrict topics, logs for every answer, fast correction of errors and the right to switch off. Keep the assistant off decisions about individual customers until the measures hold. It depends on which topics the assistant covers and on what your regulator expects for automated customer communication.
Vendor fraud model performance degraded and retraining is extra cost, how do we handle it and prevent it next time?
Measure the degradation so the loss is a number, then read the contract for what the vendor committed on performance and maintenance. Pay for a retraining only together with new terms that define performance thresholds and who retrains at what cost. It depends on what the contract says about model maintenance and on whether the bank can monitor performance itself.
Signing with a data labelling vendor, what quality and ownership terms should be in the contract?
Define quality as agreement with a gold set you own, with acceptance per batch and the right to reject. Keep ownership of the guidelines, the labels and any derived data, and define who the labellers are and what they may see. It depends on how specialised your labelling task is and on whether your data includes personal or customer information.
What customer isolation evidence should precede signing an AI platform contract?
Require evidence for isolation at every relevant access path. Suitability depends on enforcement location, administrator access, and behavior when filters are missing, malformed, or bypassed.
How I can help with this decision
- Ask or talk (Free)
- I give my view on which terms vendors accept in practice and which one decides this contract. I tell you what to require by data class and how to keep the liability small.
- Review (Pay if it was worth it)
- I write an independent review of the contract and the model's governance against your data, your regulator's expectations and your exposure. I recommend the terms to require before you sign, or whether to test first or pass.
- Retain (When it makes sense)
- I stay close through the first year to review the vendor's performance reports and error log against the thresholds you set.