Vendor control library32 practices
Build-buy-partner decision
Compare internal development, commercial purchase, managed service, and hybrid options against capability, control, time, cost, and exit needs.
Intended use and decision authority
Define the business objective, users, affected parties, decisions, permitted actions, exclusions, and human authority before approaching vendors.
Outcome and acceptance requirements
Convert desired value, consumer outcomes, risk tolerances, service needs, and evidence expectations into measurable requirements.
Market scan and shortlist
Compare providers using consistent minimum qualifications, evidence readiness, insurance fit, delivery capacity, and financial resilience.
Accountable owner and RACI
Assign one insurer owner and define business, procurement, legal, compliance, security, privacy, model risk, actuarial, audit, and vendor responsibilities.
Vendor tiering and risk classification
Tier the relationship using consumer impact, decision materiality, autonomy, data sensitivity, scale, substitutability, and regulatory exposure.
Financial stability and reputation
Assess viability, ownership, litigation, regulatory history, insurance coverage, key-person reliance, and ability to support the service.
Subcontractor and fourth-party control
Identify material subprocessors, data/model providers, hosting, open-source dependencies, locations, and flow-down controls.
Data source rights and provenance
Confirm lawful source, permitted purpose, provenance, consent or authorization, quality, lineage, and restrictions for all material data.
Training retention and deletion
Define whether insurer or consumer data may train models, how long data and prompts persist, deletion methods, backups, and proof.
Privacy minimization and location
Limit data to necessity; assess personal and sensitive data, tokenization, residency, cross-border transfers, access, and privacy impact.
IP output and indemnity rights
Allocate ownership and use rights for inputs, prompts, configurations, fine-tunes, outputs, inventions, feedback, and infringement claims.
Architecture method and limitations
Obtain understandable documentation of components, models, versions, training or configuration, dependencies, assumptions, and limitations.
Performance fairness and robustness
Require representative evaluation of accuracy, reliability, group outcomes, drift, edge cases, uncertainty, and failure modes.
Generative AI and RAG evidence
Test factuality, grounding, citations, retrieval quality, harmful output, privacy leakage, and behavior under unsupported requests.
Agent authority and tool controls
Test goals, planning, permissions, tool calls, transaction limits, approvals, memory, delegation, termination, and recovery.
Security controls and assurance
Assess identity, access, encryption, secrets, logging, secure development, testing, assurance reports, vulnerabilities, and remediation.
AI-specific threat resistance
Evaluate prompt injection, data exfiltration, model theft, poisoning, unsafe tools, insecure output handling, and excessive agency.
Availability continuity and fallback
Set availability, recovery, capacity, backup, business continuity, degraded operation, manual fallback, and restoration requirements.
Incident notification and cooperation
Require rapid notice, preservation, containment, investigation, root cause, affected-case support, regulator cooperation, and corrective action.
Service and outcome commitments
Define service levels, quality measures, response times, support, remedies, reporting, credits, and chronic-failure treatment.
Audit evidence and regulator cooperation
Secure timely access to records, testing, logs, personnel, subcontractor evidence, audits, examinations, and regulator requests.
Change notice approval and versioning
Control model, data, prompt, retrieval, tool, hosting, subprocessor, policy, and material functionality changes.
Liability insurance and remedies
Align liability, caps, carve-outs, indemnities, insurance, warranties, breach remedies, suspension, and termination with exposure.
Integration data and security readiness
Verify interfaces, identity, data mappings, controls, environments, logging, support, and operational ownership before testing.
Pilot and acceptance testing
Use representative cases and defined thresholds; record failures, remediation, retests, exceptions, and final acceptance.
User training and human oversight
Prepare users to interpret outputs, recognize limits, review decisions, override safely, escalate, and report incidents.
Production approval and rollback
Require risk-based sign-off, production controls, monitoring, support, freeze windows, rollback triggers, and go/no-go authority.
Performance and risk scorecard
Review service, quality, fairness, security, incidents, support, changes, evidence, remediation, and business outcomes.
Change incident and revalidation oversight
Route material changes and incidents through impact assessment, testing, approval, affected-case review, and monitoring updates.
Concentration and substitutability
Assess aggregate dependence on providers, cloud/model/data chokepoints, correlated failure, lock-in, alternatives, and manual capacity.
Exit transition and records
Maintain triggers, transition assistance, portability, continuity, records, data return/deletion, access removal, and regulator-ready evidence.