Third-Party AI Risk: Why Vendor Evidence Is Not Enough
[custom_breadcrumb]
Home > Blog > Your Vendor’s Model Card Will Not Defend Your Bank: Closing the Third-Party AI Evidence Gap

Banks increasingly consume AI through vendor APIs, SaaS platforms, cloud services, and embedded features. Proprietary architecture can limit visibility, but it does not transfer accountability. SR 26-2 states that model risk principles remain applicable to vendor products and calls for institution-specific understanding, validation, monitoring, and outcome analysis. A vendor model card is an input to governance, not proof that the bank’s deployment is controlled.

Is a bank responsible for decisions made using third-party AI?

Yes. Outsourcing a model or AI service does not outsource the bank’s responsibility for its use and outcomes. The institution needs evidence that the product is fit for its purpose, data, customers, risk appetite, controls, and regulatory obligations, including ongoing monitoring after deployment.

AI is entering the bank through procurement

The most visible AI programs may sit with data science or transformation teams. The fastest-growing exposure can arrive elsewhere: a new copilot inside a SaaS platform, an upgraded fraud-scoring API, generative search added to a knowledge tool, or agentic features enabled by a cloud provider.

These capabilities can reach users before the bank has classified them, established ownership, or determined what data they access. Procurement may assess commercial terms and security, while model risk reviews a narrower set of quantitative models. The gap between those processes is where third-party AI becomes shadow infrastructure.

Proprietary does not mean ungovernable

SR 26-2 recognizes that banks may not receive the code, data, or methodology available for internally developed models. It nevertheless states that model risk principles remain applicable. Sound practice includes understanding the vendor model, validating it, and conducting ongoing monitoring and outcomes analysis.
The bank may not be able to inspect every parameter, but it can still test behavior. It can evaluate performance on its customer and transaction populations, challenge limitations, measure outcomes, define acceptable use, control access, monitor drift, and establish fallback.

Opacity changes the evidence method. It does not eliminate the evidence obligation.

Why the model card is not the evidence pack

A vendor model card can explain intended use, training approach, known limitations, and benchmark performance. Those facts are useful for due diligence. They do not show whether the bank’s specific configuration works for its products, customers, geographies, data, policies, and operating environment.

A benchmark says little about a credit explanation generated from the bank’s policy corpus or a fraud model exposed to its transaction mix. Vendor-reported accuracy does not reveal how overrides, customer complaints, or false positives will behave in production.

The evidence pack must connect vendor information to institution-specific tests and outcomes.

Comparison between vendor AI model documentation and the bank-specific evidence required for third-party AI governance and accountability

Contract for evidence and control

AI contracting should address more than uptime, confidentiality, and general security. Banks need notice of material model or configuration changes, access to relevant performance information, incident notification, audit cooperation, data-use restrictions, retention and deletion terms, subprocessor visibility, service continuity, and exit support.

For higher-risk uses, the agreement should clarify how the bank can test the service, preserve decision evidence, restrict tools or data, obtain explanations, and respond when performance falls outside thresholds.

A contractual right that cannot be exercised technically is weak. Architecture and procurement must therefore design the control together.

Monitor the use, not only the provider

A third-party service can remain unchanged while the bank’s risk increases. The user population may expand. Employees may apply it to a new purpose. More sensitive data may enter the workflow. Automation may replace review. Transaction volume may grow.

Monitoring should cover performance and outcomes in the bank’s environment, usage outside approved purpose, exception and override patterns, complaints, bias indicators where relevant, changes in scale, and dependencies on shared data or providers.

Aggregate concentration also matters. Several business lines may rely on the same model, cloud service, data source, or subprocessor without recognizing the common failure point.

A minimum evidence standard for regional banks

A practical third-party AI file should state the purpose, owner, risk tier, affected business process, data used, and decision authority. It should contain vendor due diligence, institution-specific evaluation, security and privacy assessment, contractual control mapping, production monitoring, incident procedures, and an exit or fallback plan.

For generative and agentic services, add prompt and configuration governance, retrieval-source controls, tool permissions, action limits, logging, and red-team scenarios.

Regional banks can standardize this evidence by tier so lower-risk tools do not face the same burden as AI influencing credit, payments, fraud, compliance, or customer rights.

What this means for banking leaders

  • Treat embedded AI features and SaaS upgrades as part of the AI inventory, not merely software changes.
  • Test vendor AI on the bank’s own intended use, population, data, controls, and outcomes.
  • Contract for change notice, incident evidence, testing rights, continuity, and exit support.
  • Monitor changes in the bank’s use and aggregate vendor concentration, not only provider performance.

Conclusion

Third-party AI can accelerate adoption, but it cannot transfer accountability. The bank must be able to show why the service was appropriate, how it was tested, what it was allowed to do, how it behaved in production, and what happens when it fails. A vendor’s documentation begins that inquiry. It cannot finish it.

Explore the full CIO and CRO mandate

The fifth paper in Maveric Systems’ CIO Mandate Series sets out the Enterprise AI Accountability Architecture, regulatory convergence, governance maturity model, and 24-month implementation roadmap.

FAQ

What is third-party AI risk?

It is the operational, model, compliance, security, privacy, concentration, and customer risk created when a bank relies on externally supplied AI or AI-enabled services.

Is a vendor model card enough for validation?

No. It should be supplemented with institution-specific testing, control mapping, monitoring, outcome analysis, and governance evidence.

What should an AI vendor contract include?

Key terms include change notification, incident reporting, data restrictions, testing and audit cooperation, performance evidence, subprocessors, continuity, and exit support.

How should banks monitor vendor AI?

Monitor performance, outcomes, drift, approved use, scale, overrides, complaints, incidents, vendor changes, and common dependencies.

How can a regional bank make third-party oversight proportionate?

Use risk tiers and standardized evidence packages, applying deeper validation and contractual requirements to consequential uses.

Article by

Maveric Systems