Adversarial AI Frameworks: Taxonomy, Threat Landscape, and Control Framework
Read this assessment of risk levels relating to the integration of generative AI into business operations
Executive Summary
The Adversarial AI Frameworks document aims to provide an approach to tracking and assessing AI-enabled threats in the financial services sector, specifically focusing on recent developments in generative AI (GenAI). To that end, this framework assesses risk levels relating to the integration of GenAI into business operations, adopting a threat-led approach for the implementation of controls and mitigations. This review is applicable to many GenAI modalities, but utilizes large language models (LLMs) as a focus point due to their broader applications and common baseline for multi-modal models.
GenAI is an emerging technology; the evolving nature of the technology limits the ability to provide examples and resources around secure adoption and implementation. Many existing frameworks are unable to adequately convey the breadth of threats posed.
This document derives its conclusions, framework, and taxonomy from a variety of sources, and suggests control implementations agnostic of vendor.
The threats discussed in this document include those related to AI models, but extend to the data such models are trained and tested on, the third-party components, plug-ins and libraries utilized in their development, as well as the platform such models can be hosted on. The focus is the impact to the AI application, and thus some instances will include discussion of threats with existing mitigations, considerations, and best practices from many organizations. Such instances may bear repeating to sufficiently discuss the threat to or from an AI application, but are often best mitigated by existing cyber hygiene and best practices.
This report does not make recommendations on risk tolerance nor on the level of residual risk, as such assessments are use-case specific and individual to each organization.
This document contains a taxonomy of attacks and defines key terminology used in the discussion of AI threats. The taxonomy and list of technical explanations contextualizes these threats to provide an easily digestible and applicable hierarchy, as well as a financial services-sector relevant document that can be of use to non-financial sector entities. This taxonomy incorporates existing literature and public taxonomies to remain consistent with public and adopted alternatives – including the NIST Adversarial Machine