# Getting Started Welcome to the official documentation of AI Vulnerability Database (AVID)! AVID is an open-source, extensible database of failure modes in general-purpose AI (GPAI) systems, including LLMs, API-only AI systems, developer tooling, and end-to-end applications and agents. AVID aims to * prioritize high-fidelity vulnerability and report records with evidence and reproducible evaluation details * support practitioners evaluating GPAI systems across model, tool, and application layers * map records to a library of taxonomies, with AVID taxonomy as one reference framework * provide developer tooling to structure and ingest reports at scale This site contains information to get you started with different components of AVID. * [Database](/database/introduction): the primary store of structured vulnerability and report records. * [Taxonomy Library](/taxonomy/introduction): classification frameworks used to organize and query database records. * [Developer SDK](/developer-tools/python-sdk): the official Python toolkit for working with AVID. # Introduction The database is AVID's primary component. It houses full-fidelity information (metadata, harm metrics, measurements, benchmarks, and mitigation techniques when available) for concrete failure evidence in general-purpose AI (GPAI) systems. The aim is transparent and reproducible evaluation records that can be mapped to one or more taxonomy frameworks. It * is expandable to account for novel and hitherto unknown vulnerabilities * enables developers and evaluators to freely share structured evaluation records for community benefit * is composed of submissions in a schematized format, then vetted and curated. We are building the database to be both an extension of, and a bridge between, classic security-related vulnerabilities in the [National Vulnerability Database (NVD)](https://nvd.nist.gov/vuln), adversarial attack cases in [MITRE ATLAS](https://atlas.mitre.org/), and incidents in the [AI Incident Database (AIID)](https://incidentdatabase.ai/). By connecting these sources and including unintentional failure states across GPAI workflows, AVID supports a more operational view of AI risk. Developers can assess risks in models, tools, and applications they plan to build on, and make better choices with less risk of harm. Communities have a way to contest harmful systems and contribute evidence. Regulators, policy makers, and adjudicating bodies benefit from a clearer picture of failure patterns and high-risk entities. {% hint style="info" %} Some older AVID records (before 2025) were created under a broader AI/ML scope and are now considered legacy relative to the current GPAI-focused scope.\ \ Because there is no settled definition of an AI vulnerability yet, AVID currently operates with a working definition. In the current release cycle, we are prioritizing report-level evidence and have not published new vulnerability records. {% endhint %} # Framework AVID stores instantiations of AI risks---categorized using the AVID taxonomy---using two base data classes: [Report](/database/framework/auxiliary-classes) and [Vulnerability](/database/framework/base-classes). A *report* is one example of a particular vulnerability occurring, supported by qualitative or quantitative evaluation. A *vulnerability* (vuln) is a high-level evidence of an AI failure mode, similar to the NIST CVEs in the context of software vulnerabilities. As an example, the vuln [AVID-2022-V001](https://github.com/avidml/avid-db/blob/main/vulnerabilities/2022/AVID-2022-V001.json) is about gender bias in the large language model `bert-base-uncased`. This bias is measured through multiple reports, [AVID-2022-R0001](https://github.com/avidml/avid-db/blob/main/reports/2022/AVID-2022-R0001.json) and [AVID-2022-R0002](https://github.com/avidml/avid-db/blob/main/reports/2022/AVID-2022-R0002.json), which measure gender bias in two separate contexts, using different metrics and datasets, and record salient information and references on those measurements. The above formulation is similar to how incidents and incident reports are structured in the [AI Incident Database](https://incidentdatabase.ai/). See Figure D.1 for a schematic representation of this structure.

Figure D.1. Schematic of the structure of the AVID taxonomy, vulns, and reports.

To account for diverse levels of details that different groups of AI risk examples can entail, we designate a class for each vulnerability and report. Each such vuln/report class extends the respective base class to a slightly different structure that enables storage of information at different granularities as required. For example, we currently support four vuln/report classes: evaluations of large language models (`LLM Evaluation`), incidents from AIID (`AIID Incident`), vulnerabilities from CVE (`CVE Entry`), and externally sourced reports (`Third-party Report`). These classes share the same core vuln/report field structure, with slight differences in values filled under references and tags. # Base Classes ### Report Reports are occurrences of a GPAI failure mode. Based on the references provided in a specific report, reports can potentially more granular and reproducible than vulnerabilities. We classify reports in four types, in increasing degree of quantitative evidence: 1. **Issue**: qualitative evaluation based on a single sample or handful of samples, 2. **Advisory**: qualitative evaluation based on multiple Incidents, 3. **Measurement**: quantitative evaluation with associated data and metric, 4. **Detection**: A Measurement deemed critical by a threshold or statistical test. These types are reminiscent of the [three levels of AI Auditing](https://informationashvins.wordpress.com/2022/11/29/three-levels-of-ai-auditing/), and accommodate diverse AI evaluation scenarios from the user perspective. ### Vulnerability Vulnerabilities are linked to the taxonomy through multiple tags, denoting the AI risk domains (Security, Ethics, Performance) this vulnerability pertains to, (sub)categories under that domain, as well as AI lifecycle stages. A vulnerability in AVID can pertain to one or more of the three levels: **Dataset**, **Model**, or **System**. {% hint style="info" %} The field-level schema for vulnerabilities is stable, but the broader definition of an "AI vulnerability" remains an active area of standardization. AVID therefore uses a working definition and is currently prioritizing new report publication while vulnerability publication is temporarily paused in the latest release cycle. {% endhint %} # Auxiliary Classes Details of a vulnerability or report are structured in a few auxiliary data classes that are used in one or both base classes. Below we describe the main auxiliary classes and their usage. For an exhaustive list, refer to our [API documentation](https://avidml.org/avidtools) or the respective [JSON schema](https://github.com/avidml/avid-db/tree/main/schema).
description High-level description of a vulnerability or report. This is relevant for both report and vulnerability.
problemtype Information on the problem a report or vulnerability is concerned with. This has three components: * `classof`: class of a report or vuln, can take values `AIID Incident`, `ATLAS Case Study`, `CVE Entry`, `Third-party Report`, `LLM Evaluation`, or `Undefined`. * `type`: whether a report is an `Issue`, `Advisory`, `Measurement`, or `Detection`. * `description`: details of the problem that has occurred.
affects Information about the AI artifact(s) that have been affected. * `developer`: information about the developer of the affected artifact(s) * `deployer`: information about the deployer of the affected artifact(s) * `artifacts`: list of name and type (dataset, model, or system) of artifacts affected.
metrics This class is only relevant in reports. It is strucured as a list of objects that store quantitative evaluation results for a specific metric. Each such object has the following components. * The `name` of the metric being measured * The `detection_method` if any to determine if the metric value is risky enough, and its type (`static_threshold` or `statistical_test`)
references This class contains a list of references about a vulnerability or report, that contain relevant information about the problem. A reference can be the link to an article, code in an online repository, an uploaded screenshot, or freeform text giving additional details.
impact This class stores impact information, such as different taxonomy mappings, harm and severity scores. Currently it contains a single field `avid`, which encodes the AVID taxonomy mappings of the vulnerability or report.
credit This class stores information about person(s) or entity(s) that can be credited with finding out a report or vulnerability.
# Backend The first version of this component is under active development. We're building a secure and transparent system leveraging well-known infrastructure and methods. Our editorial process for reports, including contestation, will occur directly within GitHub so that every decision is made within public view. We're building our database as a serverless application using core AWS services to provide scale and security while trying to minimize our carbon footprint. As seen from the system diagram, our backend code will be open-sourced through GitHub. The backend itself will be run through AWS with direct usage of GitHub authentication, auditing, and an API that will allow people to integrate the data into their systems.
We are building our backend to enable practitioners adapt our [datamodel](/developer-tools/python-sdk/datamodels) to build their own database internal to their organization. By doing this, we encourage a network of federated vulnerability databases where AVID acts as the bridge both to the public and between private entities. Responsible public disclosure of vulnerabiilties can be done through running submissions into the public (AVID) instance of this database through AVID's editorial process. Simultaneously, the technical tools and standards we're building will enable practitioners to improve their own internal AI development and risk management processes. # Editorial Interface The first version of this component is under active development. Currently, our editorial process starts with either a submission to our [intake form](https://docs.google.com/forms/d/e/1FAIpQLSe-jxZrx257Xce7JTa_QeaQmmQEQJfLlrBKjSzgHtNs5T-oKg/viewform?usp=dialog), or by someone creating an issue in [GitHub](https://github.com/avidml/avid-db). Then we have a manual four step process: 1. An Editor maps inputs to a `Report` datamodel and, then publishes it as a JSON file [for review](https://github.com/avidml/avid-db/tree/main/reports/review), 2. The Editor checks and edits report as needed, assigns [taxonomy categories](https://github.com/avidml/avid-db/tree/grants/schema/taxonomy_misp), then moves it to the [database](https://github.com/avidml/avid-db) as `reports/20XX/AVID-20XX-RXXXX.json`, 3. The Editor may convert report to a new vuln or merge with an existing vuln, saved as `vulnerabilities/20XX/AVID-20XX-VXXX.json` (*currently paused for new releases while AVID refines a working AI vulnerability definition*), 4. Webmaster renders new reports and vulns to markdown files in the [website source](https://github.com/avidml/website/tree/main/exampleSite/content/database). We plan to make this less manual over the course of the next few months by building an editorial UI and creating necessary integrations to enable developers push reports to AVID easily, and with more complete information upfront. Decision logs for each report will be logged in GitHub for openness and transparency. # Python SDK The `avidtools` package is the official Python toolkit for working with AVID resources. It empowers engineers and developers build pipelines to export outcomes of tests in their ML pipelines as AVID reports, build an in-house vulnerability database, integrate existing sources of vulnerabilities into AVID-style reports, and much more! ### Features * šŸ“Ž **Data models**: definition of AVID report/vulnerability schema as [pydantic](https://docs.pydantic.dev/) models * šŸ„ **Connectors**: ingestion from relevant data sources, e.g. MITRE ATLAS case studies and NIST CVEs In this section, we'll provide overview and usage examples for `avidtools`. For more details, check out the [API Reference](https://avidml.org/avidtools). ### Installation To install `avidtools` from PyPI, run: ``` pip install avidtools ``` To install the latest development version from GitHub, use: ```bash python -m pip install -U git+https://github.com/avidml/avidtools.git@main ``` # Datamodels In `avidtools`, AVID schema objects are represented as Pydantic datamodels. The top-level models are `Report` and `Vulnerability`. Let's initialize an empty report: ```python from avidtools.datamodels.report import Report report = Report() report ``` This gives the following output: ```bash Report(data_type='AVID', data_version=None, metadata=None, affects=None, problemtype=None, metrics=None, references=None, description=None, impact=None, credit=None, reported_date=None) ``` Let's now populate a couple of fields, replicating the data for the AVID report [AVID-2022-R0003](https://github.com/avidml/avid-db/blob/main/reports/2022/AVID-2022-R0003.json). ```python from avidtools.datamodels.components import Affects, Artifact, LangValue, Problemtype from avidtools.datamodels.enums import ArtifactTypeEnum, ClassEnum, TypeEnum report.affects = Affects( developer = [], deployer = ['Hugging Face'], artifacts = [ Artifact(type = ArtifactTypeEnum.model, name = 'bert-base-uncased'), Artifact(type = ArtifactTypeEnum.dataset, name = 'sasha/wino_bias_cloze1'), Artifact(type = ArtifactTypeEnum.dataset, name = 'sasha/wino_bias_cloze2') ] ) report.problemtype = Problemtype( classof = ClassEnum.llm, type = TypeEnum.detection, description = LangValue( lang = 'eng', value = 'Profession bias reinforcing gender stereotypes found in bert-base-uncased, as measured on the Winobias dataset' ) ) ``` ### Auxiliary Classes In the above example, apart from the base `Report` class we also use enumerators (`ArtifactTypeEnum`, `ClassEnum`, `TypeEnum`) and components (`Affects`, `Problemtype`, `LangValue`, `Artifact`). An **enumerator**, or `Enum`, is used to standardize allowed values. For example, `TypeEnum` is defined as: ```python class TypeEnum(str, Enum): """All report/vulnerability types.""" issue = 'Issue' advisory = 'Advisory' measurement = 'Measurement' detection = 'Detection' ``` Enums live under `avidtools.datamodels.enums`. The currently supported report/vulnerability classes are: * `AIID Incident` * `ATLAS Case Study` * `CVE Entry` * `LLM Evaluation` * `Third-party Report` * `Undefined` The rest are **components**, which correspond to Pydantic versions of auxiliary schema fields. | Auxiliary data class | Pydantic data type | | -------------------- | ------------------ | | `description` | `LangValue` | | `problemtype` | `Problemtype` | | `affects` | `Affects` | | `metrics` | `List[Metric]` | | `references` | `List[Reference]` | | `impact` | `Impact` | | `credit` | `List[LangValue]` | These live under `avidtools.datamodels.components`. The `Impact` component can include AVID and ATLAS taxonomy mappings, as well as CVSS/CWE/0DIN structures: * `AvidTaxonomy` * `AtlasTaxonomy` * `CVSSScores` * `CWETaxonomy` * `OdinTaxonomy` ### Write and Read Realistically, fields in a report are programmatically populated at the end of evaluation pipelines, then handed off to downstream processes. You can save and load a report as JSON as follows. ```python import json # Saves a report report.save('test.json') # Loads a report with open('test.json', 'r') as f: datax = json.load(f) report2 = Report(**datax) ``` For an example of how the `Report` datamodel can be used in tandem with AI evaluation pipelines, check out [this space](https://huggingface.co/spaces/avid-ml/bias-detection) on Hugging Face. ### Ingest The `Vulnerability` datamodel works similarly to `Report`. It also supports ingesting information from an existing `Report`. This is useful to: 1. form a new vulnerability from a report, or 2. incorporate report-derived fields into an existing vulnerability. Here's a small example. ```python from avidtools.datamodels.vulnerability import Vulnerability vuln = Vulnerability() vuln.ingest(report) vuln # Vulnerability(data_type='AVID', data_version=None, metadata=None, affects=None, problemtype=None, references=None, description=None, reports=None, impact=None, credit=None, published_date=datetime.date(2023, 6, 14), last_modified_date=datetime.date(2023, 6, 14)) ``` After `ingest`, the vulnerability fields are updated from the input report and `published_date` / `last_modified_date` are set to today. # Integrations The first version of this component is under active development. The `avidtools` SDK and AVID data schema is designed to be interoperable with AI evaluation and monitoring frameworks. This section lists out such integrations that are currently supported. ### List of Integrations * [garak](/developer-tools/python-sdk/integrations/garak) * [ModsysML](https://github.com/modsysML/modsysML) by [Apollo](https://www.apolloapi.io/) * :turtle: [Giskard](https://github.com/Giskard-AI/giskard) # garak The open-source Large Language Model (LLM) vulnerability scanner [garak](https://github.com/leondz/garak/) provides modular and standardized detections (tests), probes (groups of tests looking for a vulnerability), and harnesses (end-to-end pipelines) to test LLMs for different vulnerabilities and downstream harms. AVID resources are integrated with `garak` in two ways. ### Taxonomy The `tags` attribute in the Python class defining a `garak` probe stores AVID taxonomy classifications in MISP format. ```python # Source: https://github.com/leondz/garak/blob/main/garak/probes/base.py ... class Probe: """Base class for objects that define and execute LLM evaluations""" name = None description = "Empty probe" uri = "" bcp47 = None # language this is for, in bcp47 format; * for all langs recommended_detector = ["always.Fail"] # send a signal if this isn't overridden active = True tags = [] # MISP-format taxonomy categories ... ``` As an example, the [`knownbadsignatures`](https://github.com/leondz/garak/blob/main/garak/probes/knownbadsignatures.py) group of probes checks whether a LLM can be made to generate signatures of known malwares, and the `knownbadsignatures.EICAR` probe checks for leakage of the signature of the [EICAR test file](https://en.wikipedia.org/wiki/EICAR_test_file). This is given two taxonomy tags. ```python # Source: https://github.com/leondz/garak/blob/main/garak/probes/knownbadsignatures.py ... class EICAR(Probe): name = "eicar" description = "Does the model check its output for viruses? Try to send the EICAR code through" bcp47 = "*" uri = "https://en.wikipedia.org/wiki/EICAR_test_file" recommended_detector = [ "knownbadsignatures.EICAR", ] tags = ["avid-effect:security:S0301", "avid-effect:security:S0403"] ... ``` In the AVID taxonomy, these tags correspond to *Information Leak* and *Adversarial Example*, respectively. In a similar manner, `garak` detectors also has the `tags` attribute. In line with the flexible MISP format, *any* taxonomy classification in the MISP format can be stored as a tag. For example, the [`lmrc.Bullying`](https://github.com/leondz/garak/blob/cd6d2ec822e63b238a7effbc3181d31b275a3f16/garak/probes/lmrc.py#L40) probe has tags `risk-cards:lmrc:bullying` and `avid-effect:ethics:E0301`, corresponding to the [Risk Card](#user-content-fn-1)[^1] category *Bullying*, and the AVID SEP category *E0301: Toxicity*. ### Reporting Scans by `garak` generate log files in JSONL format that store model metadata, prompt information, and evaluation results. This information can be structured into one or more AVID reports. Check out the following example using a sample run. 
wget https://gist.githubusercontent.com/shubhobm/9fa52d71c8bb36bfb888eee2ba3d18f2/raw/ef1808e6d3b26002d9b046e6c120d438adf49008/gpt35-0906.report.jsonl
python3 -m garak -r gpt35-0906.report.jsonl
## output:
# garak LLM security probe v0.9.0.6 ( https://github.com/leondz/garak ) at 2023-07-23T15:30:37.699120
# šŸ“œ Converting garak reports gpt35-0906.report.jsonl
# šŸ“œ AVID reports generated at gpt35-0906.avid.jsonl
[^1]: [Derczynski et al](https://arxiv.org/abs/2303.18190). Assessing Language Model Deployment with Risk Cards, arXiV, 2023. # ModsysML (Apollo) Maintained by [Apollo](https://www.apolloapi.io/), [ModsysML](https://modsys.vercel.app/) is an open-source model management toolkit for continuous model improvement. It helps generative AI developers evaluate and compare LLM outputs, test quality, as well as catch regressions and automate their evaluations. The [`modsys.connectors.avid`](https://github.com/modsysML/modsysML/tree/main/modsys/connectors/avid) module generates AVID reports from LLM evaluation runs performed by ModsysML. To do so, you need information of the model to be evaluated, a text-based summary of the outcomes, details of the evaluation outcomes dataset (description, link to its location in your local/cloud storage), and finally a path to save the report. Here is a minimal example of the above, for an evaluation done on `gpt-3.5-turbo` by OpenAI. ```python # Source: https://github.com/modsysML/modsysML/blob/main/modsys/connectors/avid/cloud.py from modsys.connectors.avid.cloud import AVIDProvider AVIDProvider().create_report( provider_name = 'openai', provider_model = 'gpt-3.5-turbo', dataset_name = 'eval_data', dataset_link - 's3://your/bucket/eval_data.csv', summary = '', path_to_save_report = '/path/to/report/eval_report.json' ) ``` # Giskard [Giskard](https://www.giskard.ai/) provides an open-source Machine Learning (ML) testing framework, covering traditional ML as well as LLM use cases. Giskard Scan is a powerful tool to detect vulnerabilities in LLMs. Its integration with AVID taxonomy provides improved **standardized reporting of vulnerabilities,** and the ability to share your vulnerability reports with the community. ### Taxonomy By default, all Giskard scan reports indicate the **AVID taxonomy categories** that are relevant to the detected vulnerabilities. You can find this information in the detail view of each issue in the scan widget.
### Exporting AVID reports You can export your Giskard scan report as an AVID report. First, make sure you have `avidtools` installed in your environment. ```bash pip install avidtools ``` Then, once you have run the Giskard scan, you can export the report as an AVID report. ```python import giskard as gsk scan_report = gsk.scan(my_model, my_dataset) # Export the report as a list of AVID reports (one per each vulnerability) avid_reports = scan_report.to_avid() ``` You can also export these reports directly in a JSONL file (one AVID report per line): ```python # Write the AVID reports to a JSONL file scan_report.to_avid("avid_report.jsonl") ``` For more details on how to use Giskard in combination with AVID, check out this [tutorial](https://docs.giskard.ai/en/latest/integrations/avid/avid-integration-llm.html). # Connectors Connectors import or transform data from external sources into AVID-compatible structures. Current first-party connectors in `avidtools` include: * `atlas`: import/convert MITRE ATLAS case studies * `cve`: import/convert NVD CVEs * `inspect`: convert Inspect AI logs into AVID reports * `url`: scrape a URL and generate an AVID report via LLM-assisted extraction * `garak`: normalize AVID report JSON/JSONL generated from garak-style outputs ### List of Connectors
ATLAS Case Study Adversarial ML case studies hosted on MITRE ATLAS double up as AVID reports or vulnerabilities. Their data are stored as yaml files in the ATLAS GitHub. Given the case study ID, we can import that data as a `Report` object. For example, the following code imports the Tay Poisoning case study. ```python from avidtools.connectors import atlas CS_ID = 'AML.CS0009' cs = atlas.import_case_study(CS_ID) # returns a dict in the original schema report = atlas.convert_case_study(cs) # returns a Report object ```
NIST CVE Some AI vulnerabilities related to application security or cybersecurity detection models may be cross-posted as [CVEs](https://nvd.nist.gov/vuln). This data can be queried from the NIST NVD API. Given a CVE ID, we can import that data into a `Vulnerability` object. For example, the following code imports the Proofpoint Evasion vulnerability, the first ever ML CVE reported. ```python from avidtools.connectors import cve CVE_ID = 'CVE-2019-20634' cv = cve.import_cve(CVE_ID) # returns a custom dict vuln = cve.convert_cve(cv) # returns a Vulnerability object ```
Inspect AI The open-source LLM evaluation toolkit [Inspect AI](https://inspect.aisi.org.uk/) enables standardized assessment of LLM behavior across a wide range of capabilities. The `avidtools.connectors.inspect` connector converts Inspect evaluation logs (`.eval` / `.json`) into one or more AVID `Report` objects. To do so, first run your evaluation with Inspect AI and save logs to a directory. For example, the following code runs the [BOLD](https://ukgovernmentbeis.github.io/inspect_evals/evals/bias/bold/) benchmark we contributed to Inspect Evals on gpt-4o-mini. ```sh inspect eval inspect_evals/bold --model openai/gpt-4o-mini --log-dir ./experiment-log ``` Following this, you can run to convert the Inspect logs to a list of AVID `Report` objects. ```python from avidtools.connectors.inspect import convert_eval_log reports = convert_eval_log("experiment-log/bold_logs.eval") ``` You can optionally pass `normalize=True` to apply post-processing that enriches descriptions and formatting: ```python reports_normalized = convert_eval_log( "experiment-log/bold_logs.eval", normalize=True, ) ``` When converting many logs, use the batch helper and write results to JSONL: ```python from pathlib import Path from avidtools.connectors.inspect import convert_eval_logs, write_reports_jsonl log_paths = Path("experiment-log").glob("*.eval") reports = convert_eval_logs(log_paths, normalize=False) written = write_reports_jsonl(reports, Path("out/inspect-reports.jsonl")) print(f"Wrote {written} reports") ``` Optionally, you can store the original Inspect logs in an S3 bucket and reference that in the report: ```python reports = convert_eval_log( "experiment-log/bold_logs.eval", s3_bucket="my-eval-logs", s3_key_prefix="inspect", s3_region="us-east-1", ) ``` > **S3 upload dependency:** S3 upload requires `boto3`, and require configuring AWS secrets using `aws configure`.
garak Since the garak CLI can produce AVID reports directly, the garak connector only does optional normalization of the AVID reports that garak produces. > **API key optional:** garak normalization works without keys, but if `OPENAI_API_KEY` is available it can use OpenAI-assisted probe-summary enrichment; otherwise it falls back to deterministic descriptions. ```python from pathlib import Path from avidtools.connectors import garak count = garak.normalize_file(Path("gpt35-0906.avid.jsonl")) print(f"Normalized {count} reports") ```
URL Generate an AVID report directly from a URL by scraping page content and extracting report fields. > **API key required:** `URLConnector` uses an OpenAI-backed agent and requires `OPENAI_API_KEY` (either passed as `api_key=...` or set in environment). ```python from avidtools.connectors.url import URLConnector connector = URLConnector(model="gpt-4o-mini") report = connector.create_report_from_url( "https://example.com/ai-vulnerability-article" ) ```
# Introduction This section describes AVID's taxonomy framework as part of AVID's broader taxonomy library. AVID's primary focus is the database of GPAI failure evidence, while taxonomies are used to classify and query those records. The AVID taxonomy is intended to serve as a common foundation for AI engineering, product, and policy teams to manage potential risks at different stages of a GPAI workflow. In spirit, this taxonomy is analogous to [MITRE ATT\&CK](https://attack.mitre.org) for cybersecurity vulnerabilities, and [MITRE ATLAS](https://atlas.mitre.org/) for adversarial attacks on ML systems. At a high level, the current AVID taxonomy consists of two views, intended to facilitate the work of two different user personas. * [Effect view](/taxonomy/effect-sep-view): for the auditor persona aiming to assess risks for a GPAI system and its components. * [Lifecycle view](/taxonomy/lifecycle-view): for the developer persona aiming to build an end-to-end GPAI system while being cognizant of potential risks. Based on case-specific needs, people involved with building a GPAI system may need to operate as either of the above personas. > For machine-readability, taxonomies are shared using the standardized [MISP](https://www.circl.lu/doc/misp-taxonomies/) format. This enables support for additional taxonomies in the AVID taxonomy library. See [Schema](/taxonomy/schema) to learn more. # Effect (SEP) View The domains, categories, and subcategories in this view provide a ā€˜risk surfaceā€™ for the AI artifact being evaluated, may it be a dataset, model, or the whole system. This view contains three top-level domains: Each domain is divided into a number of categories and subcategories, each of which is assigned a unique identifier. Figure T.1 presents a holistic view of this taxonomy matrix. See the individual pages for [Security](/taxonomy/effect-sep-view/security), [Ethics](/taxonomy/effect-sep-view/ethics), [Performance](/taxonomy/effect-sep-view/performance) for more details.

Figure T.1. The SEP Taxonomy Matrix.

# Security This domain is intended to codify the landscape of threats to a ML system.
IDSub-IDNameDescription
S0100Software VulnerabilityVulnerability in system around modelā€”a traditional vulnerability
S0200Supply Chain CompromiseCompromising development components of a ML model, e.g. data, model, hardware, and software stack.
S0201Model CompromiseInfected model file
S0202Software compromiseUpstream Dependency Compromise
S0300Over-permissive APIUnintended information leakage through API
S0301Information LeakCloud Model API leaks more information than it needs to
S0302Excessive QueriesCloud Model API isnā€™t sufficiently rate limited
S0400Model BypassIntentionally try to make a model perform poorly
S0401Bad FeaturesThe model uses features that are easily gamed by the attacker
S0402Insufficient Training DataThe bypass is not represented in the training data
S0403Adversarial ExampleInput data points intentionally supplied to draw mispredictions. Potential Cause: Over permissive API
S0500ExfiltrationDirectly or indirectly exfiltrate ML artifacts
S0501Model inversionReconstruct training data through strategic queries
S0502Model theftExtract model functionality through strategic queries
S0600Data poisoningUsage of poisoned data in the ML pipeline
S0601Ingest PoisoningAttackers inject poisoned data into the ingest pipeline
> **NOTE**\ > A number of categories map directly to techniques codified in MITRE ATLAS. In future, we intend to cover the full landscape of adversarial ML attacks under the Security domain. # Ethics This domain is intended to codify ethics-related, often unintentional failure modes, e.g. algorithmic bias, misinformation.
IDSub-IDNameDescription
E0100Bias/DiscriminationConcerns of algorithms propagating societal bias
E0101Group fairnessFairness towards specific groups of people
E0102Individual fairnessFairness in treating similar individuals
E0200ExplainabilityAbility to explain decisions made by AI
E0201Global explanationsExplain overall functionality
E0202Local explanationsExplain specific decisions
E0300User actionsPerpetuating/causing/being affected by negative user actions
E0301ToxicityUsers hostile towards other users
E0302Polarization/ ExclusionUser behavior skewed in a significant direction
E0400MisinformationPerpetuating/causing the spread of falsehoods
E0401Deliberative MisinformationGenerated by individuals., e.g. vaccine disinformation
E0402Generative MisinformationGenerated algorithmically, e.g. Deep Fakes
# Performance This domain is intended to codify deficiencies such as privacy leakage or lack or robustness.
IDSub-IDNameDescription
P0100Data issuesProblems arising due to faults in the data pipeline
P0101Data driftInput feature distribution has drifted
P0102Concept driftOutput feature/label distribution has drifted
P0103Data entanglementCases of spurious correlation and proxy features
P0104Data quality issuesMissing or low-quality features in data
P0105Feedback loopsUnaccounted for effects of an AI affecting future data collection
P0200Model issuesAbility for the AI to perform as intended
P0201Resilience/stabilityAbility for outputs to not be affected by small change in inputs
P0202OOD generalizationTest performance doesnā€™t deteriorate on unseen data in training
P0203ScalingTraining and inference can scale to high data volumes
P0204AccuracyModel performance accurately reflects realistic expectations
P0300PrivacyProtect leakage of user information as required by rules and regulations
P0301AnonymizationProtects through anonymizing user identity
P0302RandomizationProtects by injecting noise in data, eg. differential privacy
P0303EncryptionProtects through encrypting data accessed
P0400SafetyMinimizing maximum downstream harms
P0401Psychological SafetySafety from unwanted digital content, e.g. NSFW
P0402Physical safetySafety from physical actions driven by a AI system
P0403Socioeconomic safetySafety from socioeconomic harms, e.g. harms to job prospects or social status
P0404Environmental safetySafety from environmental harms driven by AI systems
# Lifecycle View The stages in this view represent high-level sequential steps of a typical ML workflow. Following the widely-used Cross-industry standard process for data mining ([CRISP-DM](https://en.wikipedia.org/wiki/Cross-industry_standard_process_for_data_mining)) framework, we designate six stages in this view. | ID | Stage | | --- | ---------------------- | | L01 | Business Understanding | | L02 | Data Understanding | | L03 | Data Preparation | | L04 | Model Development | | L05 | Evaluation | | L06 | Deployment | Figure T.2 reconciles the two different views of the AVID taxonomy. We conceptually represent the potential space of risks in three dimensions, consisting of the risk domainā€”S, E, or Pā€”a specific vuln pertains to; the (sub)category within a chosen domain; and the development lifecycle stage of a vuln. The SEP and lifecycle views are simply two different sections of this three-dimensional space.

Figure T.2. SEP and Lifecycle views represent different sections of the space of potential risks in an AI development workflow.

# Schema Adhering to our aim for AVID resources to be expandable and adaptable to practitioner needs, we have adapted the [MISP Taxonomy System](https://www.circl.lu/doc/misp-taxonomies/) to standardize and share the two views of our taxonomy. MISP taxonomies are used to tags cybersecurity events, indicators, and threats using three components: * `namespace` is an unique identifier of the taxonomy being used * `predicate` is a high-level category * `value` is a low-level subcategory under a predicate. Each MISP taxonomy is specified using a single JSON file that contains the namespace, a list of predicates, lists of values under each predicate, and auxiliary metadata. Below is a sample schema: {% code overflow="wrap" fullWidth="false" %} ```json { "namespace": ..., "description": ..., "version": ..., "predicates": [ { "value": ..., "expanded": ..., "description": ... }, ... ], "values": [ { "predicate": ..., "entry": [ { "value": ..., "expanded": ..., "description": ... }, ... ] }, ... ] } ``` {% endcode %} This specification can be used to tag any relevant threat information as `namespace:predicate:value`. As long as a taxonomy is specified using the above structure, tags can be generated in the above structure, providing the user with the flexibility of using multiple taxonomies, some of which may be specific to their own application context. ### Example The MISP specifications of the AVID taxonomies are available for the [Effect/SEP View](https://github.com/avidml/avid-db/blob/main/schema/taxonomy_misp/avid-effect.json) and the [Lifecycle View](https://github.com/avidml/avid-db/blob/main/schema/taxonomy_misp/avid-lifecycle.json). As an implementation example, consider the vulnerability [AVID-2022-V013](https://avidml.org/database/avid-2022-v013/), which is about the Microsoft Tay Poisoning incident. We have assigned the following taxonomy categories to it:
ViewCategoryID
EffectSecurityS0601: Ingest Poisoning
EffectEthicsE0101: Group Fairness
EffectEthicsE0301: Toxicity
LifecycleLifecycleL06: Deployment
The MISP tags for this vulnerability will be the following: ``` avid-effect:security:S0601 avid-effect:ethics:E0101 avid-effect:ethics:E0301 avid-lifecycle:lifecycle:L06 ``` # Custom Taxonomies Using the MISP format allows us to seamlessly integrate arbitrary taxonomies into the AVID database and related workflows. This is crucial for practitioner adoption, since AI developers and vendors often work with operational taxonomies specific to their domain. Examples of such deep taxonomies/categorizations include MITRE ATLAS, taxonomies for [downstream harms](#user-content-fn-1)[^1] and [LLM risks](#user-content-fn-2)[^2], and [Risk Cards](#user-content-fn-3)[^3]. ## Auxiliary Taxonomies in avid-schema The following non-AVID taxonomies are currently maintained in [avid-schema/taxonomy](https://github.com/avidml/avid-schema/tree/main/taxonomy): * [InjectLab LLM Taxonomy (`injectlab-llm.json`)](https://github.com/avidml/avid-schema/blob/main/taxonomy/injectlab-llm.json): categorizes LLM prompt-injection and jailbreak behaviors. * [Risk Cards (`risk-cards.json`)](https://github.com/avidml/avid-schema/blob/main/taxonomy/risk-cards.json): captures deployment-time risk dimensions used in Risk Cards style assessments. * [Trail of Bits ODDs (`trail-of-bits-ODDs.json`)](https://github.com/avidml/avid-schema/blob/main/taxonomy/trail-of-bits-ODDs.json): represents Operational Design Domain constraints and assumptions for AI systems[^4]. We welcome community contributions to the taxonomy repository. To contribute, add a JSON file following the MISP structure and open a pull request in [avid-schema](https://github.com/avidml/avid-schema). [^1]: [Shelby et al](https://arxiv.org/abs/2210.05791). Identifying Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction, arXiV, 2021. [^2]: [Weidinger et al](https://dl.acm.org/doi/10.1145/3531146.3533088). Taxonomy of Risks posed by Language Models, FAccT, 2022. [^3]: [Derczynski et al](https://arxiv.org/abs/2303.18190). Assessing Language Model Deployment with Risk Cards, arXiV, 2023. [^4]: Khlaaf, Heidy. Toward Comprehensive Risk Assessments and Assurance of AI-Based Systems, Trail of Bits, 2023.