Base Classes
Report
Reports are occurrences of a GPAI failure mode. Based on the references provided in a specific report, reports can potentially more granular and reproducible than vulnerabilities. We classify reports in four types, in increasing degree of quantitative evidence:
Issue: qualitative evaluation based on a single sample or handful of samples,
Advisory: qualitative evaluation based on multiple Incidents,
Measurement: quantitative evaluation with associated data and metric,
Detection: A Measurement deemed critical by a threshold or statistical test.
These types are reminiscent of the three levels of AI Auditing, and accommodate diverse AI evaluation scenarios from the user perspective.
Vulnerability
Vulnerabilities are linked to the taxonomy through multiple tags, denoting the AI risk domains (Security, Ethics, Performance) this vulnerability pertains to, (sub)categories under that domain, as well as AI lifecycle stages. A vulnerability in AVID can pertain to one or more of the three levels: Dataset, Model, or System.
The field-level schema for vulnerabilities is stable, but the broader definition of an "AI vulnerability" remains an active area of standardization. AVID therefore uses a working definition and is currently prioritizing new report publication while vulnerability publication is temporarily paused in the latest release cycle.
Last updated