🐢Giskard

Giskard provides an open-source Machine Learning (ML) testing framework, covering traditional ML as well as LLM use cases. Giskard Scan is a powerful tool to detect vulnerabilities in LLMs. Its integration with AVID taxonomy provides improved standardized reporting of vulnerabilities, and the ability to share your vulnerability reports with the community.

Taxonomy

By default, all Giskard scan reports indicate the AVID taxonomy categories that are relevant to the detected vulnerabilities. You can find this information in the detail view of each issue in the scan widget.

Exporting AVID reports

You can export your Giskard scan report as an AVID report. First, make sure you have avidtools installed in your environment.

pip install avidtools

Then, once you have run the Giskard scan, you can export the report as an AVID report.

import giskard as gsk

scan_report = gsk.scan(my_model, my_dataset)

# Export the report as a list of AVID reports (one per each vulnerability)
avid_reports = scan_report.to_avid()

You can also export these reports directly in a JSONL file (one AVID report per line):

# Write the AVID reports to a JSONL file
scan_report.to_avid("avid_report.jsonl")

For more details on how to use Giskard in combination with AVID, check out this tutorial.