Responsible Artificial Intelligence (RAI) Measures Dataset

Dataset WorkMeaningful governance of any system requires the system to be assessed and monitored effectively. In the domain of Artificial Intelligence (AI), global efforts have established a set of ethical principles like fairness, transparency, and privacy upon which AI governance expectations are being built. The computing research community has proposed numerous means of measuring an AI system’s normative qualities along these principles. Current reporting of these measures is principle-specific, limited in scope, or otherwise dispersed across publication platforms, hindering the domain’s ability to critique its practices. To address this, we introduce the Responsible AI Measures Dataset, consolidating 12,067 data points across 791 evaluation measures covering 11 ethical principles. It is extracted from a corpus of computing literature (n=257) published between 2011 and 2023. The dataset includes detailed descriptions of each measure, AI system characteristics, and publication metadata. An accompanying, interactive Sunburst visualization tool supports usability and interpretation. The Responsible AI Measures Dataset enables practitioners to explore existing assessment approaches and critically analyze how the computing domain measures normative concepts.Using the Interactive VisualizationThis dataset has a corresponding visualization that can be dynamically interacted with. It can be found as the "Sunburst_Visualization_Link.md" file in this repository. Note there are two versions. Version 1.0 was released in May 2025. Version 2.0 was released in August 2025.Demo Link: https://bit.ly/RAI_Measures_DemoTo use the visualization:Select a principle, followed by the component of the ML system, and the sociotechnical harm that you are interested in exploring. Note that hovering will display three pieces of metadata: the tier you are currently at in the visual, the parent (e.g., the tier prior), and the principle you are currently hovering above.Click on a measure to see the corresponding measurement process.Learn more about the measure, its formulaic variables (if quantitative), and relative context of use, please click the link on each measurement process to access the authors’ publication.Some measurement processes will include paper-specific references, terms, or formulas that may require further context to understand. Please use the paper title and lead author name(s) to further investigate the measure(s).Current VersionCurrently, this work is on Version 1.0 of the publicly shared dataset and corresponding visualization. The dataset is in a Microsoft Excel (.xslx) format. Note that it is not recommended to open the file in a .csv format due to the increased likelihood of corrupted characters and file formatting. Please read the below sections for more information on the dataset.Version 1.0 (July 2025)Target Output (Columns A and B in Blue): The resulting measures collected in this dataset.MeasureMeasurement ProcessEntry Points (Columns C and D in Orange): The primary features in narrowing down potential measures for an algorithmic system.PrinciplePart of the ML SystemConnections to Harm (Columns E - F in Pink): The sociotechnical harms for which the measure aims to make aware and/or mitigate.Primary HarmSecondary HarmMeasurement Properties (Columns G - I in Green): The standard(s) used in each measure's evaluation.Criterion NameCriterion DescriptionType of AssessmentAlgorithmic System Characteristics (Columns J - M in Purple): Additional features that a user can consider when narrowing down measures to use.Application AreaPurpose of ML SystemType of DataAlgorithm TypePublication Metadata (Columns N - P in Yellow): Details further documentation into each source that was extracted to collect each feature and measure.TitlePublication YearDOI LinkUsing the Interactive VisualizationThis dataset has a corresponding visualization that can be dynamically interacted with. It can be found as the "Version_1.0_Sunburst_Visualization_Link.md" file in this repository.