midah/license-pairwise-hf

该数据集是一个成对许可证比较数据集（Hugging Face子集），用于文本分类任务，专注于法律NLP和LLM作为评估器的应用。数据集包含93个来自Hugging Face Hub资产的SPDX标记许可证，通过成对比较判断哪个许可证更具许可性（即更宽松）。数据集提供多个配置，对应不同的提示版本（v4、v6、v7）和LLM评估器（如Anthropic Claude Sonnet 4.6、OpenAI GPT-4o、Qwen Qwen3.6 Plus Free等）。每个配置包含训练数据，存储为Parquet文件。v4版本提供三元裁决（A > B、A = B、A < B）、自信度评估和引用决定性差异的详细信息；v6版本简化为二元裁决（A > B、A < B），移除了自信度，并引入了不可比性标志；v7版本恢复三元裁决，但将不可比性标志仅限于正交义务情况。数据集还包括手动注释子集，用于计算LLM裁决与人工注释之间的一致性。数据集适用于研究许可证分类、LLM评估性能以及部分顺序分析。

This dataset is a pairwise license-comparison dataset (HF subset) for text-classification tasks, focusing on legal NLP and LLM-as-judge applications. It contains 93 SPDX-tagged licenses from Hugging Face Hub assets, with pairwise comparisons to determine which license is strictly more permissive. The dataset offers multiple configurations corresponding to different prompt versions (v4, v6, v7) and LLM raters (e.g., Anthropic Claude Sonnet 4.6, OpenAI GPT-4o, Qwen Qwen3.6 Plus Free). Each configuration includes training data stored as Parquet files. Version v4 provides ternary verdicts (A > B, A = B, A < B), self-reported confidence, and detailed decisive differences with quoted text; v6 simplifies to binary verdicts (A > B, A < B), drops confidence, and introduces incomparability flags; v7 restores ternary verdicts but limits incomparability to orthogonal obligations only. The dataset also includes a manual annotation subset for computing inter-rater agreement between LLM verdicts and human annotations. It is suitable for research on license classification, LLM judge performance, and partial-order analysis.