usejunior/legal-consensus

--- license: cc-by-4.0 task_categories: - text-classification - question-answering language: - en annotations_creators: - machine-generated language_creators: - expert-generated tags: - legal - law - compliance - citations - silver-labels - am-law-100 size_categories: - 10K<n<100K configs: - config_name: assertions data_files: assertions.parquet default: true - config_name: authorities data_files: authorities.parquet - config_name: firms data_files: firms.parquet --- # LegalConsensus: Multi-Firm Legal Interpretation Dataset ## Dataset Summary Structured legal assertions extracted from 54 Am Law 100 law firm publications. Contains 12,000+ assertions about 6,300+ legal authorities with Shepardizing-style treatment signals and short verbatim quotes. **This is the first and only multi-firm legal interpretation dataset on HuggingFace.** Unlike existing legal NLP datasets that focus on raw text or contract clauses, LegalConsensus captures how elite law firms *interpret* statutes, regulations, and case law — with treatment signals, consensus metrics, and citation-level evidence. ## Why This Dataset | Dataset | Focus | Source | What's Missing | |---------|-------|--------|----------------| | [LegalBench](https://huggingface.co/datasets/nguha/legalbench) | Benchmark tasks on case law | Academic annotations | No firm interpretations | | [CUAD](https://huggingface.co/datasets/cuad) | Contract clause extraction | SEC filings | No treatment signals | | [CaseHOLD](https://huggingface.co/datasets/casehold/casehold) | Case holdings | Court opinions | No practitioner analysis | | [Pile of Law](https://huggingface.co/datasets/pile-of-law/pile-of-law) | Raw legal text | Mixed government sources | No structure or labels | | **LegalConsensus** | **Firm interpretations of authorities** | **Am Law 100 publications** | **Silver labels (see below)** | LegalConsensus fills a unique gap: it captures the *interpretive layer* — how top law firms characterize authorities in practice, not just the raw legal text. ## Supported Tasks - **Treatment Classification** (silver-label): Given a quote about a legal authority, predict the Shepardizing-style treatment signal (7-class: cited, authoritative, analyzed, questioned, distinguished, superseded, enforced) - **Consensus Prediction**: Given an authority cited by multiple firms, predict the dominant treatment from the assertion set - **Firm Attribution**: Given a quote and authority, predict which firm authored the analysis ## Languages English only. ## Dataset Structure ### Config: `assertions` (primary) One row per assertion — a firm's treatment of a specific legal authority in a specific article. | Column | Type | Description | |--------|------|-------------| | `assertion_id` | int | Unique assertion identifier | | `document_id` | int | Source document identifier | | `authority_id` | string | Canonical authority identifier | | `citation` | string | Standard citation (e.g., "15 U.S.C. § 45") | | `authority_title` | string | Human-readable authority name | | `authority_type` | string | statute, regulation, case, executive_order, guidance, etc. | | `authority_jurisdiction` | string | federal, state:NY, state:CA, etc. | | `treatment` | string | Shepardizing signal: cited, authoritative, analyzed, questioned, distinguished, superseded, enforced | | `topic_context` | string | Specific topic within the authority (e.g., "non-compete enforceability") | | `topic_category` | string | High-level topic category | | `quote` | string | Verbatim snippet from the article (max 500 chars) | | `label_provenance` | string | Model that extracted the label (always "gemini-2.5-flash-lite") | | `key_analysis` | string | 2-3 sentence AI summary of the firm's analysis | | `fines_mentioned` | string | JSON array of fines/penalties mentioned, or null | | `firm_name` | string | Law firm name | | `firm_domain` | string | Firm website domain | | `document_title` | string | Article title | | `source_url` | string | Original article URL (for attribution) | | `document_date` | string | Publication date (ISO 8601) | | `document_jurisdiction` | string | Primary jurisdiction discussed | | `practice_area` | string | Primary practice area | | `document_type` | string | Article type: legal_update, client_alert, blog_post, etc. | | `depth_signal` | string | overview, analysis, or deep_dive | | `word_count` | int | Source article word count | ### Config: `authorities` One row per legal authority with consensus signals across firms. | Column | Type | Description | |--------|------|-------------| | `authority_id` | string | Canonical identifier | | `citation` | string | Standard citation format | | `title` | string | Human-readable title | | `authority_type` | string | statute, regulation, case, etc. | | `jurisdiction` | string | Jurisdiction | | `assertion_count` | int | Total assertions referencing this authority | | `firm_count` | int | Number of distinct firms citing this authority | | `dominant_treatment` | string | Most significant treatment signal (weighted) | | `confidence_band` | string | high, medium, or low | | `coverage_label` | string | high (10+ firms), moderate (3-9), low (1-2) | | `treatment_distribution` | string | JSON object with treatment counts | | `topic_contexts` | string | JSON array of unique topic contexts | ### Config: `firms` One row per firm in the dataset. | Column | Type | Description | |--------|------|-------------| | `firm_domain` | string | Firm website domain | | `firm_name` | string | Full firm name | | `document_count` | int | Number of substantive articles | | `assertion_count` | int | Total assertions from this firm | | `authority_count` | int | Distinct authorities cited | | `top_practice_areas` | string | JSON array of top practice areas | | `top_topics` | string | JSON array of top topic categories | ## Dataset Creation ### Source Publications from 54 of the Am Law 100 law firms — the largest and most prestigious U.S. law firms by revenue. Only publicly accessible content was collected; no login walls or anti-bot measures were circumvented. ### Extraction Method 1. **Scraping**: Public articles collected via HTTP requests, respecting robots.txt 2. **Cleaning**: Raw HTML processed with trafilatura to extract article text (~70% token reduction) 3. **Structuring**: Gemini 2.5 Flash-Lite extracted structured assertions, treatments, and quotes from cleaned text 4. **Filtering**: Only substantive articles included (word count >= 200, non-marketing document types excluded) ### Quality Controls - 3-sample verification during extraction pipeline development - 200-word minimum filters out press releases and announcements - Non-substantive document types (deal announcements, accolades, personnel news) excluded - Quote length capped at 500 characters ## Label Provenance All treatment labels were extracted by **Gemini 2.5 Flash-Lite** (`gemini-2.5-flash-lite`). These are **silver labels** — machine-generated, not human-validated. The `label_provenance` field is included on every assertion row for full transparency. A human-audited gold subset (200-500 rows) is planned for a future release to enable benchmark validation. ## Benchmark Tasks (Silver-Label) Three benchmark tasks are supported. Because labels are machine-generated, these are silver-label benchmarks — useful for model comparison and development, but not for definitive accuracy claims. ### 1. Treatment Classification (7-class) Given `quote` and `citation`, predict `treatment`. - Classes: cited, authoritative, analyzed, questioned, distinguished, superseded, enforced - Metric: macro F1 ### 2. Consensus Prediction Given an authority's assertion set (multiple rows), predict `dominant_treatment`. - Input: all assertions for a given `authority_id` - Metric: accuracy ### 3. Firm Attribution Given `quote` and `citation`, predict `firm_domain`. - 54 classes (one per firm) - Metric: top-5 accuracy ## Legal & Fair Use - **Short quotes only**: Max 500 characters per quote, from articles typically 1,000-5,000 words (~3-10% of source). Full article reconstruction is not possible. - **Original copyright**: Verbatim quotes remain under the copyright of their respective law firm authors. - **Attribution**: Source URLs provided for every assertion. CC-BY-4.0 requires attribution when using derived fields. - **Structure is ours**: Treatment labels, consensus signals, topic categories, and key_analysis are original derived works licensed CC-BY-4.0. See [DISCLAIMER.md](DISCLAIMER.md) for takedown policy and full terms. ## Limitations - **Silver labels**: All treatment signals are AI-extracted, not human-annotated. Error rates have not been formally measured. - **English only**: All source material is in English. - **Am Law 100 only**: Reflects the interpretive practices of large U.S. law firms; may not represent smaller firms, non-U.S. jurisdictions, or public interest perspectives. - **Temporal snapshot**: Publication dates vary; the dataset reflects a point-in-time view, not a continuously updated resource. - **Selection bias**: Only firms with publicly accessible content are included. Some major firms block public access to their publications. ## Takedown Policy If you are a rights holder and wish to have your content removed from this dataset, please contact [e-mail]. We will respond within 5 business days and remove requested content in the next dataset release. ## Citation ```bibtex @misc{legalconsensus2026, title={LegalConsensus: Multi-Firm Legal Interpretation Dataset}, author={UseJunior}, year={2026}, url={https://huggingface.co/datasets/usejunior/legal-consensus}, license={CC-BY-4.0} } ```