LexGLUE: A Benchmark Dataset for Legal Language Understanding in English

This benchmark dataset is published with the article: <em>Ilias Chalkidis, Abhik Jana, Dirk Hartung, Michael Bommarito, Ion Androutsopoulos, Daniel Martin Katz, and Nikolaos Aletras. 2021. LexGLUE: A Benchmark Dataset for Legal Language Understanding in English. ArXiv.</em> <strong>Short Description</strong> Inspired by the recent widespread use of the GLUE multi-task benchmark NLP dataset (Wang et al., 2018), the subsequent more difficult SuperGLUE (Wang et al., 2019), other previous multi-task NLP benchmarks (Conneau and Kiela,2018; McCann et al., 2018), and similar initiatives in other domains (Peng et al., 2019), we introduce LexGLUE, a benchmark dataset to evaluate the performance of NLP methods in legal tasks. LexGLUE is based on seven existing legal NLP datasets: ECtHR Task A (Chalkidis et al., 2019) ECtHR Task B (Chalkidis et al., 2021a) SCOTUS (Spaeth et al., 2020) EUR-LEX (Chalkidis et al., 2021b) LEDGAR (Tuggener et al. (2020) UNFAIR-ToS (Lippi et al., 2019) CaseHOLD (Zheng et al., 2021)