MultiTACRED

<h3>Introduction</h3>
<p>TAC Relation Extraction Dataset (TACRED) was developed by <a href="https://nlp.stanford.edu/">The Stanford NLP Group</a> and is a large-scale relation extraction dataset with 106,264 examples built over English newswire and web text used in the NIST <a href="https://www.ldc.upenn.edu/collaborations/current-projects/tac-kbp">TAC KBP</a> English slot filling evaluations during the period 2009-2014. The annotations were derived from TAC KBP relation types (see <a href="https://tac.nist.gov/2017/KBP/ColdStart/guidelines.html">the guidelines</a>), from human annotations developed by the Linguistic Data Consortium and from crowdsourcing using <a href="https://www.mturk.com/">Mechanical Turk</a>.</p>
<h3>Data</h3>
<p>In each year of the slot filling evaluation, 100 entities (people or organizations) were given as queries (i.e., subjects), for which participating systems should find associated relations and object entities. All sentences judged in the TAC KBP evaluation and a sampling of other sentences that contain the query entities in the evaluation corpus form TACRED. Each sentence was crowd-annotated using Mechanical Turk, where each turk annotator was asked to annotate the subject and object entity spans and the corresponding relation.</p>
<p>Data is presented in both CoNLL and JSON format, both encoded in UTF-8. Scoring tools and gold relation labels are also included. Source corpora used for this dataset were TAC KBP Comprehensive English Source Corpora 2009-2014 (<a href="../../../LDC2018T03">LDC2018T03</a>) and TAC KBP English Regular Slot Filling - Comprehensive Training and Evaluation Data 2009-2014 (<a href="../../../LDC2018T22">LDC2018T22</a>).</p>
<p>For detailed information about the dataset and benchmark results, please refer to <a href="https://nlp.stanford.edu/pubs/zhang2017tacred.pdf">the TACRED paper</a>.</p>
<h3>Samples</h3>
<p>Please view this&nbsp;<a href="desc/addenda/LDC2018T24.conll">CoNLL sample</a>&nbsp;and&nbsp;<a href="desc/addenda/LDC2018T24.json">JSON sample</a>.</p>
<h3>Updates</h3>
<p>None at this time.</p>