Blog-1K

The Blog-1K corpus is a redistributable authorship identification testbed for contemporary English prose. It has 1,000 candidate authors, 16K+ posts, and a pre-defined data split (train/dev/test proportional to ca. 8:1:1). It is a subset of the Blog Authorship Corpus from Kaggle. The MD5 for Blog-1K is '0a9e38740af9f921b6316b7f400acf06'. 1. Preprocessing We first filter out texts shorter than 1,000 characters. Then we select one thousand authors whose writings meet the following criteria:<br> - accumulatively at least 10,000 characters,<br> - accumulatively at most 49,410 characters,<br> - accumulatively at least 16 posts,<br> - accumulatively at most 40 posts, and <br> - each text has at least 50 function words found in the Koppel512 list (to filter out non-English prose). Blog-1K has three columns: 'id', 'text', and 'split', where 'id' corresponds to its parent corpus. 2. Statistics Its creation and statistics can be found in the Jupyter Notebook. Split # Authors # Posts # Characters Avg. Characters Per Author (Std.) Avg. Characters Per Post (Std.) Train 1,000 16,132 30,092,057 30,092 (5,884) 1,865 (1,007) Validation 935 2,017 3,755,362 4,016 (2,269) 1,862 (999) Test 924 2,017 3,732,448 4,039 (2,188) 1,850 (936) <br> 3. Usage <pre><code class="language-python">import pandas as pd df = pd.read_csv('blog1000.csv.gz', compression='infer') # read in training data train_text, train_label = zip(*df.loc[df.split=='train'][['text', 'id']].itertuples(index=False))</code></pre> 4. License<br> All the materials is licensed under the ISC License. <br> 5. Contact<br> Please contact its maintainer for questions.