Blog-1K

The Blog-1K corpus is a redistributable authorship identification testbed for contemporary English prose. It has 1,000 candidate authors, 16K+ posts, and a pre-defined data split (train/dev/test proportional to ca. 8:1:1). It is a subset of the Blog Authorship Corpus from Kaggle. The MD5 for Blog-1K is '0a9e38740af9f921b6316b7f400acf06'.

1. Preprocessing

We first filter out texts shorter than 1,000 characters. Then we select one thousand authors whose writings meet the following criteria:
- accumulatively at least 10,000 characters,
- accumulatively at most 49,410 characters,
- accumulatively at least 16 posts,
- accumulatively at most 40 posts, and 
- each text has at least 50 function words found in the Koppel512 list (to filter out non-English prose).

Blog-1K has three columns: 'id', 'text', and 'split', where 'id' corresponds to its parent corpus.

2. Statistics

Its creation and statistics can be found in the Jupyter Notebook.

Split
# Authors
# Posts
# Characters
Avg. Characters Per Author (Std.)
Avg. Characters Per Post (Std.)

Train
1,000
16,132
30,092,057
30,092 (5,884)
1,865 (1,007)

Validation
935
2,017
3,755,362
4,016 (2,269)
1,862 (999)

Test
924
2,017
3,732,448
4,039 (2,188)
1,850 (936)

3. Usage

import pandas as pd

df = pd.read_csv('blog1000.csv.gz', compression='infer')

# read in training data
train_text, train_label = zip(*df.loc[df.split=='train'][['text', 'id']].itertuples(index=False))

4. License
All the materials is licensed under the ISC License.

5. Contact
Please contact its maintainer for questions.