Replication Code and Data: Emoji-Based Stylometric Features for Pairwise Authorship Verification in Social Media
收藏资源简介：
<p>This archive contains the complete, standalone replication code for the manuscript:</p><p><strong>[Anonymized for peer review] (2026). Emoji fingerprints: Topic-independent authorship verification using 20 novel emoji-based stylometric features.</strong> Manuscript FSI-D-26-00293, under review at <em>Forensic Science International</em>. Author details are withheld during peer review and will be added upon acceptance.</p> <h4>What this study does</h4><p>We propose <strong>20 novel emoji-based stylometric features</strong> (covering distributional, vocabulary, behavioral, sequential, positional, and temporal dimensions of emoji usage) and evaluate them for pairwise authorship verification on social media text. The emoji features are compared against 27 classic stylometric features and a 47-feature hybrid model using a Random Forest classifier (configuration fixed a priori; no tuning) across 49 experimental configurations (7 profile sizes × 7 test sizes), and the verification scores are additionally evaluated as calibrated forensic likelihood ratios.</p> <h4>What this archive contains (v2.0.0)</h4><ul> <li><code>run_authorship_pipeline.py</code> — a single verified driver that reproduces every quantitative result in the manuscript: descriptive statistics and emoji frequency spectra; the emoji-only, classic and hybrid verification systems; significance analysis (McNemar with Holm correction + paired t-test on shared folds); ROC/confusion analyses; probabilistic calibration (Brier, ECE); a score-based likelihood-ratio layer (Cllr/Cllr_min, Tippett plots, PAV calibration diagrams, paired stratified bootstrap); cross-subreddit robustness (pair types A–D); a strict author-disjoint re-evaluation; an ablation study; and distance-metric / n-gram-vocabulary sensitivity analyses.</li> <li><code>emoji_features.py</code> and <code>classic_features.py</code> — canonical, unit-tested implementations of the 20 emoji-based and 27 classic features (the pipeline imports these directly).</li> <li><code>test_emoji_features.py</code> — unit tests for the corpus-aware features (F08, F16, F17, F20).</li> <li><code>prepare_reddit_archives.py</code> — data-provenance script that rebuilds the author archives from the public HuggingFace corpus (17 subreddit configurations, January 2021+ source period, manuscript inclusion criteria, canonical emoji detection).</li> <li><code>features_definition.json</code> — machine-readable definitions of all 47 features and of the modeling choices.</li> <li><code>README.md</code> — documentation, including a correspondence table mapping the version-1 scripts (00–08) to the cells of the unified pipeline.</li></ul> <h4>Dataset</h4><p>Experiments were conducted on <strong>4,697 unique Reddit authors</strong> (1,023,826 comments, 44 subreddits), filtered from the publicly available <a href="https://huggingface.co/datasets/HuggingFaceGECLM/REDDIT_comments">HuggingFace REDDIT_comments</a> corpus (accessed 15 January 2024). The filtered per-author comment archive used by the pipeline is included on this record so that all results can be reproduced exactly; raw Reddit data beyond this derived subset is not redistributed.</p> <h4>Key results</h4><ul> <li>Emoji-only model: 0.787 overall mean accuracy with only 20 features; classic baseline: 0.873 with 27 features; hybrid model: 0.895 with 47 features, significantly outperforming the classic baseline at moderate sample sizes (concordant McNemar, paired t-test and ΔCllr evidence at P = 25–100).</li> <li>Topic independence: when profile and test samples come from different subreddits, emoji features retain virtually identical accuracy (Δ = −0.009) while classic features degrade by 26.6 percentage points.</li> <li>Forensic calibration: log-likelihood-ratio cost falls steeply with more data, reaching Cllr = 0.127 for the hybrid model at P = 100.</li></ul> <h4>Reproducibility</h4><p>All random operations use a fixed seed (42). Random Forest hyperparameters are fixed a priori across all experiments (no tuning), with a stratified 80/20 train/test split at the pair level. The pipeline is checkpoint-resumable, and every checkpoint carries a feature-version stamp so stale results are recomputed automatically. Unit tests verify the corpus-aware feature implementations.</p> <h4>Version note</h4><p>Version 2.0.0 supersedes the earlier script-based layout (00–08). It adds the corrected emoji detection (keycap emoji are matched only as complete sequences), the exact corpus-aware features F08/F16/F17/F20, the likelihood-ratio layer, the ablation and sensitivity analyses, and the final cross-subreddit design. Version-1 scripts reproduce the pre-correction results and should not be used to check the revised manuscript.</p>
