4chan Politically Incorrect Corpus

The present published data are the groundwork for the dissertation Hate Speech in the Digital World: A Linguistic Analysis of the Politically Incorrect 4chan Corpus and consists of seven files that include raw data, processed datasets, Python scripts, and linguistic analysis outputs. The dataset aims to facilitate further research on hate speech, computational linguistics, and online discourse analysis. "Chan_data" is a Jupyter Notebook (.ipynb format) that contains Python code used to extract data from the Politically Incorrect board on 4chan. It generates two datasets: 4chan-PIC, which is the full dataset of extracted posts, and 4chan-HM, a filtered dataset containing only posts with explicit hateful language. The script was developed using Anaconda Navigator and Jupyter Notebook. "chan_comment_final" is an Excel file containing three sheets. The first sheet, 4chan-PIC, includes the complete dataset of extracted posts. The second sheet, 4chan-HM, contains the filtered dataset with only posts that include explicit hateful language. The third sheet, Explicit 4chan-HM Hate List, consists of explicit hate speech words used to filter the 4chan-PIC dataset into 4chan-HM. "Cleaningandseparating" is another Jupyter Notebook that was used to convert the scraped dataset into a textual corpus (.txt format) for the linguistic analysis. "cleaned_text" is a plain text file that contains the finalised textual corpus used to compile the 4chan Politically Incorrect Corpus on Sketch Engine. "4chan-PIC Concordancing" is an Excel file containing concordancing data from Sketch Engine, which presents words in context and was used for linguistic analysis. "4chan-PIC Wordlists" is another Excel file that includes wordlist data from Sketch Engine, providing word and lemma frequency statistics to support quantitative analysis. "4chan-PIC Keyword analysis" is an Excel file that contains keyword analysis data from Sketch Engine, identifying statistically significant keywords in the 4chan Politically Incorrect Corpus compared to a reference corpus, English Web 21. These files support research in hate speech detection and analysis, computational linguistics, corpus-based studies, and online discourse analysis on social media. They also contribute to data-driven studies of extremist language trends. The datasets and scripts are provided for academic research and reproducibility purposes.本次公开的数据集为本学位论文《数字世界中的仇恨言论：对4chan政治不正确板块语料的语言学分析》的核心支撑资料，共包含7个文件，涵盖原始数据、处理后数据集、Python脚本与语言学分析产出成果。本数据集旨在推动仇恨言论、计算语言学及在线话语分析领域的后续研究。 "Chan_data"为Jupyter Notebook（.ipynb格式）文件，内含用于从4chan平台政治不正确板块（Politically Incorrect board）提取数据的Python代码。该脚本可生成两个数据集：全量提取帖文数据集4chan-PIC，以及仅包含明确仇恨性语言帖文的过滤后数据集4chan-HM。本脚本基于Anaconda Navigator与Jupyter Notebook开发。 "chan_comment_final"为Excel文件，内含3个工作表：第一个工作表"4chan-PIC"收录全量提取的帖文数据集；第二个工作表"4chan-HM"包含仅含明确仇恨性语言的过滤后数据集；第三个工作表"Explicit 4chan-HM Hate List"收录用于将4chan-PIC数据集过滤为4chan-HM的明确仇恨性言论词汇表。 "Cleaningandseparating"为另一款Jupyter Notebook，用于将爬取得到的数据集转换为语言学分析可用的文本语料库（corpus，.txt格式）。"cleaned_text"为纯文本文件，包含最终定稿的文本语料库，用于在Sketch Engine平台构建4chan政治不正确板块语料库。 "4chan-PIC Concordancing"为Excel文件，包含来自Sketch Engine的语境共现数据，用于展示词汇的上下文语境，为语言学分析提供支撑。"4chan-PIC Wordlists"为另一款Excel文件，包含来自Sketch Engine的词表数据，提供词汇与词元（lemma）的频率统计信息，以支持量化分析。 "4chan-PIC Keyword analysis"为Excel文件，包含来自Sketch Engine的关键词分析数据，相较于参考语料库English Web 21，可识别出4chan政治不正确板块语料库中具有统计学显著性的关键词。 上述文件可为仇恨言论检测与分析、计算语言学、基于语料库的研究及社交媒体在线话语分析等领域的研究提供支撑，同时助力极端主义语言趋势的数据驱动研究。本数据集与脚本仅用于学术研究与可复现性研究目的。