HK Content Corpus (Cantonese & Traditional Chinese)

This dataset contains eight cleaned source-specific corpora of Hong Kong Cantonese and Traditional Chinese text, crawled from public websites and platforms.  It was initially created for the experiments reported in https://doi.org/10.1145/3744341 which study the effect of diglossia on Hong Kong language modeling.

This dataset is also available at HuggingFace (with file extension changed to csv and header row added for HF's dataset viewer function):https://huggingface.co/datasets/SolarisCipher/hk_content_corpus

👉 This cleaned corpus is derived from a larger MySQL database used to store raw text during the data collection stage.If you need the original database for reprocessing or reproduction, please refer to:https://doi.org/10.5281/zenodo.16875235

File format: UTF-8 plain text. One line per record, separated by blank line.

Source:

Apple Daily

Hong Kong Citizen Media

Hong Kong Citizen News

InMedia.hk

LIHKG

OpenRice

Stand News

Wikipedia (zh-hk)

Intended use: training/fine-tuning language models, linguistic analysis, text clustering.

NOTE: HKNSL became effective since 2020-6-30, which can create bias on user content created afterwards. Those portion of data should be used with caution.SHA256 checksum of files:0b6ad22b7a73230fd0e44af904c0cff1773cc871417f6ad3af11a783564bca15 appledaily_article_dedup.corpus373a5b369d5e402e58760861e2bff2e618e7ee7fd4494988a39be1156f9dba84 hkcitizenmedia_article_dedup.corpus67f909cfaf7d7a67df1dc79448f24622ec525a31b73b2e3293fbaad147470e69 hkcnews_article_dedup.corpus135251f3e8ba7587018b7b2814b9ea5bbebf69c98adca73d3ea4c9f9b5571957 inmedia_article_dedup.corpus8e5e6de9c219aeccdaf13e9162b00cce2eeb7f595023a7bf19d4b5660395a3ee lihkg_posts_dedup_demoji_128.corpusdd5835a7effe49bb96a31e0c0eab43dea17ef23b3e1d9cefdc186aca276897ce openrice_review_dedup_demoji.corpus847ef0f5809481caf4bf21100e4b34207513d724dafb9f42b162ef49079e7dba thestandnews_article_dedup.corpusbd33008802797b33df8484cf1113be6a0b38547fe13515f5c4edbf4ccad270db wiki_hk.corpusIf you use this dataset, please cite the following paper, and optionally cite the dataset DOI:@article{Yung2025HKDiglossia,  author    = {Yung, Yiu Cheong and Lin, Ying-Jia and Kao, Hung-Yu},  title     = {Exploring the Effectiveness of Pre-training Language Models with Incorporation of Diglossia for Hong Kong Content},  journal   = {ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP)},  volume    = {24},  number    = {7},  pages     = {71:1--71:16},  year      = {2025},  publisher = {Association for Computing Machinery},  doi       = {10.1145/3744341}}