HK Web Text Corpus (MySQL Dump, raw version)

Database Description

Language: Hong Kong Cantonese, Traditional Chinese

Size: ~49.2 GB (SQL dump), 11.1 GB (7z archive)

Format: MySQL dump, UTF-8 encoding

Source: public web sources (news sites, online forums, encyclopedia and restaurant reviews)

⚠ This dataset provides the MySQL dump file which contains a large-scale raw text corpus collected from various Hong Kong public web sources, primarily focused on Hong Kong Cantonese and Traditional Chinese language usage.  

It was used for generating Hong Kong Content Corpus, which was then used in the experiments reported in https://doi.org/10.1145/3744341 to study the effect of diglossia on Hong Kong language modeling.  

This MySQL database is intended for archival and reproducibility purposes, and may include noise, duplication, HTML markup, crawler residues, and records that were subsequently cleaned/filtered in the derived corpus release.  

This dataset is also available at HuggingFace as unsplited archive: https://huggingface.co/datasets/SolarisCipher/hk_content_corpus_mysql  

👉 If you are looking for the cleaned, ready-to-use corpus version, please refer to:  https://doi.org/10.5281/zenodo.16882351  

NOTE: HKNSL became effective since 2020-6-30, which can create bias on user content created afterwards. Those portion of data should be used with caution.SHA256 checksum of files:0c279f564d4fb02fe7b05c7d424d8e0497e7c26d9caeb3fd6c31d2561b6c4d83 hk_content.7z.001140c5f335799cc783d1ccadfce68f19d5efc6dba1794255c29445cec30bebfcb hk_content.7z.002efa6912d3792a21833808339725f17428341217d50d983ccf426d205c6104a38 hk_content.7z.003b3b7a600ec2e2b5c6ce9ebc1e545712e696c6f6f94b78d0473486609eb7fb854  [SQL file after decompression]If you use this database, please cite the following paper, and optionally cite the database DOI:@article{Yung2025HKDiglossia,  author    = {Yung, Yiu Cheong and Lin, Ying-Jia and Kao, Hung-Yu},  title     = {Exploring the Effectiveness of Pre-training Language Models with Incorporation of Diglossia for Hong Kong Content},  journal   = {ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP)},  volume    = {24},  number    = {7},  pages     = {71:1--71:16},  year      = {2025},  publisher = {Association for Computing Machinery},  doi       = {10.1145/3744341}}