wikititles_glossary_cleaned_824k

# wikititles_glossary_cleaned_824k ## Dataset Description This dataset (`wikititles_glossary_cleaned_824k`) is a processed and cleaned version of the English-Chinese (en-zh) bilingual title alignment corpus from [OPUS WikiTitles](https://opus.nlpl.eu/). It contains aligned Wikipedia article titles, optimized for use in machine learning workflows, machine translation, or named entity alignment tasks. * **Original Source:** [OPUS - OPUS WikiTitles](https://opus.nlpl.eu/WikiTitles.php) * **Data Origin:** Wikipedia titles (derived from Wikimedia Foundation content) * **Total Entries:** 824,392 * **Preprocessing:** Applied **Traditional-to-Simplified Chinese (繁转简)** conversion to the Chinese field to ensure consistency for Simplified Chinese NLP tasks. ## Dataset Structure ### Data Fields * `en_term`: English Wikipedia title/term. * `zh_term`: Corresponding Chinese Wikipedia title/term. ### Data Format The dataset is provided in **Apache Arrow** format, which is optimized for high-performance loading with the `datasets` library in Hugging Face. * Files: `data-00000-of-00001.arrow` * Metadata: `dataset_info.json`, `state.json` ## Licensing & Attribution * **License:** This dataset is licensed under the **Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)** license. * **Copyleft Notice:** Due to the terms of the original Wikipedia data and the ShareAlike clause of the license, any derivative works (such as fine-tuned models or processed datasets) built upon this data **must also be released under the CC BY-SA 4.0 license** or a compatible copyleft equivalent. * **Attribution:** Please attribute the source to the OPUS project and the multilingual Wikipedia community. ## Citation If you use this corpus in your research or projects, please cite the original OPUS WikiTitles project: ```bibtex @inproceedings{tiedemann-2012-parallel, title = {Parallel Data, Tools and Interfaces in OPUS}, author = {J{\"o}rg Tiedemann}, booktitle = {Proceedings of the 8th International Conference on Language Resources and Evaluation (LREC 2012)}, year = {2012}, url = {[http://www.lrec-conf.org/proceedings/lrec2012/pdf/464_Paper.pdf](http://www.lrec-conf.org/proceedings/lrec2012/pdf/464_Paper.pdf)} } ```