ipipan/silesian-wikipedia-clean-20230901

---
license: cc-by-sa-4.0
---

# Model Card for Clean Silesian Wikipedia

This is a cleaned and filtered snapshot of 20230901 Silesian Wikipedia.

## License
CC BY-SA 4.0

## Citation
If you use this model, please cite the following paper:
```
@inproceedings{rybak-2024-transferring-bert,
title = "Transferring {BERT} Capabilities from High-Resource to Low-Resource Languages Using Vocabulary Matching",
author = "Rybak, Piotr",
editor = "Calzolari, Nicoletta and
Kan, Min-Yen and
Hoste, Veronique and
Lenci, Alessandro and
Sakti, Sakriani and
Xue, Nianwen",
booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
month = may,
year = "2024",
address = "Torino, Italia",
publisher = "ELRA and ICCL",
url = "https://aclanthology.org/2024.lrec-main.1456",
pages = "16745--16750",
abstract = "Pre-trained language models have revolutionized the natural language understanding landscape, most notably BERT (Bidirectional Encoder Representations from Transformers). However, a significant challenge remains for low-resource languages, where limited data hinders the effective training of such models. This work presents a novel approach to bridge this gap by transferring BERT capabilities from high-resource to low-resource languages using vocabulary matching. We conduct experiments on the Silesian and Kashubian languages and demonstrate the effectiveness of our approach to improve the performance of BERT models even when the target language has minimal training data. Our results highlight the potential of the proposed technique to effectively train BERT models for low-resource languages, thus democratizing access to advanced language understanding models.",
}
```

## Authors
The model was created by Piotr Rybak from [Linguistic Engineering Group at Institute of Computer Science, Polish Academy of Sciences](http://zil.ipipan.waw.pl/).

This work was supported by the European Regional Development Fund as a part of 2014–2020 Smart Growth Operational Programme, CLARIN — Common Language Resources and Technology Infrastructure, project no. POIR.04.02.00-00C002/19.

原始信息汇总

数据集概述

数据集名称

Clean Silesian Wikipedia

数据集描述

这是一个经过清理和过滤的20230901 Silesian Wikipedia快照。

许可证

CC BY-SA 4.0

引用信息

若使用此模型，请引用以下论文：

@inproceedings{rybak-2024-transferring-bert, title = "Transferring {BERT} Capabilities from High-Resource to Low-Resource Languages Using Vocabulary Matching", author = "Rybak, Piotr", editor = "Calzolari, Nicoletta and Kan, Min-Yen and Hoste, Veronique and Lenci, Alessandro and Sakti, Sakriani and Xue, Nianwen", booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)", month = may, year = "2024", address = "Torino, Italia", publisher = "ELRA and ICCL", url = "https://aclanthology.org/2024.lrec-main.1456", pages = "16745--16750", abstract = "Pre-trained language models have revolutionized the natural language understanding landscape, most notably BERT (Bidirectional Encoder Representations from Transformers). However, a significant challenge remains for low-resource languages, where limited data hinders the effective training of such models. This work presents a novel approach to bridge this gap by transferring BERT capabilities from high-resource to low-resource languages using vocabulary matching. We conduct experiments on the Silesian and Kashubian languages and demonstrate the effectiveness of our approach to improve the performance of BERT models even when the target language has minimal training data. Our results highlight the potential of the proposed technique to effectively train BERT models for low-resource languages, thus democratizing access to advanced language understanding models.", }

作者

Piotr Rybak，来自波兰科学院计算机科学研究所的语言工程组。