five

Replication Data for: Cross-lingual classification of political texts using multilingual sentence embeddings

收藏
DataONE2022-10-04 更新2024-06-08 收录
下载链接:
https://search.dataone.org/view/sha256:423e7eeac1b07a87163f5a56061fc571b15eaf87a94384ac20948f271adadfd4
下载链接
链接失效反馈
官方服务:
资源简介:
Established approaches to analyze multilingual text corpora require either a duplication of analysts' efforts or high-quality machine translation (MT). In this paper, I argue that multilingual sentence embedding (MSE) is an attractive alternative approach to language-independent text representation. To support this argument, I evaluate MSE for cross-lingual supervised text classification. Specifically, I assess how reliably MSE-based classifiers detect manifesto sentences' topics and positions compared to classifiers trained using bag-of-words representations of machine-translated texts, and how this depends on the amount of training data. These analyses show that when training data is relatively scarce (e.g. 20K or less labeled sentences), MSE-based classifiers can be more reliable and are at least no less reliable than their MT-based counterparts. Further, I examine how reliable MSE-based classifiers label sentences written in languages not in the training data, focusing on the task of discriminating sentences that discuss the issue of immigration from those that do not. This analysis shows that compared to the within-language classification benchmark, such \"cross-lingual transfer\" tends to result in fewer reliability losses when relying on the MSE instead of the MT approach. This study thus presents an important addition to the cross-lingual text analysis toolkit.
创建时间:
2023-11-08
5,000+
优质数据集
54 个
任务类型
进入经典数据集
二维码
社区交流群

面向社区/商业的数据集话题

二维码
科研交流群

面向高校/科研机构的开源数据集话题

数据驱动未来

携手共赢发展

商业合作