遇见数据集

esCorpiusDialog: A Large-Scale Multilingual Dialogue Dataset in Spanish, Catalan, Basque, and Galician

收藏
Zenodo2026-10-29 更新2026-11-05 收录
官方服务:

资源简介:

esCorpiusDialog is a large-scale multilingual dialogue corpus designed to support conversational modelling and dialogue research in Spanish, Catalan, Basque, and Galician. Version 1.3 represents 30,314,428 reconstructed dialogue records and 129,164,022 turn occurrences, averaging approximately 4.3 turns per record. Spanish accounts for 30,043,525 records, followed by Basque with 115,796, Catalan with 92,009, and Galician with 63,098. These totals describe reconstructed sequences and retain repeated occurrences; they are not counts of unique source messages. The corpus combines movie and television subtitles from OpenSubtitles, Usenet newsgroups, online forums, including Menéame, Mediavida, and Reddit, and literary works from Project Gutenberg. Version 1.3 preserves the Gutenberg and dehydrated forum archives from version 1.2 while replacing the text-bearing OpenSubtitles and Usenet components with dehydrated representations. Dialogue text is included for 12,865 Gutenberg dialogues comprising 95,325 turn occurrences, drawn from 95 books checked and confirmed as public domain in the United States. The remaining 30,301,563 dialogue records are distributed through identifiers, ordered structures, and reconstruction metadata. Each dehydrated component preserves its source-specific organisation. Forum records contain thread or post identifiers and ordered comment-identifier sequences representing root-to-leaf dialogue paths, without raw forum text or user handles. Usenet records retain ordered message-hash chains and message-table hashes and dates, with every subject and body field replaced by the marker [dehydrated]. The OpenSubtitles component provides a manifest covering 206,712 historical subtitle files, with dialogue counts, ordered-list fingerprints, reconstruction settings, and historical metadata, without subtitle or dialogue text. These representations support structural analysis and verification of independently reconstructed content. Current source availability and exact reconstruction are not guaranteed, and dehydration does not make identifiers or metadata anonymous. CC BY 4.0 applies to the authors’ original metadata, structural contributions, and any compilation or database rights they hold. It permits copying, redistribution, and adaptation, including commercially, with appropriate attribution, a licence link, and an indication of changes. Public-domain book text retains its existing status wherever that status applies; reuse outside the United States depends on the applicable jurisdiction. The authors’ licence does not extend to independently retrieved third-party content. Access, rehydration, training, and redistribution involving such content remain subject to the relevant source terms, permissions, and applicable law. The accompanying README and data-rights documentation explain the package contents, representation formats, licensing scope, and limitations.

提供机构:
Zenodo
创建时间:
2026-10-29
二维码
社区交流群
二维码
科研交流群
商业服务