遇见数据集

Chinese <-> English Name Entity Lists v 1.0

收藏
DataCite Commons2021-07-01 更新2025-04-16 收录
官方服务:

资源简介:

<h3>Introduction</h3><br> <p>This file contains documentation for Chinese English Name Entity Lists, v.1.0, Linguistic Data Consortium (LDC) catalog number LDC2005T34 and ISBN 1-58563-368-2.</p><br> <p>These Chinese-English bi-directional name entity lists are compiled from Xinhua News Agency newswire texts. Not every irregularity in the original source has been detected and normalized. Some Chinese characters are not encoded in the source and brackets are used to describe their composition. Except for the person name lists, most instances were left untouched in the created lists. An effort was made to replace GB-encoded characters (such as Roman numbers) in the English translation with ASCII characters. However no attempt has been made to do the opposite for Chinese names. The use of slashes as delimiters presents another problem. Some names may have internal slashes. Initially, double quotes ("") were used to enclose the name with an internal slash to avoid confusion without realizing that these is just one " in ASCII (as opposed to a set of enclosing " in GB). Later it was decided to use &amp;slash;. In future releases, some lists will be changed for greater consistency. Finally, most of the English names in the source use lower cases throughout. An effort was made to capitalize the initial letter (and possibly some middle ones) for person names, but not for any other kind of names as most other names have multiple words, some of which may contain articles and prepositions.</p><br> <p>The word "English" is somewhat misleading here. Although most of the foreign words are English or can appear in English texts, there are also many non-English words written in Roman alphabet - some of which may have English equivalents while others do not. No efforts have been made to eliminate those non-English names where English equivlants are available.</p><br> <p>The entire set consists of nine pairs of lists. The English-&gt;Chinese version of each pair was created by reversing the Chinese-&gt;English, both sorted by the Unix built-in sort function.</p><br> <p>The contents are as follows</p><br> <ul><br> <li>Place Names, Chinese to English: 276,382</li><br> <li>Place Names, English to Chinese: 298,993</li><br> <li>Organization Names, Chinese to English: 30,800</li><br> <li>Organization Names, English to Chinese: 37,145</li><br> <li>Corporate Names, Chinese to English: 54,747</li><br> <li>Corporate Names, English to Chinese 58,468</li><br> <li>Press Organization Names, Chinese to English: 29,757</li><br> <li>Press Organization Names, English to Chinese: 32,922</li><br> <li>Intl. Organization Names, Chinese to English: 7,040</li><br> <li>Intl. Organization Names, English to Chinese: 7,040</li><br> </ul><br> <h3>Samples</h3><br> <p>For an example of the data in this publication, please view this <a href="desc/addenda/LDC2005T34.jpg" rel="nofollow">screen capture</a> of the corporate names list.</p><br> <h3>Additional Licensing Instructions</h3><br> <p>This 'members-only' corpora is available to current members who can request the data at the listed reduced-license fee. Contact&nbsp;<a href="mailto:ldc@ldc.upenn.edu">ldc@ldc.upenn.edu</a>&nbsp;for information about becoming a member.</p></br> Portions © 2001 Xinhua News Agency, © 2002, 2005 Trustees of the University of Pennsylvania

<h3>引言</h3><br><p>本文件为汉英双向命名实体列表(Chinese English Name Entity Lists)v1.0版的说明文档,由语言数据联盟(Linguistic Data Consortium, LDC)发布,编号为LDC2005T34,ISBN为1-58563-368-2。</p><br><p>本汉英双向命名实体列表均由新华社新闻电讯文本汇编而成。原始文本中的部分不规范之处未被全部检出并归一化处理。部分汉字未在源文件中完成编码,因此使用括号标注其构成方式。除人名列表外,其余生成列表中的绝大多数条目均保留原始格式。本次发布已尝试将英文译文中采用GB编码的字符(如罗马数字)替换为ASCII字符,但未针对中文名称进行反向替换。使用斜杠作为分隔符也带来了另一问题:部分名称本身包含内部斜杠。最初采用双引号("")对含内部斜杠的名称进行括注以避免混淆,但未注意到ASCII编码中仅存在单个双引号,与GB编码中的成对双引号存在差异。后续改为使用&amp;slash;进行标注。未来版本将对部分列表进行调整以提升一致性。此外,源文件中的绝大多数英文名称均采用全小写格式,本次发布已尝试对人名的首字母(以及部分中间单词的首字母)进行大写处理,但其余类型的名称未做此类调整——由于多数非人名名称由多个单词构成,其中可能包含冠词、介词等成分。</p><br><p>此处使用的"英文"一词存在一定误导性。尽管绝大多数外来词汇均为英语词汇或可在英文文本中出现,但仍有大量采用罗马字母书写的非英语词汇——部分词汇存在对应英文等价词,部分则无。本次发布未对存在对应英文等价词的非英语名称进行清理。</p><br><p>本数据集共包含九组双向列表。每组列表的英译中版本均通过反转汉译英列表生成,两组列表均采用Unix内置的sort排序函数完成排序。</p><br><p>具体内容如下</p><br><ul><br><li>地名:汉译英,共276,382条</li><br><li>地名:英译汉,共298,993条</li><br><li>组织机构名称:汉译英,共30,800条</li><br><li>组织机构名称:英译汉,共37,145条</li><br><li>企业名称:汉译英,共54,747条</li><br><li>企业名称:英译汉,共58,468条</li><br><li>新闻机构名称:汉译英,共29,757条</li><br><li>新闻机构名称:英译汉,共32,922条</li><br><li>国际组织名称:汉译英,共7,040条</li><br><li>国际组织名称:英译汉,共7,040条</li><br></ul><br><h3>示例</h3><br><p>若需查看本数据集的示例数据,请点击<a href="desc/addenda/LDC2005T34.jpg" rel="nofollow">企业名称列表截图</a>。</p><br><h3>附加授权说明</h3><br><p>本"仅对会员开放"语料库仅面向现有会员开放,会员可按照标注的优惠授权费申请获取该数据集。如需了解成为会员的相关信息,请联系<a href="mailto:ldc@ldc.upenn.edu">ldc@ldc.upenn.edu</a>&nbsp;。</p><br>部分内容 © 2001 新华通讯社,© 2002、2005 宾夕法尼亚大学托管委员会

创建时间:
2020-11-30
搜集汇总
数据集介绍
Chinese <-> English Name Entity Lists v 1.0 数据集图片
背景与挑战
背景概述
该数据集是中文与英文之间的双向命名实体列表,版本1.0,由Linguistic Data Consortium (LDC)发布,基于新华通讯社新闻文本编译。它包含九对列表,涵盖地名、组织名、公司名、新闻机构名和国际组织名,每对列表提供中文到英文和英文到中文的映射,条目总数从数千到数十万不等,但数据存在原始不一致性(如字符编码和分隔符问题)且未完全规范化。
以上内容由遇见数据集搜集并总结生成
二维码
社区交流群
二维码
科研交流群
商业服务