Evaluating Web Table Annotation Methods: From Entity Lookups to Entity Embeddings

Data sets used for experimental evaluation in the related publication: Evaluating Web Table Annotation Methods: From Entity Lookups to Entity Embeddings The data sets are contained within archive folders corresponding the three gold standard data sets used in the related publication. Each is presented in both .csv and .json formats. The gold standard data sets are collections of web tables:T2D consists of a schema-level gold standard of 1,748Web tables, manually annotated with class- and property-mappings, as well as an entity-level gold standard of 233 Web tables. Limaye consists of 400 manually annotated Web tables with entity-, class-, and property-level correspondences, where single cells (not rows) are mapped to entities. The corrected version of this gold standard is adapted to annotate rows with entities, from the annotations of the label column cells. WikipediaGS is an instance-level gold standard developed from 485K Wikipedia tables, in which links in the label column are used to infer the annotation of a row to a DBpedia entity. Data format CSV:The .csv files are formatted as double quoted (' " ') fields, separated by commas (',').In the tables files, each file corresponds to one table, each field represents a column, and each line represents a different row.In the entities files, there are only three fields:"DBpedia uri","cell string","row number"representing the correct annotation, the string of the label column cell, and the row (starting from 0) in which this mapping is found, respectively. Tables and entities files that correspond to the same table have the same filename.The same formatting and naming convention is used in T2D gold standard (http://webdatacommons.org/webtables/goldstandard.html). JSON:Each line in a .json file corresponds to a table, written as a JSONObject. T2D and Limaye tables files contain only one line (table) per file, while the Wikipedia gold standard contains multiple lines (tables) per .json file. In T2D and Limaye, the entity mappings of those tables can be found in the entities files with the same filename, while in Wikipedia, the entity mappings of each table can be found the line of the entities files having the "tableId" field as the one of the corresponding table. The contents of a table in .json are given as a two-dimensional array (a JSONArray of JSONArray s), called "contents". Each JSONArray in the contents represents a table row. Each element of this array is a JSONObject, representing one cell of the row. The field "data" of each cell contains the cell string contents, while there may also be a field "isHeader" to denote of the current cell is in a header row. In the Wikipedia gold standard there may also be a "wikiPageId" field, denoting the existing hyperlink of this cell to a Wikipedia page. It only contains the suffix of a Wikipedia URL, skipping the first part "https://en.wikipedia.org/wiki/". The entity mappings files are in the same format as in csv:["DBpedia uri","cell string",row number] inside the "mappings" field of a json file. Note on license: please refer to the README.txt. Data is derived from Wikipedia and other sources may have different licenses. Wikipedia contents can be shared under the terms of Creative Commons Attribution-ShareAlike License as outlined on Wikipedia: https://en.wikipedia.org/wiki/Wikipedia:Reusing_Wikipedia_content The correspondences of the T2D Gold standard is provided under the terms of the Apache license. The Web tables are provided according the same terms of use, disclaimer of warranties and limitation of liabilities that apply to the Common Crawl corpus. The DBpedia subset is licensed under the terms of the Creative Commons Attribution-ShareAlike License and the GNU Free Documentation License that applies to DBpedia. Limaye gold standard is downloaded from: http://websail-fe.cs.northwestern.edu/TabEL/ (download date: August 25, 2016). Please refer to the original website and the following paper for more details and citation information: G. Limaye, S. Sarawagi, and S. Chakrabarti. Annotating and Searching Web Tables Using Entities, Types and Relationships. PVLDB, 3(1):1338–1347, 2010. Also: THIS DATA IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.本数据集用于相关学术论文《网页表格标注方法评估：从实体查询到实体嵌入》（Evaluating Web Table Annotation Methods: From Entity Lookups to Entity Embeddings）中的实验评估。 本数据集包含三个对应于该论文中使用的金标准（gold standard）数据集的归档文件夹，每个数据集均提供.csv与.json两种格式。 本次涉及的金标准数据集均为网页表格（Web tables）集合： 1. T2D数据集包含1748张具备模式级金标准的网页表格，经人工标注了类别与属性映射关系；同时还包含233张具备实体级金标准的网页表格。 2. Limaye数据集包含400张经人工标注的网页表格，涵盖实体、类别与属性级别的对应关系，其标注对象为单个单元格（而非行）与实体的映射。该数据集的修正版本基于标签列单元格的标注，适配为行与实体的标注形式。 3. WikipediaGS是一个基于48.5万张维基百科（Wikipedia）表格构建的实例级金标准数据集，其利用标签列中的链接推断每行表格对应的DBpedia实体标注。 ### CSV格式说明 .csv文件采用双引号（""）包裹字段，以逗号（,）作为分隔符。在表格文件中，每个文件对应一张网页表格，每个字段代表一列，每一行代表一条不同的表格行。在实体标注文件中，仅包含三个字段：DBpedia统一资源标识符（DBpedia URI）、单元格字符串（cell string）以及行号（row number），分别对应正确的实体标注、标签列单元格的文本内容，以及该映射所在的行号（行号从0开始计数）。同名的表格文件与实体标注文件对应同一张网页表格。T2D金标准数据集采用相同的格式与命名规范（详见http://webdatacommons.org/webtables/goldstandard.html）。 ### JSON格式说明 .json文件的每一行对应一张网页表格，以JSON对象（JSONObject）形式存储。T2D与Limaye数据集的表格文件中，单个文件仅包含一行（即一张表格）；而维基百科金标准数据集的单个.json文件可包含多行（即多张表格）。在T2D与Limaye数据集中，表格的实体标注信息可在同名的实体标注文件中找到；而在维基百科金标准数据集中，每张表格的实体标注信息可在实体标注文件中"tableId"字段与对应表格匹配的行内找到。 JSON格式下的表格内容以二维数组（即JSON数组嵌套JSON数组）形式存储，对应字段名为"contents"。contents中的每个JSON数组代表表格的一行，数组中的每个元素为代表单个单元格的JSON对象。每个单元格对象的"data"字段存储单元格的文本内容，此外还可包含"isHeader"字段，用于标识当前单元格是否位于表头行。在维基百科金标准数据集中，单元格对象还可包含"wikiPageId"字段，用于标识该单元格对应的维基百科页面超链接，仅存储维基百科URL的后缀部分，省略前缀"https://en.wikipedia.org/wiki/"。 实体标注文件的格式与CSV格式一致，以JSON数组形式存储于JSON文件的"mappings"字段下，结构为["DBpedia uri","cell string",row number]。 ### 许可证说明 详见README.txt文件。本数据集衍生自维基百科及其他来源，不同来源可能适用不同的许可证条款。维基百科内容可按照知识共享署名-相同方式共享许可协议（Creative Commons Attribution-ShareAlike License）条款进行共享，具体条款可参见维基百科官网：https://en.wikipedia.org/wiki/Wikipedia:Reusing_Wikipedia_content。T2D金标准数据集的对应关系标注基于Apache许可证（Apache license）提供。本网页表格数据集的使用条款、免责声明与责任限制规则与通用爬虫语料库（Common Crawl corpus）一致。DBpedia子集的授权遵循适用于DBpedia的知识共享署名-相同方式共享许可协议与GNU自由文档许可证（GNU Free Documentation License）。 Limaye金标准数据集下载自：http://websail-fe.cs.northwestern.edu/TabEL/（下载日期：2016年8月25日）。如需了解更多细节与引用信息，请参考原始网站及以下论文：G. Limaye, S. Sarawagi, 和 S. Chakrabarti. 《使用实体、类型与关系标注并搜索网页表格》（Annotating and Searching Web Tables Using Entities, Types and Relationships）. PVLDB, 3(1):1338–1347, 2010. 此外：本数据集按“现状”提供，不附带任何形式的明示或默示担保，包括但不限于适销性、特定用途适用性以及非侵权性的担保。在任何情况下，作者或版权持有人均不对因本数据集的使用或其他交易而产生的任何索赔、损害或其他责任承担责任，无论是合同诉讼、侵权行为还是其他诉讼，无论是否由本数据集相关的使用或其他操作引起。