Record Linkage Datasets

This simulated dataset is a corrupted segment from the Social Security Death Master File (SSDMF) available at https://ssdmf.info/. There are 11 original datasets: ``dsxo`` where `x` runs from `1...11` and the suffix `o` stands for `original`. The sizes (number of original records) of these datasets are as follows: | dataset | size ||:----------:|:----:|| ds1o | 10K || ds2o | 20K | | ds3o | 40K || ds4o | 80K || ds5o | 120K || ds6o | 160K || ds7o | 200K || ds8o | 400K || ds9o | 600K || ds10o | 800K || ds11o | 1M | These original records are then corrupted via a modified version of the `dsgen` Python script by `Peter Christen`.The modified/corrupted files are saved as: ``dsxm`` where the suffix `m` stands for `modified`.The modified records plus four original replicates are concatenated and mixed up (by the Linux command tool `shuf`).The resultant datasets are named: ``dsx.0`` ``(dsx.1)`` before(after) shuffling.So, the sizes of these datasets are as follows: | dataset | size ||:-------:|:----:|| ds1.1 | 50k || ds2.1 | 100k || ds3.1 | 200k || ds4.1 | 400k || ds5.1 | 600k || ds6.1 | 800k || ds7.1 | 1M || ds8.1 | 2M || ds9.1 | 3M || ds10.1 | 4M || ds11.1 | 5M | Furthermore, each dataset is split into two halves to serve as input for record linkage algorithms. For example, ds1.1 is split into ds1.1.1 & ds1.1.2.本模拟数据集取自可通过https://ssdmf.info/获取的社会保障死亡主文件（Social Security Death Master File，SSDMF）的损坏片段。本数据集包含11组原始数据，命名格式为`dsxo`，其中x的取值范围为1至11，后缀`o`代表“原始（original）”。各组原始数据集的原始记录数规模如下： | 数据集 | 记录数 | |:-----:|:-----:| | ds1o | 10K | | ds2o | 20K | | ds3o | 40K | | ds4o | 80K | | ds5o | 120K | | ds6o | 160K | | ds7o | 200K | | ds8o | 400K | | ds9o | 600K | | ds10o | 800K | | ds11o | 1M | 随后，通过彼得·克里斯滕（Peter Christen）开发的修改版`dsgen` Python脚本对上述原始记录进行损坏处理。经修改/损坏后的文件命名为`dsxm`，其中后缀`m`代表“修改（modified）”。将修改后的记录与4份原始副本进行拼接，并通过Linux命令行工具`shuf`执行打乱操作。打乱前与打乱后的结果数据集分别命名为`dsx.0`与`dsx.1`。据此，各组结果数据集的规模如下： | 数据集 | 记录数 | |:-----:|:-----:| | ds1.1 | 50k | | ds2.1 | 100k | | ds3.1 | 200k | | ds4.1 | 400k | | ds5.1 | 600k | | ds6.1 | 800k | | ds7.1 | 1M | | ds8.1 | 2M | | ds9.1 | 3M | | ds10.1 | 4M | | ds11.1 | 5M | 此外，每组结果数据集均被拆分为两个子集，用作记录链接算法的输入数据。例如，`ds1.1`被拆分为`ds1.1.1`与`ds1.1.2`。