SANTOS Benchmark for Table Union Search

This record contains the datasets released with SIGMOD 2023 paper entitled "SANTOS: Relationship-based Semantic Table Union Search". We release two new tabular benchmarks to evaluate the table union search problem over the data lakes. Furthermore, we also release relabeled ground truth for an existing TUS benchmark by taking the binary relationship between the columns into account. Please visit our paper for further details.

If you use our dataset for your work, please cite our paper as:

Aamod Khatiwada, Grace Fan, Roee Shraga, Zixuan Chen, Wolfgang Gatterbauer, Renée J. Miller, and MirekRiedewald. 2023. SANTOS: Relationship-based Semantic Table Union Search. SIGMOD Conference 2023, ACM

@article{DBLP:journals/pacmmod/KhatiwadaFSCGMR23,  author       = {Aamod Khatiwada and                  Grace Fan and                  Roee Shraga and                  Zixuan Chen and                  Wolfgang Gatterbauer and                  Ren{\'{e}}e J. Miller and                  Mirek Riedewald},  title        = {{SANTOS:} Relationship-based Semantic Table Union Search},  journal      = {Proc. {ACM} Manag. Data},  volume       = {1},  number       = {1},  pages        = {9:1--9:25},  year         = {2023},  doi          = {10.1145/3588689},}

You can find SANTOS implementation at: https://github.com/northeastern-datalab/santos

You can find the original TUS benchmark at: https://github.com/RJMillerLab/table-union-search-benchmark

Abstract: Existing techniques for unionable table search define unionability using metadata (tables must have the same or similar schemas) or column-based metrics (for example, the values in a table should be drawn from the same domain). In this work, we introduce the use of semantic relationships between pairs of columns in a table to improve the accuracy of union search. Consequently, we introduce a new notion of unionability that considers relationships between columns, together with the semantics of columns, in a principled way. To do so, we present two new methods to discover semantic relationship between pairs of columns: The first uses an existing knowledge base (KB), the second (which we call a “synthesized KB”) uses knowledge from the data lake itself. We adopt an existing Table Union Search benchmark and present new (open) benchmarks that represent small and large real data lakes. We show that our new unionability search algorithm called SANTOS outperforms a state-of-the-art union search that uses a wide variety of column-based semantics, including word embeddings and regular expressions. We show empirically in all benchmarks that our synthesized KB improves the accuracy of union search by representing relationship semantics that may not be contained in an available KB. This result hints at a promising future of creating a synthesized KBs from data lakes with limited KB coverage and using them for union search.