Replication Data for: Data pre-processing pipeline generation for AutoETL

Data pre-processing plays a key role in a data analytics process (e.g., applying a classification algorithm on a predictive task). It encompasses a broad range of activities that span from correcting errors to selecting the most relevant features for the analysis phase. There is no clear evidence, or rules defined, on how pre-processing transformations impact the final results of the analysis. The problem is exacerbated when transformations are combined into pre-processing pipeline prototypes. Data scientists cannot easily foresee the impact of pipeline prototypes and hence require a method to discriminate between them and find the most relevant ones (e.g., with highest positive impact) for their study at hand. Once found, these prototypes can be instantiated and optimized e.g., using Bayesian Optimization. In this work, we study the impact of transformations when chained together into prototypes, and the impact of transformations when instantiated via various operators. We develop and scrutinize a generic method that allows to generate pre-processing pipelines, as a step towards AutoETL. We make use of rules that enable the construction of prototypes (i.e., define the order of transformations), and rules that guide the instantiation of the transformations inside the prototypes (i.e., define the operator for each transformation). The optimization of our effective pipeline prototypes provide results that compared to an exhaustive search, get 90% of the predictive accuracy in the median, but with a time cost that is 24 times smaller.数据预处理在数据分析流程（例如在预测任务中应用分类算法）中扮演着核心角色。其涵盖了从修正数据错误到为分析阶段遴选最具相关性特征在内的诸多操作。目前尚无明确证据或既定规则，可阐明预处理变换如何影响数据分析的最终结果。当将多种预处理变换整合为预处理流水线原型时，该问题会进一步加剧。数据科学家难以预判各类预处理流水线原型的实际影响，因此亟需一种方法来甄别这些原型，并从中找出适配当前研究需求的最优方案（例如具备最高正向影响的原型）。筛选得到的原型可进行实例化与优化，例如采用贝叶斯优化(Bayesian Optimization)方法。本研究聚焦于变换串联为原型时的影响，以及通过各类算子实例化变换时的影响。我们开发并深入审视了一种可生成预处理流水线的通用方法，以此作为迈向自动ETL(AutoETL)的研究步骤。我们利用两类规则构建原型：一类用于定义变换的执行顺序，另一类用于指导原型内部变换的实例化，即针对每个变换指定其所使用的算子。相较于穷举搜索方案，我们优化后的高效流水线原型在中位数层面可达到90%的预测准确率，同时时间开销仅为前者的二十四分之一。