SDCOR Synthetic Datasets

SDCOR: Scalable Density-based Clustering for Local Outlier Detection in Massive-Scale Datasets Link to arXiv e-print: https://arxiv.org/pdf/2006.07616.pdf Link to ResearchGate e-print: https://www.researchgate.net/publication/342197681_SDCOR_Scalable_Density-based_Clustering_for_Local_Outlier_Detection_in_Massive-Scale_Datasets This paper presents a method for local outlier detection in massive-scale datasets, which is based on a batch-wise density-based clustering approach. SDCOR consists of three major phases: 1) Sampling; 2) Scalable Clustering; and 3) Scoring. In the Sampling phase, a preliminary random sampling is conducted to obtain an abstraction of the entire data, named temporary clustering model; and also to acquire some information over the necessary parameters for the clustering procedures. Then, the Scalable Clustering phase will commence and the input data will be processed in chunks; as by processing successive chunks, the temporary clustering model gets gradual updates, till it turns into the final clustering model after processing the last chunk. Ultimately, at the last phase of the algorithm, regarding the final clustering model attained through the batch-wise clustering, and by employing the Mahalanobis distance criterion, each object is given an outlying score called SDCOR, which is equal to its local Mahalanobis distance. Each synthetic dataset in this repository is made of some Gaussian clusters with arbitrary mean vectors, far enough from each other, to impede probable overlappings among multidimensional clusters. For each of these artificial datasets, a specific amount of outliers are added around every cluster in the corresponding data; and moreover, the outliers "truth" is available along with each synthetic data. For every artificial dataset, there is a n-by-p matrix of dataset X (as n and p stand for the cardinality and dimensionality of the input data, respectively), along with the n-by-1 vector y of outlier labels, all together as a single binary MAT-file. We have implemented our code in MATLAB 9, which due to becoming reproducible, is accessible through our GitHub page (https://github.com/sana33/SDCOR). Finally, if you are interested in the idea or you are using this data for your research, please cite our paper as: @article{naghavi2021sdcor, title={SDCOR: Scalable density-based clustering for local outlier detection in massive-scale datasets}, author={Naghavi Nozad, Sayyed Ahmad and Amir Haeri, Maryam and Folino, Gianluigi}, journal={Knowledge-Based Systems}, pages={107256}, year={2021}, publisher={Elsevier} } Thanks a lot ...SDCOR：面向大规模数据集局部离群点检测的可扩展密度聚类（Scalable Density-based Clustering for Local Outlier Detection in Massive-Scale Datasets） 相关arXiv预印本链接：https://arxiv.org/pdf/2006.07616.pdf 相关ResearchGate预印本链接：https://www.researchgate.net/publication/342197681_SDCOR_Scalable_Density-based_Clustering_for_Local_Outlier_Detection_in_Massive-Scale_Datasets 本文提出一种基于批处理密度聚类方法的大规模数据集局部离群点检测方案。SDCOR主要包含三大核心阶段：1）采样阶段；2）可扩展聚类阶段；3）打分阶段。在采样阶段，首先通过初步随机采样获取全量数据的抽象表示，命名为临时聚类模型，同时采集聚类过程所需的必要参数信息。随后进入可扩展聚类阶段，输入数据将以分块形式进行处理；随着逐块处理的推进，临时聚类模型将逐步更新，直至处理完最后一个数据块后，得到最终的聚类模型。最终，在算法的最后阶段，依托批处理聚类得到的最终聚类模型，结合马哈拉诺比斯距离（Mahalanobis distance）准则，为每个数据对象赋予一个名为SDCOR的离群得分，该得分等价于其局部马哈拉诺比斯距离。 本仓库中的每个合成数据集均由若干高斯聚类（Gaussian clusters）构成，各聚类的均值向量间距足够大，以避免多维聚类间出现可能的重叠。针对每个人工数据集，会在对应聚类的周围添加特定数量的离群点；此外，每个人工数据集均附带对应的离群点真实标签。每个人工数据集均以单个二进制MAT-file的形式存储，包含一个n×p的数据集矩阵X（其中n和p分别代表输入数据的样本量与维度），以及一个n×1的离群点标签向量y。 我们基于MATLAB 9实现了本文的代码，为确保实验可复现，代码可通过我们的GitHub页面（https://github.com/sana33/SDCOR）获取。 若您对本方案感兴趣，或使用本数据集开展相关研究，请引用我们的论文： @article{naghavi2021sdcor, title={"SDCOR: Scalable density-based clustering for local outlier detection in massive-scale datasets"}, author={Naghavi Nozad, Sayyed Ahmad and Amir Haeri, Maryam and Folino, Gianluigi}, journal={Knowledge-Based Systems}, pages={107256}, year={2021}, publisher={Elsevier} } 非常感谢……