Automated exploratory data analysis method for large-scale complex data based on iterative data segmentation - dataSet

This paper proposes a novel automated exploratory data analysis system based on data segmentation and visualization. In terms of data segmentation, a method based on attribute frequency statistical feature ratio is introduced to address the issue of field sequence focusing in exploratory data analysis. Meanwhile, a data filter based on parallel coordinates is adopted to enable iterative exploratory data analysis oriented towards user reasoning and cognition, further enhancing the effectiveness of subset analysis. In terms of data visualization, this paper combines the length of chart encoding with a visual analysis scheme recommendation algorithm designed around data field schemes and aggregation methods, proposing a custom search strategy evaluation method and chart matching rules. In the Seg+Vis iterative exploration module, by implementing cluster visualization effects and analysis schemes based on field type search trees, it innovatively integrates automatic segmentation and automatic visualization functions to achieve cluster comparison portraits. It also incorporates an interactive visualization backtracking mechanism, designs multiple iterative exploration modes, builds a bridge between segmentation and visualization, and forms a closed-loop process for exploratory data analysis. Finally, to verify the effectiveness and feasibility of this system, public structured data was used for testing and validation. Data analysts with different levels of experience and visual professional graduate students collaborated to conduct collaborative discussions, evaluating the EDASegVis system from two aspects: automated data exploration management and visual analysis recommendation