A Nonparametric Adaptive Sampling Strategy for Online Monitoring of Big Data Streams

With the rapid advancement of sensor technology, a huge amount of data is generated in various applications, which poses new and unique challenges for Statistical Process Control (SPC). In this paper, we propose a Nonparametric Adaptive Sampling (NAS) strategy to online monitor non-normal big data streams in the context of limited resources, where only a subset of observations are available at each acquisition time. In particular, this proposed method integrates a rank-based CUSUM scheme and an innovative idea that corrects the anti-rank statistics with partial observations, which can effectively detect a wide range of possible mean shifts when data streams are exchangeable and follow arbitrary distributions. Two theoretical properties on the sampling layout of the proposed NAS algorithm are investigated when the process is in control and out of control. Both simulations and case studies are conducted under different scenarios to illustrate and evaluate the performance of the proposed method.