UGR'16 Tensor Time-Series Dataset

This paper presents an online anomaly detection system capable of handling operational network traffic of large networks (such as an ISP). We also aim at an effective practical anomaly diagnosis to collect actionable intelligence enabling an automated response. To achieve these objectives, we use the following approach: (1) We model the network status as a stream of tensors where each cell models a time series in the network. (2) We detect anomalous tensors at time steps by using an unsupervised tensor representation learning model. (3) We produce actionable intelligence by diagnosis of anomaly detection results and by identifying the abnormal time series that are most likely the causes of each anomaly in the tensor, and (4) we further analyze the traffic corresponding to the anomalous time-series by an innovative method to extract and isolate the attack traffic. (5) We provide solutions for the challenges of streaming data anomaly detection such as large volume, high velocity, seasonality, and concept drift. We apply our approach to the complete test set of UGR data to show its practicality and effectiveness. Not only can we detect and isolate most of the labeled attack traffic, but we also identify many organic attack activities in the UGR data. We report our results on the complete UGR dataset that shows high detection and isolation rate for labelled attacks in the dataset. We also report some of the organic attacks detected (labeled as background in the dataset). Our analysis shows that the isolated background traffic represent interesting and potentially malicious behaviour and can provide invaluable insight for cyber-threat researchers.