TY - JOUR
T1 - Network Volume Anomaly Detection and Identification in Large-Scale Networks Based on Online Time-Structured Traffic Tensor Tracking
AU - Kasai, Hiroyuki
AU - Kellerer, Wolfgang
AU - Kleinsteuber, Martin
N1 - Funding Information:
Part of this work is part of a project that has received funding from the European Research Council (ERC) under the European Unions Horizon 2020 research and innovation program (grant agreement No 647158-FlexNets).
Publisher Copyright:
© 2016 IEEE.
PY - 2016/9
Y1 - 2016/9
N2 - This paper addresses network anomography, that is, the problem of inferring network-level anomalies from indirect link measurements. This problem is cast as a low-rank subspace tracking problem for normal flows under incomplete observations and an outlier detection problem for abnormal flows. Since traffic data is large-scale time-structured data accompanied with noise and outliers under partial observations, an efficient modeling method is essential. To this end, this paper proposes an online subspace tracking of a Hankelized time-structured traffic tensor for normal flows based on the Candecomp/PARAFAC decomposition exploiting the recursive least squares algorithm. We estimate abnormal flows as outlier sparse flows via sparsity maximization in the underlying under-constrained linear-inverse problem. A major advantage is that our algorithm estimates normal flows by low-dimensional matrices with time-directional features as well as the spatial correlation of multiple links without using the past observed measurements and the past model parameters. Extensive numerical evaluations show that the proposed algorithm achieves faster convergence per iteration of model approximation and better volume anomaly detection performance compared to state-of-the-art algorithms.
AB - This paper addresses network anomography, that is, the problem of inferring network-level anomalies from indirect link measurements. This problem is cast as a low-rank subspace tracking problem for normal flows under incomplete observations and an outlier detection problem for abnormal flows. Since traffic data is large-scale time-structured data accompanied with noise and outliers under partial observations, an efficient modeling method is essential. To this end, this paper proposes an online subspace tracking of a Hankelized time-structured traffic tensor for normal flows based on the Candecomp/PARAFAC decomposition exploiting the recursive least squares algorithm. We estimate abnormal flows as outlier sparse flows via sparsity maximization in the underlying under-constrained linear-inverse problem. A major advantage is that our algorithm estimates normal flows by low-dimensional matrices with time-directional features as well as the spatial correlation of multiple links without using the past observed measurements and the past model parameters. Extensive numerical evaluations show that the proposed algorithm achieves faster convergence per iteration of model approximation and better volume anomaly detection performance compared to state-of-the-art algorithms.
KW - Outlier detection
KW - Traffic volume anomaly
KW - anomaly detection and identification
KW - flow matrix estimation
KW - online subspace tracking
KW - traffic tensor tracking
UR - http://www.scopus.com/inward/record.url?scp=84991474964&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84991474964&partnerID=8YFLogxK
U2 - 10.1109/TNSM.2016.2598788
DO - 10.1109/TNSM.2016.2598788
M3 - Article
AN - SCOPUS:84991474964
VL - 13
SP - 636
EP - 650
JO - IEEE Transactions on Network and Service Management
JF - IEEE Transactions on Network and Service Management
SN - 1932-4537
IS - 3
M1 - 7536642
ER -