Detection accuracy of network anomalies using sampled flow statistics

Ryoichi Kawahara, Keisuke Ishibashi, Tatsuya Mori, Noriaki Kamiyama, Shigeaki Harada, Haruhisa Hasegawa, Shoichiro Asano

Research output: Contribution to journalArticle

5 Citations (Scopus)

Abstract

We investigated the detection accuracy of network anomalies when using flow statistics obtained through packet sampling. Through a case study based on measurement data, we showed that network anomalies generating a large number of small flows, such as network scans or SYN flooding, become difficult to detect during packet sampling. We then developed an analytical model that enables us to quantitatively evaluate the effect of packet sampling and traffic conditions, such as anomalous traffic volume, on detection accuracy. We also investigated how the detection accuracy worsens when the packet sampling rate decreases. In addition, we show that, even with a low sampling rate, spatially partitioning monitored traffic into groups makes it possible to increase detection accuracy. We also developed a method of determining an appropriate number of partitioned groups, and we show its effectiveness.

Original languageEnglish
Pages (from-to)513-535
Number of pages23
JournalInternational Journal of Network Management
Volume21
Issue number6
DOIs
Publication statusPublished - 2011 Nov
Externally publishedYes

Fingerprint

Statistics
Sampling
Analytical models

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications

Cite this

Kawahara, R., Ishibashi, K., Mori, T., Kamiyama, N., Harada, S., Hasegawa, H., & Asano, S. (2011). Detection accuracy of network anomalies using sampled flow statistics. International Journal of Network Management, 21(6), 513-535. https://doi.org/10.1002/nem.777

Detection accuracy of network anomalies using sampled flow statistics. / Kawahara, Ryoichi; Ishibashi, Keisuke; Mori, Tatsuya; Kamiyama, Noriaki; Harada, Shigeaki; Hasegawa, Haruhisa; Asano, Shoichiro.

In: International Journal of Network Management, Vol. 21, No. 6, 11.2011, p. 513-535.

Research output: Contribution to journalArticle

Kawahara, R, Ishibashi, K, Mori, T, Kamiyama, N, Harada, S, Hasegawa, H & Asano, S 2011, 'Detection accuracy of network anomalies using sampled flow statistics', International Journal of Network Management, vol. 21, no. 6, pp. 513-535. https://doi.org/10.1002/nem.777
Kawahara, Ryoichi ; Ishibashi, Keisuke ; Mori, Tatsuya ; Kamiyama, Noriaki ; Harada, Shigeaki ; Hasegawa, Haruhisa ; Asano, Shoichiro. / Detection accuracy of network anomalies using sampled flow statistics. In: International Journal of Network Management. 2011 ; Vol. 21, No. 6. pp. 513-535.
@article{c34ef2587c7a4b458bed1ed4124c6b00,
title = "Detection accuracy of network anomalies using sampled flow statistics",
abstract = "We investigated the detection accuracy of network anomalies when using flow statistics obtained through packet sampling. Through a case study based on measurement data, we showed that network anomalies generating a large number of small flows, such as network scans or SYN flooding, become difficult to detect during packet sampling. We then developed an analytical model that enables us to quantitatively evaluate the effect of packet sampling and traffic conditions, such as anomalous traffic volume, on detection accuracy. We also investigated how the detection accuracy worsens when the packet sampling rate decreases. In addition, we show that, even with a low sampling rate, spatially partitioning monitored traffic into groups makes it possible to increase detection accuracy. We also developed a method of determining an appropriate number of partitioned groups, and we show its effectiveness.",
author = "Ryoichi Kawahara and Keisuke Ishibashi and Tatsuya Mori and Noriaki Kamiyama and Shigeaki Harada and Haruhisa Hasegawa and Shoichiro Asano",
year = "2011",
month = "11",
doi = "10.1002/nem.777",
language = "English",
volume = "21",
pages = "513--535",
journal = "International Journal of Network Management",
issn = "1055-7148",
publisher = "John Wiley and Sons Ltd",
number = "6",

}

TY - JOUR

T1 - Detection accuracy of network anomalies using sampled flow statistics

AU - Kawahara, Ryoichi

AU - Ishibashi, Keisuke

AU - Mori, Tatsuya

AU - Kamiyama, Noriaki

AU - Harada, Shigeaki

AU - Hasegawa, Haruhisa

AU - Asano, Shoichiro

PY - 2011/11

Y1 - 2011/11

N2 - We investigated the detection accuracy of network anomalies when using flow statistics obtained through packet sampling. Through a case study based on measurement data, we showed that network anomalies generating a large number of small flows, such as network scans or SYN flooding, become difficult to detect during packet sampling. We then developed an analytical model that enables us to quantitatively evaluate the effect of packet sampling and traffic conditions, such as anomalous traffic volume, on detection accuracy. We also investigated how the detection accuracy worsens when the packet sampling rate decreases. In addition, we show that, even with a low sampling rate, spatially partitioning monitored traffic into groups makes it possible to increase detection accuracy. We also developed a method of determining an appropriate number of partitioned groups, and we show its effectiveness.

AB - We investigated the detection accuracy of network anomalies when using flow statistics obtained through packet sampling. Through a case study based on measurement data, we showed that network anomalies generating a large number of small flows, such as network scans or SYN flooding, become difficult to detect during packet sampling. We then developed an analytical model that enables us to quantitatively evaluate the effect of packet sampling and traffic conditions, such as anomalous traffic volume, on detection accuracy. We also investigated how the detection accuracy worsens when the packet sampling rate decreases. In addition, we show that, even with a low sampling rate, spatially partitioning monitored traffic into groups makes it possible to increase detection accuracy. We also developed a method of determining an appropriate number of partitioned groups, and we show its effectiveness.

UR - http://www.scopus.com/inward/record.url?scp=81755162040&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=81755162040&partnerID=8YFLogxK

U2 - 10.1002/nem.777

DO - 10.1002/nem.777

M3 - Article

VL - 21

SP - 513

EP - 535

JO - International Journal of Network Management

JF - International Journal of Network Management

SN - 1055-7148

IS - 6

ER -