Evaluation of anomaly detection based on sketch and PCA

Yoshiki Kanda, Kensuke Fukuda, Toshiharu Sugawara

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    25 Citations (Scopus)

    Abstract

    Using traffic random projections (sketches) and Principal Component Analysis (PCA) for Internet traffic anomaly detection has become popular topics in the anomaly detection fields, but few studies have been undertaken on the subjective and quantitative comparison of multiple methods using the data traces open to the community. In this paper, we propose a new method that combines sketches and PCA to detect and identify the source IP addresses associated with the traffic anomalies in the backbone traces measured at a single link. We compare the results with those of a method incorporating sketches and multi-resolution gamma modeling using the trans-Pacific link traces. The comparison indicates that each method has its own advantages and disadvantages. Our method is good at detecting worm activities with many packets, whereas the gamma method is good at detecting scan activities for peer hosts with only a few packets, but it reports many false positives for traces of worm outbreaks. Therefore, their use in combination would be effective. We also examined the impact of adaptive decision making on a parameter (the number of normal subspaces in PCA) on the basis of the cumulative proportion of each sketched traffic and conclude that it performs at a higher level than the previous method deciding only on one specific value of the parameter for every divided traffics.

    Original languageEnglish
    Title of host publicationGLOBECOM - IEEE Global Telecommunications Conference
    DOIs
    Publication statusPublished - 2010
    Event53rd IEEE Global Communications Conference, GLOBECOM 2010 - Miami, FL
    Duration: 2010 Dec 62010 Dec 10

    Other

    Other53rd IEEE Global Communications Conference, GLOBECOM 2010
    CityMiami, FL
    Period10/12/610/12/10

    Fingerprint

    Principal component analysis
    Decision making
    Internet

    ASJC Scopus subject areas

    • Electrical and Electronic Engineering

    Cite this

    Kanda, Y., Fukuda, K., & Sugawara, T. (2010). Evaluation of anomaly detection based on sketch and PCA. In GLOBECOM - IEEE Global Telecommunications Conference [5683878] https://doi.org/10.1109/GLOCOM.2010.5683878

    Evaluation of anomaly detection based on sketch and PCA. / Kanda, Yoshiki; Fukuda, Kensuke; Sugawara, Toshiharu.

    GLOBECOM - IEEE Global Telecommunications Conference. 2010. 5683878.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Kanda, Y, Fukuda, K & Sugawara, T 2010, Evaluation of anomaly detection based on sketch and PCA. in GLOBECOM - IEEE Global Telecommunications Conference., 5683878, 53rd IEEE Global Communications Conference, GLOBECOM 2010, Miami, FL, 10/12/6. https://doi.org/10.1109/GLOCOM.2010.5683878
    Kanda Y, Fukuda K, Sugawara T. Evaluation of anomaly detection based on sketch and PCA. In GLOBECOM - IEEE Global Telecommunications Conference. 2010. 5683878 https://doi.org/10.1109/GLOCOM.2010.5683878
    Kanda, Yoshiki ; Fukuda, Kensuke ; Sugawara, Toshiharu. / Evaluation of anomaly detection based on sketch and PCA. GLOBECOM - IEEE Global Telecommunications Conference. 2010.
    @inproceedings{22fe0477af854863a2600c434edcd6fb,
    title = "Evaluation of anomaly detection based on sketch and PCA",
    abstract = "Using traffic random projections (sketches) and Principal Component Analysis (PCA) for Internet traffic anomaly detection has become popular topics in the anomaly detection fields, but few studies have been undertaken on the subjective and quantitative comparison of multiple methods using the data traces open to the community. In this paper, we propose a new method that combines sketches and PCA to detect and identify the source IP addresses associated with the traffic anomalies in the backbone traces measured at a single link. We compare the results with those of a method incorporating sketches and multi-resolution gamma modeling using the trans-Pacific link traces. The comparison indicates that each method has its own advantages and disadvantages. Our method is good at detecting worm activities with many packets, whereas the gamma method is good at detecting scan activities for peer hosts with only a few packets, but it reports many false positives for traces of worm outbreaks. Therefore, their use in combination would be effective. We also examined the impact of adaptive decision making on a parameter (the number of normal subspaces in PCA) on the basis of the cumulative proportion of each sketched traffic and conclude that it performs at a higher level than the previous method deciding only on one specific value of the parameter for every divided traffics.",
    author = "Yoshiki Kanda and Kensuke Fukuda and Toshiharu Sugawara",
    year = "2010",
    doi = "10.1109/GLOCOM.2010.5683878",
    language = "English",
    isbn = "9781424456383",
    booktitle = "GLOBECOM - IEEE Global Telecommunications Conference",

    }

    TY - GEN

    T1 - Evaluation of anomaly detection based on sketch and PCA

    AU - Kanda, Yoshiki

    AU - Fukuda, Kensuke

    AU - Sugawara, Toshiharu

    PY - 2010

    Y1 - 2010

    N2 - Using traffic random projections (sketches) and Principal Component Analysis (PCA) for Internet traffic anomaly detection has become popular topics in the anomaly detection fields, but few studies have been undertaken on the subjective and quantitative comparison of multiple methods using the data traces open to the community. In this paper, we propose a new method that combines sketches and PCA to detect and identify the source IP addresses associated with the traffic anomalies in the backbone traces measured at a single link. We compare the results with those of a method incorporating sketches and multi-resolution gamma modeling using the trans-Pacific link traces. The comparison indicates that each method has its own advantages and disadvantages. Our method is good at detecting worm activities with many packets, whereas the gamma method is good at detecting scan activities for peer hosts with only a few packets, but it reports many false positives for traces of worm outbreaks. Therefore, their use in combination would be effective. We also examined the impact of adaptive decision making on a parameter (the number of normal subspaces in PCA) on the basis of the cumulative proportion of each sketched traffic and conclude that it performs at a higher level than the previous method deciding only on one specific value of the parameter for every divided traffics.

    AB - Using traffic random projections (sketches) and Principal Component Analysis (PCA) for Internet traffic anomaly detection has become popular topics in the anomaly detection fields, but few studies have been undertaken on the subjective and quantitative comparison of multiple methods using the data traces open to the community. In this paper, we propose a new method that combines sketches and PCA to detect and identify the source IP addresses associated with the traffic anomalies in the backbone traces measured at a single link. We compare the results with those of a method incorporating sketches and multi-resolution gamma modeling using the trans-Pacific link traces. The comparison indicates that each method has its own advantages and disadvantages. Our method is good at detecting worm activities with many packets, whereas the gamma method is good at detecting scan activities for peer hosts with only a few packets, but it reports many false positives for traces of worm outbreaks. Therefore, their use in combination would be effective. We also examined the impact of adaptive decision making on a parameter (the number of normal subspaces in PCA) on the basis of the cumulative proportion of each sketched traffic and conclude that it performs at a higher level than the previous method deciding only on one specific value of the parameter for every divided traffics.

    UR - http://www.scopus.com/inward/record.url?scp=79551647326&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=79551647326&partnerID=8YFLogxK

    U2 - 10.1109/GLOCOM.2010.5683878

    DO - 10.1109/GLOCOM.2010.5683878

    M3 - Conference contribution

    AN - SCOPUS:79551647326

    SN - 9781424456383

    BT - GLOBECOM - IEEE Global Telecommunications Conference

    ER -