Network failure detection and diagnosis by analyzing Syslog and SNS data

Applying big data analysis to network operations

Tatsuaki Kimura, Kei Takeshita, Tsuyoshi Toyono, Masahiro Yokota, Ken Nishimatsu, Tatsuya Mori

    Research output: Contribution to journalArticle

    3 Citations (Scopus)

    Abstract

    We introduce two big data analysis methods for diagnosing the causes of network failures and for detecting network failures early Syslogs contain log data generated by the system. We analyzed syslogs and succeeded in detecting the cause of a network failure by automatically learning over 100 million logs without needing any previous knowledge of log data. Analysis of the data of a social networking service (namely, Twitter) enabled us to detect possible network failures by extracting network-failure related tweets, which account for less than 1% of all tweets, in real time and with high accuracy.

    Original languageEnglish
    JournalNTT Technical Review
    Volume11
    Issue number11
    Publication statusPublished - 2013 Nov

    Fingerprint

    Big data

    Keywords

    • Big data
    • Network failure detection
    • Syslog

    ASJC Scopus subject areas

    • Electrical and Electronic Engineering
    • Computer Networks and Communications
    • Computer Science Applications

    Cite this

    Network failure detection and diagnosis by analyzing Syslog and SNS data : Applying big data analysis to network operations. / Kimura, Tatsuaki; Takeshita, Kei; Toyono, Tsuyoshi; Yokota, Masahiro; Nishimatsu, Ken; Mori, Tatsuya.

    In: NTT Technical Review, Vol. 11, No. 11, 11.2013.

    Research output: Contribution to journalArticle

    Kimura, Tatsuaki ; Takeshita, Kei ; Toyono, Tsuyoshi ; Yokota, Masahiro ; Nishimatsu, Ken ; Mori, Tatsuya. / Network failure detection and diagnosis by analyzing Syslog and SNS data : Applying big data analysis to network operations. In: NTT Technical Review. 2013 ; Vol. 11, No. 11.
    @article{9c7e727062f3453a9fb045ba989096ef,
    title = "Network failure detection and diagnosis by analyzing Syslog and SNS data: Applying big data analysis to network operations",
    abstract = "We introduce two big data analysis methods for diagnosing the causes of network failures and for detecting network failures early Syslogs contain log data generated by the system. We analyzed syslogs and succeeded in detecting the cause of a network failure by automatically learning over 100 million logs without needing any previous knowledge of log data. Analysis of the data of a social networking service (namely, Twitter) enabled us to detect possible network failures by extracting network-failure related tweets, which account for less than 1{\%} of all tweets, in real time and with high accuracy.",
    keywords = "Big data, Network failure detection, Syslog",
    author = "Tatsuaki Kimura and Kei Takeshita and Tsuyoshi Toyono and Masahiro Yokota and Ken Nishimatsu and Tatsuya Mori",
    year = "2013",
    month = "11",
    language = "English",
    volume = "11",
    journal = "NTT Technical Review",
    issn = "1348-3447",
    publisher = "Nippon Telegraph and Telephone Corp.",
    number = "11",

    }

    TY - JOUR

    T1 - Network failure detection and diagnosis by analyzing Syslog and SNS data

    T2 - Applying big data analysis to network operations

    AU - Kimura, Tatsuaki

    AU - Takeshita, Kei

    AU - Toyono, Tsuyoshi

    AU - Yokota, Masahiro

    AU - Nishimatsu, Ken

    AU - Mori, Tatsuya

    PY - 2013/11

    Y1 - 2013/11

    N2 - We introduce two big data analysis methods for diagnosing the causes of network failures and for detecting network failures early Syslogs contain log data generated by the system. We analyzed syslogs and succeeded in detecting the cause of a network failure by automatically learning over 100 million logs without needing any previous knowledge of log data. Analysis of the data of a social networking service (namely, Twitter) enabled us to detect possible network failures by extracting network-failure related tweets, which account for less than 1% of all tweets, in real time and with high accuracy.

    AB - We introduce two big data analysis methods for diagnosing the causes of network failures and for detecting network failures early Syslogs contain log data generated by the system. We analyzed syslogs and succeeded in detecting the cause of a network failure by automatically learning over 100 million logs without needing any previous knowledge of log data. Analysis of the data of a social networking service (namely, Twitter) enabled us to detect possible network failures by extracting network-failure related tweets, which account for less than 1% of all tweets, in real time and with high accuracy.

    KW - Big data

    KW - Network failure detection

    KW - Syslog

    UR - http://www.scopus.com/inward/record.url?scp=84888622600&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=84888622600&partnerID=8YFLogxK

    M3 - Article

    VL - 11

    JO - NTT Technical Review

    JF - NTT Technical Review

    SN - 1348-3447

    IS - 11

    ER -