Machine learning to evaluate evolvability defects: Code metrics thresholds for a given context

Naohiko Tsuda, Hironori Washizaki, Yoshiaki Fukazawa, Yuichiro Yasuda, Shunsuke Sugimura

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    Evolvability defects are non-understandable and non-modifiable states that do not directly produce runtime behavioral failures. Automatic source code evaluation by metrics and thresholds can help reduce the burden of a manual inspection. This study addresses two problems. (1) Evolvability defects are not usually managed in bug tracking systems. (2) Conventional methods cannot fully interpret the relations among the metrics in a given context (e.g., programming language, application domain). The key actions of our method are to (1) gather trainingdata for machine learning by experts' manual inspection of some of the files in given systems (benchmark) and (2) employ a classification-tree learner algorithm, C5.0, which can deal with non-orthogonal relations between metrics. Furthermore, we experimentally confirm that, even with less training-data, our method provides a more precise evaluation than four conventional methods (the percentile, Alves' method, Bender's method, and the ROC curve-based method).

    Original languageEnglish
    Title of host publicationProceedings - 2018 IEEE 18th International Conference on Software Quality, Reliability, and Security, QRS 2018
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages83-94
    Number of pages12
    ISBN (Print)9781538677575
    DOIs
    Publication statusPublished - 2018 Aug 2
    Event18th IEEE International Conference on Software Quality, Reliability, and Security, QRS 2018 - Lisbon, Portugal
    Duration: 2018 Jul 162018 Jul 20

    Other

    Other18th IEEE International Conference on Software Quality, Reliability, and Security, QRS 2018
    CountryPortugal
    CityLisbon
    Period18/7/1618/7/20

    Fingerprint

    Learning systems
    Inspection
    Defects
    Computer programming languages

    Keywords

    • Classification-tree
    • Contextual thresholds
    • Evolvability defects
    • Goal-Question-Metrics (GQM)
    • Machine learning
    • Software metrics

    ASJC Scopus subject areas

    • Software
    • Safety, Risk, Reliability and Quality

    Cite this

    Tsuda, N., Washizaki, H., Fukazawa, Y., Yasuda, Y., & Sugimura, S. (2018). Machine learning to evaluate evolvability defects: Code metrics thresholds for a given context. In Proceedings - 2018 IEEE 18th International Conference on Software Quality, Reliability, and Security, QRS 2018 (pp. 83-94). [8424960] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/QRS.2018.00022

    Machine learning to evaluate evolvability defects : Code metrics thresholds for a given context. / Tsuda, Naohiko; Washizaki, Hironori; Fukazawa, Yoshiaki; Yasuda, Yuichiro; Sugimura, Shunsuke.

    Proceedings - 2018 IEEE 18th International Conference on Software Quality, Reliability, and Security, QRS 2018. Institute of Electrical and Electronics Engineers Inc., 2018. p. 83-94 8424960.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Tsuda, N, Washizaki, H, Fukazawa, Y, Yasuda, Y & Sugimura, S 2018, Machine learning to evaluate evolvability defects: Code metrics thresholds for a given context. in Proceedings - 2018 IEEE 18th International Conference on Software Quality, Reliability, and Security, QRS 2018., 8424960, Institute of Electrical and Electronics Engineers Inc., pp. 83-94, 18th IEEE International Conference on Software Quality, Reliability, and Security, QRS 2018, Lisbon, Portugal, 18/7/16. https://doi.org/10.1109/QRS.2018.00022
    Tsuda N, Washizaki H, Fukazawa Y, Yasuda Y, Sugimura S. Machine learning to evaluate evolvability defects: Code metrics thresholds for a given context. In Proceedings - 2018 IEEE 18th International Conference on Software Quality, Reliability, and Security, QRS 2018. Institute of Electrical and Electronics Engineers Inc. 2018. p. 83-94. 8424960 https://doi.org/10.1109/QRS.2018.00022
    Tsuda, Naohiko ; Washizaki, Hironori ; Fukazawa, Yoshiaki ; Yasuda, Yuichiro ; Sugimura, Shunsuke. / Machine learning to evaluate evolvability defects : Code metrics thresholds for a given context. Proceedings - 2018 IEEE 18th International Conference on Software Quality, Reliability, and Security, QRS 2018. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 83-94
    @inproceedings{e78c40c7238c4045819bb850fa0ea9f3,
    title = "Machine learning to evaluate evolvability defects: Code metrics thresholds for a given context",
    abstract = "Evolvability defects are non-understandable and non-modifiable states that do not directly produce runtime behavioral failures. Automatic source code evaluation by metrics and thresholds can help reduce the burden of a manual inspection. This study addresses two problems. (1) Evolvability defects are not usually managed in bug tracking systems. (2) Conventional methods cannot fully interpret the relations among the metrics in a given context (e.g., programming language, application domain). The key actions of our method are to (1) gather trainingdata for machine learning by experts' manual inspection of some of the files in given systems (benchmark) and (2) employ a classification-tree learner algorithm, C5.0, which can deal with non-orthogonal relations between metrics. Furthermore, we experimentally confirm that, even with less training-data, our method provides a more precise evaluation than four conventional methods (the percentile, Alves' method, Bender's method, and the ROC curve-based method).",
    keywords = "Classification-tree, Contextual thresholds, Evolvability defects, Goal-Question-Metrics (GQM), Machine learning, Software metrics",
    author = "Naohiko Tsuda and Hironori Washizaki and Yoshiaki Fukazawa and Yuichiro Yasuda and Shunsuke Sugimura",
    year = "2018",
    month = "8",
    day = "2",
    doi = "10.1109/QRS.2018.00022",
    language = "English",
    isbn = "9781538677575",
    pages = "83--94",
    booktitle = "Proceedings - 2018 IEEE 18th International Conference on Software Quality, Reliability, and Security, QRS 2018",
    publisher = "Institute of Electrical and Electronics Engineers Inc.",

    }

    TY - GEN

    T1 - Machine learning to evaluate evolvability defects

    T2 - Code metrics thresholds for a given context

    AU - Tsuda, Naohiko

    AU - Washizaki, Hironori

    AU - Fukazawa, Yoshiaki

    AU - Yasuda, Yuichiro

    AU - Sugimura, Shunsuke

    PY - 2018/8/2

    Y1 - 2018/8/2

    N2 - Evolvability defects are non-understandable and non-modifiable states that do not directly produce runtime behavioral failures. Automatic source code evaluation by metrics and thresholds can help reduce the burden of a manual inspection. This study addresses two problems. (1) Evolvability defects are not usually managed in bug tracking systems. (2) Conventional methods cannot fully interpret the relations among the metrics in a given context (e.g., programming language, application domain). The key actions of our method are to (1) gather trainingdata for machine learning by experts' manual inspection of some of the files in given systems (benchmark) and (2) employ a classification-tree learner algorithm, C5.0, which can deal with non-orthogonal relations between metrics. Furthermore, we experimentally confirm that, even with less training-data, our method provides a more precise evaluation than four conventional methods (the percentile, Alves' method, Bender's method, and the ROC curve-based method).

    AB - Evolvability defects are non-understandable and non-modifiable states that do not directly produce runtime behavioral failures. Automatic source code evaluation by metrics and thresholds can help reduce the burden of a manual inspection. This study addresses two problems. (1) Evolvability defects are not usually managed in bug tracking systems. (2) Conventional methods cannot fully interpret the relations among the metrics in a given context (e.g., programming language, application domain). The key actions of our method are to (1) gather trainingdata for machine learning by experts' manual inspection of some of the files in given systems (benchmark) and (2) employ a classification-tree learner algorithm, C5.0, which can deal with non-orthogonal relations between metrics. Furthermore, we experimentally confirm that, even with less training-data, our method provides a more precise evaluation than four conventional methods (the percentile, Alves' method, Bender's method, and the ROC curve-based method).

    KW - Classification-tree

    KW - Contextual thresholds

    KW - Evolvability defects

    KW - Goal-Question-Metrics (GQM)

    KW - Machine learning

    KW - Software metrics

    UR - http://www.scopus.com/inward/record.url?scp=85052334218&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85052334218&partnerID=8YFLogxK

    U2 - 10.1109/QRS.2018.00022

    DO - 10.1109/QRS.2018.00022

    M3 - Conference contribution

    AN - SCOPUS:85052334218

    SN - 9781538677575

    SP - 83

    EP - 94

    BT - Proceedings - 2018 IEEE 18th International Conference on Software Quality, Reliability, and Security, QRS 2018

    PB - Institute of Electrical and Electronics Engineers Inc.

    ER -