Machine learning to evaluate evolvability defects: Code metrics thresholds for a given context

Evolvability defects are non-understandable and non-modifiable states that do not directly produce runtime behavioral failures. Automatic source code evaluation by metrics and thresholds can help reduce the burden of a manual inspection. This study addresses two problems. (1) Evolvability defects are not usually managed in bug tracking systems. (2) Conventional methods cannot fully interpret the relations among the metrics in a given context (e.g., programming language, application domain). The key actions of our method are to (1) gather trainingdata for machine learning by experts' manual inspection of some of the files in given systems (benchmark) and (2) employ a classification-tree learner algorithm, C5.0, which can deal with non-orthogonal relations between metrics. Furthermore, we experimentally confirm that, even with less training-data, our method provides a more precise evaluation than four conventional methods (the percentile, Alves' method, Bender's method, and the ROC curve-based method).