Open code coverage framework: A framework for consistent, flexible and complete measurement of test coverage supporting multiple programming languages

Kazunori Sakamoto, Fuyuki Ishikawa, Hironori Washizaki, Yoshiaki Fukazawa

Research output: Contribution to journalArticle

1 Citation (Scopus)


Test coverage is an important indicator of whether software has been sufficiently tested. However, there are several problems with the existing measurement tools for test coverage, such as their cost of development and maintenance, inconsistency, and inflexibility in measurement. We propose a consistent and flexible measurement framework for test coverage that we call the Open Code Coverage Framework (OCCF). It supports multiple programming languages by extracting the commonalities from multiple programming languages using an abstract syntax tree to help in the development of the measurement tools for the test coverage of new programming languages. OCCF allows users to add programming language support independently of the test-coverage-criteria and also to add test-coverage-criteria support independently of programming languages in order to take consistent measurements in each programming language. Moreover, OCCF provides two methods for changin the measurement range and elements using XPath and adding user code in order to make more flexible measurements. We implemented a sample tool for C, Java, and Python using OCCF. OCCF can measure four test-coveragecriteria. We also confirmed that OCCF can support C#, Ruby, JavaScript, and Lua. Moreover, we reduced the lines of code (LOCs) required to implement measurement tools for test coverage by approximately 90% and the time to implement a new test-coverage-criterion by over 80% in an experiment that compared OCCF with the conventional non-framework-based tools.

Original languageEnglish
Pages (from-to)2418-2430
Number of pages13
JournalIEICE Transactions on Information and Systems
Issue number12
Publication statusPublished - 2011 Dec



  • Code coverage
  • Framework
  • Metrics
  • Software testing
  • Test coverage

ASJC Scopus subject areas

  • Software
  • Hardware and Architecture
  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering
  • Artificial Intelligence

Cite this