Reduce test cost by reusing test oracles through combinatorial join

Hiroshi Ukai, Xiao Qu, Hironori Washizaki, Yoshiaki Fukazawa

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Methods to generate combinatorial test suites have been extensively studied in the combinatorial interaction testing (CIT) community, but the creation of test oracles for the test cases remains a challenging and expensive task because they are created manually. In this paper, we propose a novel technique to 'join' multiple test suites into one while satisfying the required combinatorial coverage of the system under test. This technique allows test oracles designed and created in earlier testing phases to be reused in later ones. Our study results indicate that the technique can reduce the total test cost to design and execute a test suite by more than 55% compared to the conventional testing approach.

Original languageEnglish
Title of host publicationProceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages260-263
Number of pages4
ISBN (Electronic)9781728108889
DOIs
Publication statusPublished - 2019 Apr 1
Event12th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019 - Xi'an, China
Duration: 2019 Apr 222019 Apr 27

Publication series

NameProceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019

Conference

Conference12th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019
CountryChina
CityXi'an
Period19/4/2219/4/27

Fingerprint

Testing
Costs

Keywords

  • Combinatorial interaction testing
  • Functional testing
  • Test oracle

ASJC Scopus subject areas

  • Software
  • Safety, Risk, Reliability and Quality

Cite this

Ukai, H., Qu, X., Washizaki, H., & Fukazawa, Y. (2019). Reduce test cost by reusing test oracles through combinatorial join. In Proceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019 (pp. 260-263). [8728961] (Proceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICSTW.2019.00061

Reduce test cost by reusing test oracles through combinatorial join. / Ukai, Hiroshi; Qu, Xiao; Washizaki, Hironori; Fukazawa, Yoshiaki.

Proceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 260-263 8728961 (Proceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Ukai, H, Qu, X, Washizaki, H & Fukazawa, Y 2019, Reduce test cost by reusing test oracles through combinatorial join. in Proceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019., 8728961, Proceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019, Institute of Electrical and Electronics Engineers Inc., pp. 260-263, 12th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019, Xi'an, China, 19/4/22. https://doi.org/10.1109/ICSTW.2019.00061
Ukai H, Qu X, Washizaki H, Fukazawa Y. Reduce test cost by reusing test oracles through combinatorial join. In Proceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 260-263. 8728961. (Proceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019). https://doi.org/10.1109/ICSTW.2019.00061
Ukai, Hiroshi ; Qu, Xiao ; Washizaki, Hironori ; Fukazawa, Yoshiaki. / Reduce test cost by reusing test oracles through combinatorial join. Proceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 260-263 (Proceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019).
@inproceedings{81b3a0236ee743cba425d2737f0b2676,
title = "Reduce test cost by reusing test oracles through combinatorial join",
abstract = "Methods to generate combinatorial test suites have been extensively studied in the combinatorial interaction testing (CIT) community, but the creation of test oracles for the test cases remains a challenging and expensive task because they are created manually. In this paper, we propose a novel technique to 'join' multiple test suites into one while satisfying the required combinatorial coverage of the system under test. This technique allows test oracles designed and created in earlier testing phases to be reused in later ones. Our study results indicate that the technique can reduce the total test cost to design and execute a test suite by more than 55{\%} compared to the conventional testing approach.",
keywords = "Combinatorial interaction testing, Functional testing, Test oracle",
author = "Hiroshi Ukai and Xiao Qu and Hironori Washizaki and Yoshiaki Fukazawa",
year = "2019",
month = "4",
day = "1",
doi = "10.1109/ICSTW.2019.00061",
language = "English",
series = "Proceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "260--263",
booktitle = "Proceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019",

}

TY - GEN

T1 - Reduce test cost by reusing test oracles through combinatorial join

AU - Ukai, Hiroshi

AU - Qu, Xiao

AU - Washizaki, Hironori

AU - Fukazawa, Yoshiaki

PY - 2019/4/1

Y1 - 2019/4/1

N2 - Methods to generate combinatorial test suites have been extensively studied in the combinatorial interaction testing (CIT) community, but the creation of test oracles for the test cases remains a challenging and expensive task because they are created manually. In this paper, we propose a novel technique to 'join' multiple test suites into one while satisfying the required combinatorial coverage of the system under test. This technique allows test oracles designed and created in earlier testing phases to be reused in later ones. Our study results indicate that the technique can reduce the total test cost to design and execute a test suite by more than 55% compared to the conventional testing approach.

AB - Methods to generate combinatorial test suites have been extensively studied in the combinatorial interaction testing (CIT) community, but the creation of test oracles for the test cases remains a challenging and expensive task because they are created manually. In this paper, we propose a novel technique to 'join' multiple test suites into one while satisfying the required combinatorial coverage of the system under test. This technique allows test oracles designed and created in earlier testing phases to be reused in later ones. Our study results indicate that the technique can reduce the total test cost to design and execute a test suite by more than 55% compared to the conventional testing approach.

KW - Combinatorial interaction testing

KW - Functional testing

KW - Test oracle

UR - http://www.scopus.com/inward/record.url?scp=85068405073&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85068405073&partnerID=8YFLogxK

U2 - 10.1109/ICSTW.2019.00061

DO - 10.1109/ICSTW.2019.00061

M3 - Conference contribution

AN - SCOPUS:85068405073

T3 - Proceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019

SP - 260

EP - 263

BT - Proceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -