DNN-Based image classification for software GUI testing

Huijuan Lu, Li Wang, Minchao Ye, Ke Yan, Qun Jin

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    Since visual design of Graphical User Interface (GUI) is one of the indispensable parts of every software and to some extent, it determines whether the software is attractive. And with the increasing amount of GUI designs, it is usually impractical to deal with a variety of designs manually with limited staff. Therefore, a solution for that program is needed. This work designs and implements an automatic GUI testing system based on Deep Neural Network (DNN) and cloud service. The system includes screenshot capturing and uploading module, web server, DNN models, etc. It provides accurate and efficient GUI automatic testing technology.

    Original languageEnglish
    Title of host publicationProceedings - 2018 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018
    EditorsFrederic Loulergue, Guojun Wang, Md Zakirul Alam Bhuiyan, Xiaoxing Ma, Peng Li, Manuel Roveri, Qi Han, Lei Chen
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages1818-1823
    Number of pages6
    ISBN (Electronic)9781538693803
    DOIs
    Publication statusPublished - 2018 Dec 4
    Event4th IEEE SmartWorld, 15th IEEE International Conference on Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018 - Guangzhou, China
    Duration: 2018 Oct 72018 Oct 11

    Other

    Other4th IEEE SmartWorld, 15th IEEE International Conference on Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018
    CountryChina
    CityGuangzhou
    Period18/10/718/10/11

    Fingerprint

    Image classification
    Graphical user interfaces
    Testing
    Automatic testing
    Servers
    Deep neural networks
    Graphical user interface
    Software
    Neural networks

    Keywords

    • Cloud service
    • Deep neural network
    • GUI testing

    ASJC Scopus subject areas

    • Computer Networks and Communications
    • Computer Science Applications
    • Information Systems and Management
    • Artificial Intelligence

    Cite this

    Lu, H., Wang, L., Ye, M., Yan, K., & Jin, Q. (2018). DNN-Based image classification for software GUI testing. In F. Loulergue, G. Wang, M. Z. A. Bhuiyan, X. Ma, P. Li, M. Roveri, Q. Han, ... L. Chen (Eds.), Proceedings - 2018 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018 (pp. 1818-1823). [8560284] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SmartWorld.2018.00306

    DNN-Based image classification for software GUI testing. / Lu, Huijuan; Wang, Li; Ye, Minchao; Yan, Ke; Jin, Qun.

    Proceedings - 2018 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018. ed. / Frederic Loulergue; Guojun Wang; Md Zakirul Alam Bhuiyan; Xiaoxing Ma; Peng Li; Manuel Roveri; Qi Han; Lei Chen. Institute of Electrical and Electronics Engineers Inc., 2018. p. 1818-1823 8560284.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Lu, H, Wang, L, Ye, M, Yan, K & Jin, Q 2018, DNN-Based image classification for software GUI testing. in F Loulergue, G Wang, MZA Bhuiyan, X Ma, P Li, M Roveri, Q Han & L Chen (eds), Proceedings - 2018 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018., 8560284, Institute of Electrical and Electronics Engineers Inc., pp. 1818-1823, 4th IEEE SmartWorld, 15th IEEE International Conference on Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018, Guangzhou, China, 18/10/7. https://doi.org/10.1109/SmartWorld.2018.00306
    Lu H, Wang L, Ye M, Yan K, Jin Q. DNN-Based image classification for software GUI testing. In Loulergue F, Wang G, Bhuiyan MZA, Ma X, Li P, Roveri M, Han Q, Chen L, editors, Proceedings - 2018 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018. Institute of Electrical and Electronics Engineers Inc. 2018. p. 1818-1823. 8560284 https://doi.org/10.1109/SmartWorld.2018.00306
    Lu, Huijuan ; Wang, Li ; Ye, Minchao ; Yan, Ke ; Jin, Qun. / DNN-Based image classification for software GUI testing. Proceedings - 2018 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018. editor / Frederic Loulergue ; Guojun Wang ; Md Zakirul Alam Bhuiyan ; Xiaoxing Ma ; Peng Li ; Manuel Roveri ; Qi Han ; Lei Chen. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 1818-1823
    @inproceedings{985cb361592048b491833c457ea139c4,
    title = "DNN-Based image classification for software GUI testing",
    abstract = "Since visual design of Graphical User Interface (GUI) is one of the indispensable parts of every software and to some extent, it determines whether the software is attractive. And with the increasing amount of GUI designs, it is usually impractical to deal with a variety of designs manually with limited staff. Therefore, a solution for that program is needed. This work designs and implements an automatic GUI testing system based on Deep Neural Network (DNN) and cloud service. The system includes screenshot capturing and uploading module, web server, DNN models, etc. It provides accurate and efficient GUI automatic testing technology.",
    keywords = "Cloud service, Deep neural network, GUI testing",
    author = "Huijuan Lu and Li Wang and Minchao Ye and Ke Yan and Qun Jin",
    year = "2018",
    month = "12",
    day = "4",
    doi = "10.1109/SmartWorld.2018.00306",
    language = "English",
    pages = "1818--1823",
    editor = "Frederic Loulergue and Guojun Wang and Bhuiyan, {Md Zakirul Alam} and Xiaoxing Ma and Peng Li and Manuel Roveri and Qi Han and Lei Chen",
    booktitle = "Proceedings - 2018 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018",
    publisher = "Institute of Electrical and Electronics Engineers Inc.",

    }

    TY - GEN

    T1 - DNN-Based image classification for software GUI testing

    AU - Lu, Huijuan

    AU - Wang, Li

    AU - Ye, Minchao

    AU - Yan, Ke

    AU - Jin, Qun

    PY - 2018/12/4

    Y1 - 2018/12/4

    N2 - Since visual design of Graphical User Interface (GUI) is one of the indispensable parts of every software and to some extent, it determines whether the software is attractive. And with the increasing amount of GUI designs, it is usually impractical to deal with a variety of designs manually with limited staff. Therefore, a solution for that program is needed. This work designs and implements an automatic GUI testing system based on Deep Neural Network (DNN) and cloud service. The system includes screenshot capturing and uploading module, web server, DNN models, etc. It provides accurate and efficient GUI automatic testing technology.

    AB - Since visual design of Graphical User Interface (GUI) is one of the indispensable parts of every software and to some extent, it determines whether the software is attractive. And with the increasing amount of GUI designs, it is usually impractical to deal with a variety of designs manually with limited staff. Therefore, a solution for that program is needed. This work designs and implements an automatic GUI testing system based on Deep Neural Network (DNN) and cloud service. The system includes screenshot capturing and uploading module, web server, DNN models, etc. It provides accurate and efficient GUI automatic testing technology.

    KW - Cloud service

    KW - Deep neural network

    KW - GUI testing

    UR - http://www.scopus.com/inward/record.url?scp=85060304294&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85060304294&partnerID=8YFLogxK

    U2 - 10.1109/SmartWorld.2018.00306

    DO - 10.1109/SmartWorld.2018.00306

    M3 - Conference contribution

    SP - 1818

    EP - 1823

    BT - Proceedings - 2018 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovations, SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 2018

    A2 - Loulergue, Frederic

    A2 - Wang, Guojun

    A2 - Bhuiyan, Md Zakirul Alam

    A2 - Ma, Xiaoxing

    A2 - Li, Peng

    A2 - Roveri, Manuel

    A2 - Han, Qi

    A2 - Chen, Lei

    PB - Institute of Electrical and Electronics Engineers Inc.

    ER -