Exploring the effectiveness of using temporal order information for the early-recognition of suture surgery's six steps based on video image analyses of surgeons' hand actions

Miwa Tsubota, Ye Li, Jun Ohya

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    To alleviate the recent shortage problem of nurses, the actualization of RSN (Robotic Scrub Nurse) that can autonomously judge the current step of the surgery and pass the surgical instruments needed for the next step to surgeons is desired. The authors developed a computer vision based algorithm that can early-recognize only two steps of suture surgery. Based on the past work, this paper explores the effectiveness of utilizing temporal order of the six steps in suture surgery for the early-recognition. Our early-recognition algorithm consists of two modules: start point detection and hand action early-recognition. Segments of the test video that start from each quasi-start point are compared with the training data, and their probabilities are calculated. According to the calculated probabilities, hand actions could be early-recognized. To improve the early-recognition accuracy, temporal order information could be useful. This paper checks confusions of three steps' early recognition results, and if necessary, early-recognizes again after eliminating the wrong result, while for the other three steps, temporal order information is not utilized. Experimental results show our early-recognition method that utilizes the temporal order information achieves better performances.

    Original languageEnglish
    Title of host publicationRO-MAN 2017 - 26th IEEE International Symposium on Robot and Human Interactive Communication
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages464-469
    Number of pages6
    Volume2017-January
    ISBN (Electronic)9781538635186
    DOIs
    Publication statusPublished - 2017 Dec 8
    Event26th IEEE International Symposium on Robot and Human Interactive Communication, RO-MAN 2017 - Lisbon, Portugal
    Duration: 2017 Aug 282017 Sep 1

    Other

    Other26th IEEE International Symposium on Robot and Human Interactive Communication, RO-MAN 2017
    CountryPortugal
    CityLisbon
    Period17/8/2817/9/1

    Fingerprint

    Surgery
    Computer vision
    Robotics
    Recognition Algorithm
    Shortage
    Computer Vision
    Module
    Necessary
    Experimental Results

    ASJC Scopus subject areas

    • Artificial Intelligence
    • Computer Networks and Communications
    • Human-Computer Interaction
    • Control and Optimization

    Cite this

    Tsubota, M., Li, Y., & Ohya, J. (2017). Exploring the effectiveness of using temporal order information for the early-recognition of suture surgery's six steps based on video image analyses of surgeons' hand actions. In RO-MAN 2017 - 26th IEEE International Symposium on Robot and Human Interactive Communication (Vol. 2017-January, pp. 464-469). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ROMAN.2017.8172343

    Exploring the effectiveness of using temporal order information for the early-recognition of suture surgery's six steps based on video image analyses of surgeons' hand actions. / Tsubota, Miwa; Li, Ye; Ohya, Jun.

    RO-MAN 2017 - 26th IEEE International Symposium on Robot and Human Interactive Communication. Vol. 2017-January Institute of Electrical and Electronics Engineers Inc., 2017. p. 464-469.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Tsubota, M, Li, Y & Ohya, J 2017, Exploring the effectiveness of using temporal order information for the early-recognition of suture surgery's six steps based on video image analyses of surgeons' hand actions. in RO-MAN 2017 - 26th IEEE International Symposium on Robot and Human Interactive Communication. vol. 2017-January, Institute of Electrical and Electronics Engineers Inc., pp. 464-469, 26th IEEE International Symposium on Robot and Human Interactive Communication, RO-MAN 2017, Lisbon, Portugal, 17/8/28. https://doi.org/10.1109/ROMAN.2017.8172343
    Tsubota M, Li Y, Ohya J. Exploring the effectiveness of using temporal order information for the early-recognition of suture surgery's six steps based on video image analyses of surgeons' hand actions. In RO-MAN 2017 - 26th IEEE International Symposium on Robot and Human Interactive Communication. Vol. 2017-January. Institute of Electrical and Electronics Engineers Inc. 2017. p. 464-469 https://doi.org/10.1109/ROMAN.2017.8172343
    Tsubota, Miwa ; Li, Ye ; Ohya, Jun. / Exploring the effectiveness of using temporal order information for the early-recognition of suture surgery's six steps based on video image analyses of surgeons' hand actions. RO-MAN 2017 - 26th IEEE International Symposium on Robot and Human Interactive Communication. Vol. 2017-January Institute of Electrical and Electronics Engineers Inc., 2017. pp. 464-469
    @inproceedings{4f95b4f8ad1a46058d390a9d7b611e31,
    title = "Exploring the effectiveness of using temporal order information for the early-recognition of suture surgery's six steps based on video image analyses of surgeons' hand actions",
    abstract = "To alleviate the recent shortage problem of nurses, the actualization of RSN (Robotic Scrub Nurse) that can autonomously judge the current step of the surgery and pass the surgical instruments needed for the next step to surgeons is desired. The authors developed a computer vision based algorithm that can early-recognize only two steps of suture surgery. Based on the past work, this paper explores the effectiveness of utilizing temporal order of the six steps in suture surgery for the early-recognition. Our early-recognition algorithm consists of two modules: start point detection and hand action early-recognition. Segments of the test video that start from each quasi-start point are compared with the training data, and their probabilities are calculated. According to the calculated probabilities, hand actions could be early-recognized. To improve the early-recognition accuracy, temporal order information could be useful. This paper checks confusions of three steps' early recognition results, and if necessary, early-recognizes again after eliminating the wrong result, while for the other three steps, temporal order information is not utilized. Experimental results show our early-recognition method that utilizes the temporal order information achieves better performances.",
    author = "Miwa Tsubota and Ye Li and Jun Ohya",
    year = "2017",
    month = "12",
    day = "8",
    doi = "10.1109/ROMAN.2017.8172343",
    language = "English",
    volume = "2017-January",
    pages = "464--469",
    booktitle = "RO-MAN 2017 - 26th IEEE International Symposium on Robot and Human Interactive Communication",
    publisher = "Institute of Electrical and Electronics Engineers Inc.",

    }

    TY - GEN

    T1 - Exploring the effectiveness of using temporal order information for the early-recognition of suture surgery's six steps based on video image analyses of surgeons' hand actions

    AU - Tsubota, Miwa

    AU - Li, Ye

    AU - Ohya, Jun

    PY - 2017/12/8

    Y1 - 2017/12/8

    N2 - To alleviate the recent shortage problem of nurses, the actualization of RSN (Robotic Scrub Nurse) that can autonomously judge the current step of the surgery and pass the surgical instruments needed for the next step to surgeons is desired. The authors developed a computer vision based algorithm that can early-recognize only two steps of suture surgery. Based on the past work, this paper explores the effectiveness of utilizing temporal order of the six steps in suture surgery for the early-recognition. Our early-recognition algorithm consists of two modules: start point detection and hand action early-recognition. Segments of the test video that start from each quasi-start point are compared with the training data, and their probabilities are calculated. According to the calculated probabilities, hand actions could be early-recognized. To improve the early-recognition accuracy, temporal order information could be useful. This paper checks confusions of three steps' early recognition results, and if necessary, early-recognizes again after eliminating the wrong result, while for the other three steps, temporal order information is not utilized. Experimental results show our early-recognition method that utilizes the temporal order information achieves better performances.

    AB - To alleviate the recent shortage problem of nurses, the actualization of RSN (Robotic Scrub Nurse) that can autonomously judge the current step of the surgery and pass the surgical instruments needed for the next step to surgeons is desired. The authors developed a computer vision based algorithm that can early-recognize only two steps of suture surgery. Based on the past work, this paper explores the effectiveness of utilizing temporal order of the six steps in suture surgery for the early-recognition. Our early-recognition algorithm consists of two modules: start point detection and hand action early-recognition. Segments of the test video that start from each quasi-start point are compared with the training data, and their probabilities are calculated. According to the calculated probabilities, hand actions could be early-recognized. To improve the early-recognition accuracy, temporal order information could be useful. This paper checks confusions of three steps' early recognition results, and if necessary, early-recognizes again after eliminating the wrong result, while for the other three steps, temporal order information is not utilized. Experimental results show our early-recognition method that utilizes the temporal order information achieves better performances.

    UR - http://www.scopus.com/inward/record.url?scp=85045746638&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85045746638&partnerID=8YFLogxK

    U2 - 10.1109/ROMAN.2017.8172343

    DO - 10.1109/ROMAN.2017.8172343

    M3 - Conference contribution

    VL - 2017-January

    SP - 464

    EP - 469

    BT - RO-MAN 2017 - 26th IEEE International Symposium on Robot and Human Interactive Communication

    PB - Institute of Electrical and Electronics Engineers Inc.

    ER -