An image processing method for changing endoscope direction based on pupil movement

Yang Cao, Satoshi Miura, Quanquan Liu, Yo Kobayashi, Kazuya Kawamura, Shigeki Sugano, Masakatsu G. Fujie

    Research output: Contribution to journalArticle

    Abstract

    Increased attention has been focused on laparoscopic surgery because of its minimal invasiveness and improved cosmetic properties. However, the procedure of laparoscopic surgery is considerably difficult for surgeons, thus paving the way for the introduction of robotic technology to reduce the surgeon’s burden. Thus, we have developed a single-port surgery assistive robot with a master–slave structure that has two surgical manipulators and a sheath manipulator for the alteration of endoscope direction. During the development of the surgical robotic system, achieving intuitive operation is very important. In this paper, we propose a new laparoscope manipulator control system based on the movement of the pupils to enhance intuitive operability. We achieve this using a webcam and an image processing method. After the pupil movement data are obtained, the master computer transforms these data into an output signal, and then the slave computer receives and uses that signal to drive the robot. The details of the system and the pupil detection procedure are explained. The aim of the present experiment is to verify the effectiveness of the image processing method applied to the alteration of endoscope direction control system. For this purpose, we need to determine an appropriate pupil motion activation threshold to begin the sheath manipulator’s movement. We used four kinds of activation threshold, measuring the time cost of a particular operation: to move the image of the endoscope to a specific target position. Moreover, we identified an appropriate activation threshold that can be used to determine whether the endoscope is moving.

    Original languageEnglish
    Article number3
    JournalROBOMECH Journal
    Volume3
    Issue number1
    DOIs
    Publication statusPublished - 2016 Dec 1

    Fingerprint

    Endoscope
    endoscopes
    Endoscopy
    pupils
    Manipulator
    Manipulators
    image processing
    manipulators
    Image Processing
    Image processing
    surgery
    Surgery
    Activation
    surgeons
    Chemical activation
    activation
    robotics
    robots
    sheaths
    thresholds

    Keywords

    • Activation threshold
    • Double-screw-drive mechanism
    • Image processing
    • Master–slave structure
    • Non-rigid face tracking
    • Single-port endoscopic surgery

    ASJC Scopus subject areas

    • Artificial Intelligence
    • Mechanical Engineering
    • Control and Optimization
    • Modelling and Simulation
    • Instrumentation

    Cite this

    An image processing method for changing endoscope direction based on pupil movement. / Cao, Yang; Miura, Satoshi; Liu, Quanquan; Kobayashi, Yo; Kawamura, Kazuya; Sugano, Shigeki; Fujie, Masakatsu G.

    In: ROBOMECH Journal, Vol. 3, No. 1, 3, 01.12.2016.

    Research output: Contribution to journalArticle

    Cao, Yang ; Miura, Satoshi ; Liu, Quanquan ; Kobayashi, Yo ; Kawamura, Kazuya ; Sugano, Shigeki ; Fujie, Masakatsu G. / An image processing method for changing endoscope direction based on pupil movement. In: ROBOMECH Journal. 2016 ; Vol. 3, No. 1.
    @article{edd7aacf6f13431fa3ba2918593f1efc,
    title = "An image processing method for changing endoscope direction based on pupil movement",
    abstract = "Increased attention has been focused on laparoscopic surgery because of its minimal invasiveness and improved cosmetic properties. However, the procedure of laparoscopic surgery is considerably difficult for surgeons, thus paving the way for the introduction of robotic technology to reduce the surgeon’s burden. Thus, we have developed a single-port surgery assistive robot with a master–slave structure that has two surgical manipulators and a sheath manipulator for the alteration of endoscope direction. During the development of the surgical robotic system, achieving intuitive operation is very important. In this paper, we propose a new laparoscope manipulator control system based on the movement of the pupils to enhance intuitive operability. We achieve this using a webcam and an image processing method. After the pupil movement data are obtained, the master computer transforms these data into an output signal, and then the slave computer receives and uses that signal to drive the robot. The details of the system and the pupil detection procedure are explained. The aim of the present experiment is to verify the effectiveness of the image processing method applied to the alteration of endoscope direction control system. For this purpose, we need to determine an appropriate pupil motion activation threshold to begin the sheath manipulator’s movement. We used four kinds of activation threshold, measuring the time cost of a particular operation: to move the image of the endoscope to a specific target position. Moreover, we identified an appropriate activation threshold that can be used to determine whether the endoscope is moving.",
    keywords = "Activation threshold, Double-screw-drive mechanism, Image processing, Master–slave structure, Non-rigid face tracking, Single-port endoscopic surgery",
    author = "Yang Cao and Satoshi Miura and Quanquan Liu and Yo Kobayashi and Kazuya Kawamura and Shigeki Sugano and Fujie, {Masakatsu G.}",
    year = "2016",
    month = "12",
    day = "1",
    doi = "10.1186/s40648-016-0042-6",
    language = "English",
    volume = "3",
    journal = "ROBOMECH Journal",
    issn = "2197-4225",
    publisher = "Springer International Publishing AG",
    number = "1",

    }

    TY - JOUR

    T1 - An image processing method for changing endoscope direction based on pupil movement

    AU - Cao, Yang

    AU - Miura, Satoshi

    AU - Liu, Quanquan

    AU - Kobayashi, Yo

    AU - Kawamura, Kazuya

    AU - Sugano, Shigeki

    AU - Fujie, Masakatsu G.

    PY - 2016/12/1

    Y1 - 2016/12/1

    N2 - Increased attention has been focused on laparoscopic surgery because of its minimal invasiveness and improved cosmetic properties. However, the procedure of laparoscopic surgery is considerably difficult for surgeons, thus paving the way for the introduction of robotic technology to reduce the surgeon’s burden. Thus, we have developed a single-port surgery assistive robot with a master–slave structure that has two surgical manipulators and a sheath manipulator for the alteration of endoscope direction. During the development of the surgical robotic system, achieving intuitive operation is very important. In this paper, we propose a new laparoscope manipulator control system based on the movement of the pupils to enhance intuitive operability. We achieve this using a webcam and an image processing method. After the pupil movement data are obtained, the master computer transforms these data into an output signal, and then the slave computer receives and uses that signal to drive the robot. The details of the system and the pupil detection procedure are explained. The aim of the present experiment is to verify the effectiveness of the image processing method applied to the alteration of endoscope direction control system. For this purpose, we need to determine an appropriate pupil motion activation threshold to begin the sheath manipulator’s movement. We used four kinds of activation threshold, measuring the time cost of a particular operation: to move the image of the endoscope to a specific target position. Moreover, we identified an appropriate activation threshold that can be used to determine whether the endoscope is moving.

    AB - Increased attention has been focused on laparoscopic surgery because of its minimal invasiveness and improved cosmetic properties. However, the procedure of laparoscopic surgery is considerably difficult for surgeons, thus paving the way for the introduction of robotic technology to reduce the surgeon’s burden. Thus, we have developed a single-port surgery assistive robot with a master–slave structure that has two surgical manipulators and a sheath manipulator for the alteration of endoscope direction. During the development of the surgical robotic system, achieving intuitive operation is very important. In this paper, we propose a new laparoscope manipulator control system based on the movement of the pupils to enhance intuitive operability. We achieve this using a webcam and an image processing method. After the pupil movement data are obtained, the master computer transforms these data into an output signal, and then the slave computer receives and uses that signal to drive the robot. The details of the system and the pupil detection procedure are explained. The aim of the present experiment is to verify the effectiveness of the image processing method applied to the alteration of endoscope direction control system. For this purpose, we need to determine an appropriate pupil motion activation threshold to begin the sheath manipulator’s movement. We used four kinds of activation threshold, measuring the time cost of a particular operation: to move the image of the endoscope to a specific target position. Moreover, we identified an appropriate activation threshold that can be used to determine whether the endoscope is moving.

    KW - Activation threshold

    KW - Double-screw-drive mechanism

    KW - Image processing

    KW - Master–slave structure

    KW - Non-rigid face tracking

    KW - Single-port endoscopic surgery

    UR - http://www.scopus.com/inward/record.url?scp=85042654194&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85042654194&partnerID=8YFLogxK

    U2 - 10.1186/s40648-016-0042-6

    DO - 10.1186/s40648-016-0042-6

    M3 - Article

    AN - SCOPUS:85042654194

    VL - 3

    JO - ROBOMECH Journal

    JF - ROBOMECH Journal

    SN - 2197-4225

    IS - 1

    M1 - 3

    ER -