An image processing method for changing endoscope direction based on pupil movement

Yang Cao*, Satoshi Miura, Quanquan Liu, Yo Kobayashi, Kazuya Kawamura, Shigeki Sugano, Masakatsu G. Fujie

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)


Increased attention has been focused on laparoscopic surgery because of its minimal invasiveness and improved cosmetic properties. However, the procedure of laparoscopic surgery is considerably difficult for surgeons, thus paving the way for the introduction of robotic technology to reduce the surgeon’s burden. Thus, we have developed a single-port surgery assistive robot with a master–slave structure that has two surgical manipulators and a sheath manipulator for the alteration of endoscope direction. During the development of the surgical robotic system, achieving intuitive operation is very important. In this paper, we propose a new laparoscope manipulator control system based on the movement of the pupils to enhance intuitive operability. We achieve this using a webcam and an image processing method. After the pupil movement data are obtained, the master computer transforms these data into an output signal, and then the slave computer receives and uses that signal to drive the robot. The details of the system and the pupil detection procedure are explained. The aim of the present experiment is to verify the effectiveness of the image processing method applied to the alteration of endoscope direction control system. For this purpose, we need to determine an appropriate pupil motion activation threshold to begin the sheath manipulator’s movement. We used four kinds of activation threshold, measuring the time cost of a particular operation: to move the image of the endoscope to a specific target position. Moreover, we identified an appropriate activation threshold that can be used to determine whether the endoscope is moving.

Original languageEnglish
Article number3
JournalROBOMECH Journal
Issue number1
Publication statusPublished - 2016 Dec 1


  • Activation threshold
  • Double-screw-drive mechanism
  • Image processing
  • Master–slave structure
  • Non-rigid face tracking
  • Single-port endoscopic surgery

ASJC Scopus subject areas

  • Modelling and Simulation
  • Instrumentation
  • Mechanical Engineering
  • Control and Optimization
  • Artificial Intelligence


Dive into the research topics of 'An image processing method for changing endoscope direction based on pupil movement'. Together they form a unique fingerprint.

Cite this