Locating mechanical switches using RGB-D sensor mounted on a disaster response robot

Takuya Kanda, Kazuya Miyakawa, Jeonghwang Hayashi, Jun Ohya, Hiroyuki Ogata, Kenji Hashimoto*, Xiao Sun, Takashi Matsuzawa, Hiroshi Naito, Atsuo Takanishi

*Corresponding author for this work

Research output: Contribution to journalConference articlepeer-review

1 Citation (Scopus)


To achieve one of the tasks required for disaster response robots, this paper proposes a method for locating 3D structured switches' points to be pressed by the robot in disaster sites using RGBD images acquired by Kinect sensor attached to our disaster response robot. Our method consists of the following five steps: 1)Obtain RGB and depth images using an RGB-D sensor. 2) Detect the bounding box of switch area from the RGB image using YOLOv3. 3)Generate 3D point cloud data of the target switch by combining the bounding box and the depth image.4)Detect the center position of the switch button from the RGB image in the bounding box using Convolutional Neural Network (CNN). 5)Estimate the center of the button's face in real space from the detection result in step 4) and the 3D point cloud data generated in step3) In the experiment, the proposed method is applied to two types of 3D structured switch boxes to evaluate the effectiveness. The results show that our proposed method can locate the switch button accurately enough for the robot operation.

Original languageEnglish
Article number016
JournalIS and T International Symposium on Electronic Imaging Science and Technology
Issue number6
Publication statusPublished - 2020 Jan 26
Event2020 Intelligent Robotics and Industrial Applications Using Computer Vision Conference, IRIACV 2020 - Burlingame, United States
Duration: 2020 Jan 262020 Jan 30

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Computer Science Applications
  • Human-Computer Interaction
  • Software
  • Electrical and Electronic Engineering
  • Atomic and Molecular Physics, and Optics


Dive into the research topics of 'Locating mechanical switches using RGB-D sensor mounted on a disaster response robot'. Together they form a unique fingerprint.

Cite this