Locating mechanical switches using RGB-D sensor mounted on a disaster response robot

Takuya Kanda, Kazuya Miyakawa, Jeonghwang Hayashi, Jun Ohya, Hiroyuki Ogata, Kenji Hashimoto*, Xiao Sun, Takashi Matsuzawa, Hiroshi Naito, Atsuo Takanishi


研究成果: Conference article査読

1 被引用数 (Scopus)


To achieve one of the tasks required for disaster response robots, this paper proposes a method for locating 3D structured switches' points to be pressed by the robot in disaster sites using RGBD images acquired by Kinect sensor attached to our disaster response robot. Our method consists of the following five steps: 1)Obtain RGB and depth images using an RGB-D sensor. 2) Detect the bounding box of switch area from the RGB image using YOLOv3. 3)Generate 3D point cloud data of the target switch by combining the bounding box and the depth image.4)Detect the center position of the switch button from the RGB image in the bounding box using Convolutional Neural Network (CNN). 5)Estimate the center of the button's face in real space from the detection result in step 4) and the 3D point cloud data generated in step3) In the experiment, the proposed method is applied to two types of 3D structured switch boxes to evaluate the effectiveness. The results show that our proposed method can locate the switch button accurately enough for the robot operation.

ジャーナルIS and T International Symposium on Electronic Imaging Science and Technology
出版ステータスPublished - 2020 1月 26
イベント2020 Intelligent Robotics and Industrial Applications Using Computer Vision Conference, IRIACV 2020 - Burlingame, United States
継続期間: 2020 1月 262020 1月 30

ASJC Scopus subject areas

  • コンピュータ グラフィックスおよびコンピュータ支援設計
  • コンピュータ サイエンスの応用
  • 人間とコンピュータの相互作用
  • ソフトウェア
  • 電子工学および電気工学
  • 原子分子物理学および光学


「Locating mechanical switches using RGB-D sensor mounted on a disaster response robot」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。