TY - JOUR
T1 - Locating mechanical switches using RGB-D sensor mounted on a disaster response robot
AU - Kanda, Takuya
AU - Miyakawa, Kazuya
AU - Hayashi, Jeonghwang
AU - Ohya, Jun
AU - Ogata, Hiroyuki
AU - Hashimoto, Kenji
AU - Sun, Xiao
AU - Matsuzawa, Takashi
AU - Naito, Hiroshi
AU - Takanishi, Atsuo
N1 - Funding Information:
Takashi Matsuzawa received M.S. degree from the Department of Modern Mechanical Engineering, the Graduate School of Creative Science and Engineering, Waseda University, Japan in 2017. Currently, he belongs to the Department of Integrative Bioscience and Biomedical Engineering, the Graduate School of the Advanced Science and Engineering, Waseda University as an Ph.D. candidate and Research Fellowship for Young Scientists of Japan Society for the Promotion of Science (JSPS). His research areas include legged robots, motion planning and locomotion on rough terrain.
Funding Information:
The authors acknowledge the support of Mr. Li Qi, Mr. Kota Umebayashi, Ms. Reina Yoshizaki of Waseda University and Mr. Keishi Nishikawa of Mitsubishi Electric Corporation.
Funding Information:
Kenji Hashimoto received B.E. and M.E. degrees in 2004 and 2006, respectively, and the Ph.D. degree from the Department of Integrative Bioscience and Biomedical Engineering in 2009, all from Waseda University, Japan. During the Ph.D. course, he was funded by the Japan Society for the Promotion Science (JSPS) as a Research Fellow. He was a Postdoctoral Researcher at the Laboratoire de Physiologie de la Perception et de l 'Action in UMR 7152 College de France-CNRS, France from 2012 to 2013. He was an Assistant Professor from 2015 to 2017, and an Associate Professor from 2017 to 2018 at the Waseda Institute for Advanced Study, Waseda University. From April 2018, he joined the faculty of the Department of Mechanical Engineering Informatics, Meiji University as an Associate Professor. His research interests include legged robots and humanoid robots. He is a member of IEEE, RSJ, JSME and Japan Council of IFToMM.
Publisher Copyright:
© 2020 Society for Imaging Science and Technology.
PY - 2020/1/26
Y1 - 2020/1/26
N2 - To achieve one of the tasks required for disaster response robots, this paper proposes a method for locating 3D structured switches' points to be pressed by the robot in disaster sites using RGBD images acquired by Kinect sensor attached to our disaster response robot. Our method consists of the following five steps: 1)Obtain RGB and depth images using an RGB-D sensor. 2) Detect the bounding box of switch area from the RGB image using YOLOv3. 3)Generate 3D point cloud data of the target switch by combining the bounding box and the depth image.4)Detect the center position of the switch button from the RGB image in the bounding box using Convolutional Neural Network (CNN). 5)Estimate the center of the button's face in real space from the detection result in step 4) and the 3D point cloud data generated in step3) In the experiment, the proposed method is applied to two types of 3D structured switch boxes to evaluate the effectiveness. The results show that our proposed method can locate the switch button accurately enough for the robot operation.
AB - To achieve one of the tasks required for disaster response robots, this paper proposes a method for locating 3D structured switches' points to be pressed by the robot in disaster sites using RGBD images acquired by Kinect sensor attached to our disaster response robot. Our method consists of the following five steps: 1)Obtain RGB and depth images using an RGB-D sensor. 2) Detect the bounding box of switch area from the RGB image using YOLOv3. 3)Generate 3D point cloud data of the target switch by combining the bounding box and the depth image.4)Detect the center position of the switch button from the RGB image in the bounding box using Convolutional Neural Network (CNN). 5)Estimate the center of the button's face in real space from the detection result in step 4) and the 3D point cloud data generated in step3) In the experiment, the proposed method is applied to two types of 3D structured switch boxes to evaluate the effectiveness. The results show that our proposed method can locate the switch button accurately enough for the robot operation.
UR - http://www.scopus.com/inward/record.url?scp=85101762641&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85101762641&partnerID=8YFLogxK
U2 - 10.2352/ISSN.2470-1173.2020.6.IRIACV-016
DO - 10.2352/ISSN.2470-1173.2020.6.IRIACV-016
M3 - Conference article
AN - SCOPUS:85101762641
SN - 2470-1173
VL - 2020
JO - IS and T International Symposium on Electronic Imaging Science and Technology
JF - IS and T International Symposium on Electronic Imaging Science and Technology
IS - 6
M1 - 016
T2 - 2020 Intelligent Robotics and Industrial Applications Using Computer Vision Conference, IRIACV 2020
Y2 - 26 January 2020 through 30 January 2020
ER -