Predicting object dynamics from visual images through active sensing experiences

Shun Nishide, Tetsuya Ogata, Jun Tani, Kazunori Komatani, Hiroshi G. Okuno

Research output: Contribution to journalArticle

17 Citations (Scopus)

Abstract

Prediction of dynamic features is an important task for determining the manipulation strategies of an object. This paper presents a technique for predicting dynamics of objects relative to the robot's motion from visual images. During the training phase, the authors use the recurrent neural network with parametric bias (RNNPB) to self-organize the dynamics of objects manipulated by the robot into the PB space. The acquired PB values, static images of objects and robot motor values are input into a hierarchical neural network to link the images to dynamic features (PB values). The neural network extracts prominent features that each induce object dynamics. For prediction of the motion sequence of an unknown object, the static image of the object and robot motor value are input into the neural network to calculate the PB values. By inputting the PB values into the closed loop RNNPB, the predicted movements of the object relative to the robot motion are calculated recursively. Experiments were conducted with the humanoid robot Robovie-IIs pushing objects at different heights. The results of the experiment predicting the dynamics of target objects proved that the technique is efficient for predicting the dynamics of the objects.

Original languageEnglish
Pages (from-to)527-546
Number of pages20
JournalAdvanced Robotics
Volume22
Issue number5
DOIs
Publication statusPublished - 2008 Apr 1
Externally publishedYes

Fingerprint

Robots
Recurrent neural networks
Neural networks
Experiments

Keywords

  • Active sensing
  • Dynamics
  • Humanoid robot
  • Neural networks
  • Object manipulation

ASJC Scopus subject areas

  • Control and Systems Engineering

Cite this

Predicting object dynamics from visual images through active sensing experiences. / Nishide, Shun; Ogata, Tetsuya; Tani, Jun; Komatani, Kazunori; Okuno, Hiroshi G.

In: Advanced Robotics, Vol. 22, No. 5, 01.04.2008, p. 527-546.

Research output: Contribution to journalArticle

Nishide, Shun ; Ogata, Tetsuya ; Tani, Jun ; Komatani, Kazunori ; Okuno, Hiroshi G. / Predicting object dynamics from visual images through active sensing experiences. In: Advanced Robotics. 2008 ; Vol. 22, No. 5. pp. 527-546.
@article{7e66f5166ccf4b0bbce304fc46ea0fa7,
title = "Predicting object dynamics from visual images through active sensing experiences",
abstract = "Prediction of dynamic features is an important task for determining the manipulation strategies of an object. This paper presents a technique for predicting dynamics of objects relative to the robot's motion from visual images. During the training phase, the authors use the recurrent neural network with parametric bias (RNNPB) to self-organize the dynamics of objects manipulated by the robot into the PB space. The acquired PB values, static images of objects and robot motor values are input into a hierarchical neural network to link the images to dynamic features (PB values). The neural network extracts prominent features that each induce object dynamics. For prediction of the motion sequence of an unknown object, the static image of the object and robot motor value are input into the neural network to calculate the PB values. By inputting the PB values into the closed loop RNNPB, the predicted movements of the object relative to the robot motion are calculated recursively. Experiments were conducted with the humanoid robot Robovie-IIs pushing objects at different heights. The results of the experiment predicting the dynamics of target objects proved that the technique is efficient for predicting the dynamics of the objects.",
keywords = "Active sensing, Dynamics, Humanoid robot, Neural networks, Object manipulation",
author = "Shun Nishide and Tetsuya Ogata and Jun Tani and Kazunori Komatani and Okuno, {Hiroshi G.}",
year = "2008",
month = "4",
day = "1",
doi = "10.1163/156855308X294879",
language = "English",
volume = "22",
pages = "527--546",
journal = "Advanced Robotics",
issn = "0169-1864",
publisher = "Taylor and Francis Ltd.",
number = "5",

}

TY - JOUR

T1 - Predicting object dynamics from visual images through active sensing experiences

AU - Nishide, Shun

AU - Ogata, Tetsuya

AU - Tani, Jun

AU - Komatani, Kazunori

AU - Okuno, Hiroshi G.

PY - 2008/4/1

Y1 - 2008/4/1

N2 - Prediction of dynamic features is an important task for determining the manipulation strategies of an object. This paper presents a technique for predicting dynamics of objects relative to the robot's motion from visual images. During the training phase, the authors use the recurrent neural network with parametric bias (RNNPB) to self-organize the dynamics of objects manipulated by the robot into the PB space. The acquired PB values, static images of objects and robot motor values are input into a hierarchical neural network to link the images to dynamic features (PB values). The neural network extracts prominent features that each induce object dynamics. For prediction of the motion sequence of an unknown object, the static image of the object and robot motor value are input into the neural network to calculate the PB values. By inputting the PB values into the closed loop RNNPB, the predicted movements of the object relative to the robot motion are calculated recursively. Experiments were conducted with the humanoid robot Robovie-IIs pushing objects at different heights. The results of the experiment predicting the dynamics of target objects proved that the technique is efficient for predicting the dynamics of the objects.

AB - Prediction of dynamic features is an important task for determining the manipulation strategies of an object. This paper presents a technique for predicting dynamics of objects relative to the robot's motion from visual images. During the training phase, the authors use the recurrent neural network with parametric bias (RNNPB) to self-organize the dynamics of objects manipulated by the robot into the PB space. The acquired PB values, static images of objects and robot motor values are input into a hierarchical neural network to link the images to dynamic features (PB values). The neural network extracts prominent features that each induce object dynamics. For prediction of the motion sequence of an unknown object, the static image of the object and robot motor value are input into the neural network to calculate the PB values. By inputting the PB values into the closed loop RNNPB, the predicted movements of the object relative to the robot motion are calculated recursively. Experiments were conducted with the humanoid robot Robovie-IIs pushing objects at different heights. The results of the experiment predicting the dynamics of target objects proved that the technique is efficient for predicting the dynamics of the objects.

KW - Active sensing

KW - Dynamics

KW - Humanoid robot

KW - Neural networks

KW - Object manipulation

UR - http://www.scopus.com/inward/record.url?scp=42149194330&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=42149194330&partnerID=8YFLogxK

U2 - 10.1163/156855308X294879

DO - 10.1163/156855308X294879

M3 - Article

AN - SCOPUS:42149194330

VL - 22

SP - 527

EP - 546

JO - Advanced Robotics

JF - Advanced Robotics

SN - 0169-1864

IS - 5

ER -