Real-time liquid pouring motion generation: End-to-end sensorimotor coordination for unknown liquid dynamics trained with deep neural networks

Namiko Saito, Nguyen Ba Dai, Tetsuya Ogata, Hiroki Mori, Shigeki Sugano

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We propose a sensorimotor dynamical system model for pouring unknown liquids. With our system, a robot holds and shakes a bottle to estimate the characteristics of the contained liquid, such as viscosity and fill level, without calculating to determine their parameters. Next, the robot pours a specified amount of the liquid into another container. The system needs to integrate information on the robot's actions, the liquids, the container, and the surrounding environment to perform the estimation and execute a continuous pouring motion using the same model. We use deep neural networks (DNN) to construct the system. The DNN model repeats prediction and execution of the actions to be taken in the next time step based on the input sensorimotor data, including camera images, force sensor data, and joint angles. At the same time, the DNN model acquires liquid characteristics in the internal state. We confirmed that the DNN model can control the robot to pour a desired amount of liquid with unknown viscosity and fill level.

Original languageEnglish
Title of host publicationIEEE International Conference on Robotics and Biomimetics, ROBIO 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1077-1082
Number of pages6
ISBN (Electronic)9781728163215
DOIs
Publication statusPublished - 2019 Dec
Event2019 IEEE International Conference on Robotics and Biomimetics, ROBIO 2019 - Dali, China
Duration: 2019 Dec 62019 Dec 8

Publication series

NameIEEE International Conference on Robotics and Biomimetics, ROBIO 2019

Conference

Conference2019 IEEE International Conference on Robotics and Biomimetics, ROBIO 2019
CountryChina
CityDali
Period19/12/619/12/8

Keywords

  • Liquid characteristics estimation
  • Long Short-Term Memory (LSTM) networks
  • Neural networks
  • Pouring

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Science Applications
  • Hardware and Architecture
  • Mechanical Engineering
  • Control and Optimization

Fingerprint Dive into the research topics of 'Real-time liquid pouring motion generation: End-to-end sensorimotor coordination for unknown liquid dynamics trained with deep neural networks'. Together they form a unique fingerprint.

  • Cite this

    Saito, N., Dai, N. B., Ogata, T., Mori, H., & Sugano, S. (2019). Real-time liquid pouring motion generation: End-to-end sensorimotor coordination for unknown liquid dynamics trained with deep neural networks. In IEEE International Conference on Robotics and Biomimetics, ROBIO 2019 (pp. 1077-1082). [8961718] (IEEE International Conference on Robotics and Biomimetics, ROBIO 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ROBIO49542.2019.8961718