End-to-End Visuomotor Learning of Drawing Sequences using Recurrent Neural Networks

Kazuma Sasaki, Tetsuya Ogata

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    Drawing is one of the complex cognitive abilities of humans. Cognitive neuropsychological studies have attempted to develop models that can explain the observations of the drawing behavior. These models exhibit limitations to reproduce the drawing behaviors because of individual factors that are related to the drawing style or non-reproducibility of motions. A constructive approach provides another methodology to investigate the complex systems by constructing models that can reproducibly replicate the behaviors. In this study, we focus on an ability to reuse the integrated visuomotor memory of drawing to associate the drawing motion from an image. Existing computational models of drawing have not considered the visual information in hand-drawn pictures. Therefore, we propose a dynamical model of the visuomotor process of drawing. The proposed model does not require any prior knowledge of the process such as the pre-designed shape primitives or the image processing algorithms. The proposed model is implemented by utilizing a recurrent neural network that learns the visuomotor transition of the drawing process. The association of the model's drawing motion by reusing the obtained memory can be obtained by minimizing the prediction error of the image. By performing simulator experiments, the proposed model demonstrates its association ability in case of pictures that comprise multiple lines.

    Original languageEnglish
    Title of host publication2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Volume2018-July
    ISBN (Electronic)9781509060146
    DOIs
    Publication statusPublished - 2018 Oct 10
    Event2018 International Joint Conference on Neural Networks, IJCNN 2018 - Rio de Janeiro, Brazil
    Duration: 2018 Jul 82018 Jul 13

    Other

    Other2018 International Joint Conference on Neural Networks, IJCNN 2018
    CountryBrazil
    CityRio de Janeiro
    Period18/7/818/7/13

    Fingerprint

    Recurrent neural networks
    Data storage equipment
    Large scale systems
    Image processing
    Simulators

    ASJC Scopus subject areas

    • Software
    • Artificial Intelligence

    Cite this

    Sasaki, K., & Ogata, T. (2018). End-to-End Visuomotor Learning of Drawing Sequences using Recurrent Neural Networks. In 2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings (Vol. 2018-July). [8489744] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IJCNN.2018.8489744

    End-to-End Visuomotor Learning of Drawing Sequences using Recurrent Neural Networks. / Sasaki, Kazuma; Ogata, Tetsuya.

    2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings. Vol. 2018-July Institute of Electrical and Electronics Engineers Inc., 2018. 8489744.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Sasaki, K & Ogata, T 2018, End-to-End Visuomotor Learning of Drawing Sequences using Recurrent Neural Networks. in 2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings. vol. 2018-July, 8489744, Institute of Electrical and Electronics Engineers Inc., 2018 International Joint Conference on Neural Networks, IJCNN 2018, Rio de Janeiro, Brazil, 18/7/8. https://doi.org/10.1109/IJCNN.2018.8489744
    Sasaki K, Ogata T. End-to-End Visuomotor Learning of Drawing Sequences using Recurrent Neural Networks. In 2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings. Vol. 2018-July. Institute of Electrical and Electronics Engineers Inc. 2018. 8489744 https://doi.org/10.1109/IJCNN.2018.8489744
    Sasaki, Kazuma ; Ogata, Tetsuya. / End-to-End Visuomotor Learning of Drawing Sequences using Recurrent Neural Networks. 2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings. Vol. 2018-July Institute of Electrical and Electronics Engineers Inc., 2018.
    @inproceedings{5d9a3914742646b3bac00b89798bb5f6,
    title = "End-to-End Visuomotor Learning of Drawing Sequences using Recurrent Neural Networks",
    abstract = "Drawing is one of the complex cognitive abilities of humans. Cognitive neuropsychological studies have attempted to develop models that can explain the observations of the drawing behavior. These models exhibit limitations to reproduce the drawing behaviors because of individual factors that are related to the drawing style or non-reproducibility of motions. A constructive approach provides another methodology to investigate the complex systems by constructing models that can reproducibly replicate the behaviors. In this study, we focus on an ability to reuse the integrated visuomotor memory of drawing to associate the drawing motion from an image. Existing computational models of drawing have not considered the visual information in hand-drawn pictures. Therefore, we propose a dynamical model of the visuomotor process of drawing. The proposed model does not require any prior knowledge of the process such as the pre-designed shape primitives or the image processing algorithms. The proposed model is implemented by utilizing a recurrent neural network that learns the visuomotor transition of the drawing process. The association of the model's drawing motion by reusing the obtained memory can be obtained by minimizing the prediction error of the image. By performing simulator experiments, the proposed model demonstrates its association ability in case of pictures that comprise multiple lines.",
    author = "Kazuma Sasaki and Tetsuya Ogata",
    year = "2018",
    month = "10",
    day = "10",
    doi = "10.1109/IJCNN.2018.8489744",
    language = "English",
    volume = "2018-July",
    booktitle = "2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings",
    publisher = "Institute of Electrical and Electronics Engineers Inc.",

    }

    TY - GEN

    T1 - End-to-End Visuomotor Learning of Drawing Sequences using Recurrent Neural Networks

    AU - Sasaki, Kazuma

    AU - Ogata, Tetsuya

    PY - 2018/10/10

    Y1 - 2018/10/10

    N2 - Drawing is one of the complex cognitive abilities of humans. Cognitive neuropsychological studies have attempted to develop models that can explain the observations of the drawing behavior. These models exhibit limitations to reproduce the drawing behaviors because of individual factors that are related to the drawing style or non-reproducibility of motions. A constructive approach provides another methodology to investigate the complex systems by constructing models that can reproducibly replicate the behaviors. In this study, we focus on an ability to reuse the integrated visuomotor memory of drawing to associate the drawing motion from an image. Existing computational models of drawing have not considered the visual information in hand-drawn pictures. Therefore, we propose a dynamical model of the visuomotor process of drawing. The proposed model does not require any prior knowledge of the process such as the pre-designed shape primitives or the image processing algorithms. The proposed model is implemented by utilizing a recurrent neural network that learns the visuomotor transition of the drawing process. The association of the model's drawing motion by reusing the obtained memory can be obtained by minimizing the prediction error of the image. By performing simulator experiments, the proposed model demonstrates its association ability in case of pictures that comprise multiple lines.

    AB - Drawing is one of the complex cognitive abilities of humans. Cognitive neuropsychological studies have attempted to develop models that can explain the observations of the drawing behavior. These models exhibit limitations to reproduce the drawing behaviors because of individual factors that are related to the drawing style or non-reproducibility of motions. A constructive approach provides another methodology to investigate the complex systems by constructing models that can reproducibly replicate the behaviors. In this study, we focus on an ability to reuse the integrated visuomotor memory of drawing to associate the drawing motion from an image. Existing computational models of drawing have not considered the visual information in hand-drawn pictures. Therefore, we propose a dynamical model of the visuomotor process of drawing. The proposed model does not require any prior knowledge of the process such as the pre-designed shape primitives or the image processing algorithms. The proposed model is implemented by utilizing a recurrent neural network that learns the visuomotor transition of the drawing process. The association of the model's drawing motion by reusing the obtained memory can be obtained by minimizing the prediction error of the image. By performing simulator experiments, the proposed model demonstrates its association ability in case of pictures that comprise multiple lines.

    UR - http://www.scopus.com/inward/record.url?scp=85056550133&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85056550133&partnerID=8YFLogxK

    U2 - 10.1109/IJCNN.2018.8489744

    DO - 10.1109/IJCNN.2018.8489744

    M3 - Conference contribution

    VL - 2018-July

    BT - 2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings

    PB - Institute of Electrical and Electronics Engineers Inc.

    ER -