End-to-End Visuomotor Learning of Drawing Sequences using Recurrent Neural Networks

Drawing is one of the complex cognitive abilities of humans. Cognitive neuropsychological studies have attempted to develop models that can explain the observations of the drawing behavior. These models exhibit limitations to reproduce the drawing behaviors because of individual factors that are related to the drawing style or non-reproducibility of motions. A constructive approach provides another methodology to investigate the complex systems by constructing models that can reproducibly replicate the behaviors. In this study, we focus on an ability to reuse the integrated visuomotor memory of drawing to associate the drawing motion from an image. Existing computational models of drawing have not considered the visual information in hand-drawn pictures. Therefore, we propose a dynamical model of the visuomotor process of drawing. The proposed model does not require any prior knowledge of the process such as the pre-designed shape primitives or the image processing algorithms. The proposed model is implemented by utilizing a recurrent neural network that learns the visuomotor transition of the drawing process. The association of the model's drawing motion by reusing the obtained memory can be obtained by minimizing the prediction error of the image. By performing simulator experiments, the proposed model demonstrates its association ability in case of pictures that comprise multiple lines.