Sensorimotor input as a language generalisation tool

a neurorobotics model for generation and generalisation of noun-verb combinations with sensorimotor inputs

Junpei Zhong, Martin Peniak, Jun Tani, Tetsuya Ogata, Angelo Cangelosi

    Research output: Contribution to journalArticle

    Abstract

    The paper presents a neurorobotics cognitive model explaining the understanding and generalisation of nouns and verbs combinations when a vocal command consisting of a verb-noun sentence is provided to a humanoid robot. The dataset used for training was obtained from object manipulation tasks with a humanoid robot platform; it includes 9 motor actions and 9 objects placing placed in 6 different locations), which enables the robot to learn to handle real-world objects and actions. Based on the multiple time-scale recurrent neural networks, this study demonstrates its generalisation capability using a large data-set, with which the robot was able to generalise semantic representation of novel combinations of noun-verb sentences, and therefore produce the corresponding motor behaviours. This generalisation process is done via the grounding process: different objects are being interacted, and associated, with different motor behaviours, following a learning approach inspired by developmental language acquisition in infants. Further analyses of the learned network dynamics and representations also demonstrate how the generalisation is possible via the exploitation of this functional hierarchical recurrent network.

    Original languageEnglish
    JournalAutonomous Robots
    DOIs
    Publication statusAccepted/In press - 2018 Jan 1

    Fingerprint

    Robots
    Recurrent neural networks
    Electric grounding
    Semantics

    Keywords

    • Developmental robotics
    • Language learning
    • Multiple time-scale recurrent neural network
    • Neurorobotics
    • Recurrent artificial neural networks

    ASJC Scopus subject areas

    • Artificial Intelligence

    Cite this

    Sensorimotor input as a language generalisation tool : a neurorobotics model for generation and generalisation of noun-verb combinations with sensorimotor inputs. / Zhong, Junpei; Peniak, Martin; Tani, Jun; Ogata, Tetsuya; Cangelosi, Angelo.

    In: Autonomous Robots, 01.01.2018.

    Research output: Contribution to journalArticle

    @article{2695120ff3644fa7a889519bf4cdd630,
    title = "Sensorimotor input as a language generalisation tool: a neurorobotics model for generation and generalisation of noun-verb combinations with sensorimotor inputs",
    abstract = "The paper presents a neurorobotics cognitive model explaining the understanding and generalisation of nouns and verbs combinations when a vocal command consisting of a verb-noun sentence is provided to a humanoid robot. The dataset used for training was obtained from object manipulation tasks with a humanoid robot platform; it includes 9 motor actions and 9 objects placing placed in 6 different locations), which enables the robot to learn to handle real-world objects and actions. Based on the multiple time-scale recurrent neural networks, this study demonstrates its generalisation capability using a large data-set, with which the robot was able to generalise semantic representation of novel combinations of noun-verb sentences, and therefore produce the corresponding motor behaviours. This generalisation process is done via the grounding process: different objects are being interacted, and associated, with different motor behaviours, following a learning approach inspired by developmental language acquisition in infants. Further analyses of the learned network dynamics and representations also demonstrate how the generalisation is possible via the exploitation of this functional hierarchical recurrent network.",
    keywords = "Developmental robotics, Language learning, Multiple time-scale recurrent neural network, Neurorobotics, Recurrent artificial neural networks",
    author = "Junpei Zhong and Martin Peniak and Jun Tani and Tetsuya Ogata and Angelo Cangelosi",
    year = "2018",
    month = "1",
    day = "1",
    doi = "10.1007/s10514-018-9793-7",
    language = "English",
    journal = "Autonomous Robots",
    issn = "0929-5593",
    publisher = "Springer Netherlands",

    }

    TY - JOUR

    T1 - Sensorimotor input as a language generalisation tool

    T2 - a neurorobotics model for generation and generalisation of noun-verb combinations with sensorimotor inputs

    AU - Zhong, Junpei

    AU - Peniak, Martin

    AU - Tani, Jun

    AU - Ogata, Tetsuya

    AU - Cangelosi, Angelo

    PY - 2018/1/1

    Y1 - 2018/1/1

    N2 - The paper presents a neurorobotics cognitive model explaining the understanding and generalisation of nouns and verbs combinations when a vocal command consisting of a verb-noun sentence is provided to a humanoid robot. The dataset used for training was obtained from object manipulation tasks with a humanoid robot platform; it includes 9 motor actions and 9 objects placing placed in 6 different locations), which enables the robot to learn to handle real-world objects and actions. Based on the multiple time-scale recurrent neural networks, this study demonstrates its generalisation capability using a large data-set, with which the robot was able to generalise semantic representation of novel combinations of noun-verb sentences, and therefore produce the corresponding motor behaviours. This generalisation process is done via the grounding process: different objects are being interacted, and associated, with different motor behaviours, following a learning approach inspired by developmental language acquisition in infants. Further analyses of the learned network dynamics and representations also demonstrate how the generalisation is possible via the exploitation of this functional hierarchical recurrent network.

    AB - The paper presents a neurorobotics cognitive model explaining the understanding and generalisation of nouns and verbs combinations when a vocal command consisting of a verb-noun sentence is provided to a humanoid robot. The dataset used for training was obtained from object manipulation tasks with a humanoid robot platform; it includes 9 motor actions and 9 objects placing placed in 6 different locations), which enables the robot to learn to handle real-world objects and actions. Based on the multiple time-scale recurrent neural networks, this study demonstrates its generalisation capability using a large data-set, with which the robot was able to generalise semantic representation of novel combinations of noun-verb sentences, and therefore produce the corresponding motor behaviours. This generalisation process is done via the grounding process: different objects are being interacted, and associated, with different motor behaviours, following a learning approach inspired by developmental language acquisition in infants. Further analyses of the learned network dynamics and representations also demonstrate how the generalisation is possible via the exploitation of this functional hierarchical recurrent network.

    KW - Developmental robotics

    KW - Language learning

    KW - Multiple time-scale recurrent neural network

    KW - Neurorobotics

    KW - Recurrent artificial neural networks

    UR - http://www.scopus.com/inward/record.url?scp=85052556161&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85052556161&partnerID=8YFLogxK

    U2 - 10.1007/s10514-018-9793-7

    DO - 10.1007/s10514-018-9793-7

    M3 - Article

    JO - Autonomous Robots

    JF - Autonomous Robots

    SN - 0929-5593

    ER -