Integrative learning between language and action: A neuro-robotics experiment

Hiroaki Arie, Tetsuro Endo, Sungmoon Jeong, Minho Lee, Shigeki Sugano, Jun Tani

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    7 Citations (Scopus)

    Abstract

    This paper introduces a model for associative learning combining both linguistic and behavior modalities. The model consists of language and behavior modules both implemented by a hierarchical dynamic network model and interacting densely through hub-like neurons, the so-called parametric biases (PB). By implementing this model for a humanoid robot with the task of manipulating multiple objects, the robot was tutored to associate sentences of two different grammatical types with corresponding sensory-motor schemata. The first type was a verb followed by an objective noun such as "hold red" or "hit blue"; the second was a verb followed by an objective noun and further followed by an adverb phrase such as "Put red on blue". Our analysis of the results of a learning experiment showed that two clusters corresponding to these two types of grammatical sentences appear in the PB activity space, such that a specific micro structure is organized for each cluster.

    Original languageEnglish
    Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Pages256-265
    Number of pages10
    Volume6353 LNCS
    EditionPART 2
    DOIs
    Publication statusPublished - 2010
    Event20th International Conference on Artificial Neural Networks, ICANN 2010 - Thessaloniki
    Duration: 2010 Sep 152010 Sep 18

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    NumberPART 2
    Volume6353 LNCS
    ISSN (Print)03029743
    ISSN (Electronic)16113349

    Other

    Other20th International Conference on Artificial Neural Networks, ICANN 2010
    CityThessaloniki
    Period10/9/1510/9/18

    Fingerprint

    Robotics
    Experiment
    Hierarchical Networks
    Humanoid Robot
    Experiments
    Dynamic Networks
    Robots
    Hits
    Modality
    Schema
    Network Model
    Microstructure
    Neuron
    Dynamic Model
    Robot
    Linguistics
    Model
    Neurons
    Module
    Language

    ASJC Scopus subject areas

    • Computer Science(all)
    • Theoretical Computer Science

    Cite this

    Arie, H., Endo, T., Jeong, S., Lee, M., Sugano, S., & Tani, J. (2010). Integrative learning between language and action: A neuro-robotics experiment. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (PART 2 ed., Vol. 6353 LNCS, pp. 256-265). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 6353 LNCS, No. PART 2). https://doi.org/10.1007/978-3-642-15822-3_33

    Integrative learning between language and action : A neuro-robotics experiment. / Arie, Hiroaki; Endo, Tetsuro; Jeong, Sungmoon; Lee, Minho; Sugano, Shigeki; Tani, Jun.

    Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 6353 LNCS PART 2. ed. 2010. p. 256-265 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 6353 LNCS, No. PART 2).

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Arie, H, Endo, T, Jeong, S, Lee, M, Sugano, S & Tani, J 2010, Integrative learning between language and action: A neuro-robotics experiment. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). PART 2 edn, vol. 6353 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), no. PART 2, vol. 6353 LNCS, pp. 256-265, 20th International Conference on Artificial Neural Networks, ICANN 2010, Thessaloniki, 10/9/15. https://doi.org/10.1007/978-3-642-15822-3_33
    Arie H, Endo T, Jeong S, Lee M, Sugano S, Tani J. Integrative learning between language and action: A neuro-robotics experiment. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). PART 2 ed. Vol. 6353 LNCS. 2010. p. 256-265. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 2). https://doi.org/10.1007/978-3-642-15822-3_33
    Arie, Hiroaki ; Endo, Tetsuro ; Jeong, Sungmoon ; Lee, Minho ; Sugano, Shigeki ; Tani, Jun. / Integrative learning between language and action : A neuro-robotics experiment. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 6353 LNCS PART 2. ed. 2010. pp. 256-265 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 2).
    @inproceedings{6bed5ffcb15d405a8bb3ef7770563ad6,
    title = "Integrative learning between language and action: A neuro-robotics experiment",
    abstract = "This paper introduces a model for associative learning combining both linguistic and behavior modalities. The model consists of language and behavior modules both implemented by a hierarchical dynamic network model and interacting densely through hub-like neurons, the so-called parametric biases (PB). By implementing this model for a humanoid robot with the task of manipulating multiple objects, the robot was tutored to associate sentences of two different grammatical types with corresponding sensory-motor schemata. The first type was a verb followed by an objective noun such as {"}hold red{"} or {"}hit blue{"}; the second was a verb followed by an objective noun and further followed by an adverb phrase such as {"}Put red on blue{"}. Our analysis of the results of a learning experiment showed that two clusters corresponding to these two types of grammatical sentences appear in the PB activity space, such that a specific micro structure is organized for each cluster.",
    author = "Hiroaki Arie and Tetsuro Endo and Sungmoon Jeong and Minho Lee and Shigeki Sugano and Jun Tani",
    year = "2010",
    doi = "10.1007/978-3-642-15822-3_33",
    language = "English",
    isbn = "3642158218",
    volume = "6353 LNCS",
    series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
    number = "PART 2",
    pages = "256--265",
    booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
    edition = "PART 2",

    }

    TY - GEN

    T1 - Integrative learning between language and action

    T2 - A neuro-robotics experiment

    AU - Arie, Hiroaki

    AU - Endo, Tetsuro

    AU - Jeong, Sungmoon

    AU - Lee, Minho

    AU - Sugano, Shigeki

    AU - Tani, Jun

    PY - 2010

    Y1 - 2010

    N2 - This paper introduces a model for associative learning combining both linguistic and behavior modalities. The model consists of language and behavior modules both implemented by a hierarchical dynamic network model and interacting densely through hub-like neurons, the so-called parametric biases (PB). By implementing this model for a humanoid robot with the task of manipulating multiple objects, the robot was tutored to associate sentences of two different grammatical types with corresponding sensory-motor schemata. The first type was a verb followed by an objective noun such as "hold red" or "hit blue"; the second was a verb followed by an objective noun and further followed by an adverb phrase such as "Put red on blue". Our analysis of the results of a learning experiment showed that two clusters corresponding to these two types of grammatical sentences appear in the PB activity space, such that a specific micro structure is organized for each cluster.

    AB - This paper introduces a model for associative learning combining both linguistic and behavior modalities. The model consists of language and behavior modules both implemented by a hierarchical dynamic network model and interacting densely through hub-like neurons, the so-called parametric biases (PB). By implementing this model for a humanoid robot with the task of manipulating multiple objects, the robot was tutored to associate sentences of two different grammatical types with corresponding sensory-motor schemata. The first type was a verb followed by an objective noun such as "hold red" or "hit blue"; the second was a verb followed by an objective noun and further followed by an adverb phrase such as "Put red on blue". Our analysis of the results of a learning experiment showed that two clusters corresponding to these two types of grammatical sentences appear in the PB activity space, such that a specific micro structure is organized for each cluster.

    UR - http://www.scopus.com/inward/record.url?scp=78049358956&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=78049358956&partnerID=8YFLogxK

    U2 - 10.1007/978-3-642-15822-3_33

    DO - 10.1007/978-3-642-15822-3_33

    M3 - Conference contribution

    AN - SCOPUS:78049358956

    SN - 3642158218

    SN - 9783642158216

    VL - 6353 LNCS

    T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

    SP - 256

    EP - 265

    BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

    ER -