Learning to reproduce fluctuating time series by inferring their time-dependent stochastic properties: Application in Robot learning via tutoring

Shingo Murata, Jun Namikawa, Hiroaki Arie, Shigeki Sugano, Jun Tani

    Research output: Contribution to journalArticle

    32 Citations (Scopus)

    Abstract

    This study proposes a novel type of dynamic neural network model that can learn to extract stochastic or fluctuating structures hidden in time series data. The network learns to predict not only the mean of the next input state, but also its time-dependent variance. The training method is based on maximum likelihood estimation by using the gradient descent method and the likelihood function is expressed as a function of the estimated variance. Regarding the model evaluation, we present numerical experiments in which training data were generated in different ways utilizing Gaussian noise. Our analysis showed that the network can predict the time-dependent variance and the mean and it can also reproduce the target stochastic sequence data by utilizing the estimated variance. Furthermore, it was shown that a humanoid robot using the proposed network can learn to reproduce latent stochastic structures hidden in fluctuating tutoring trajectories. This learning scheme is essential for the acquisition of sensory-guided skilled behavior.

    Original languageEnglish
    Article number6502665
    Pages (from-to)298-310
    Number of pages13
    JournalIEEE Transactions on Autonomous Mental Development
    Volume5
    Issue number4
    DOIs
    Publication statusPublished - 2013 Dec

    Fingerprint

    Robot learning
    Time series
    Maximum likelihood estimation
    Trajectories
    Robots
    Neural networks
    Experiments

    Keywords

    • Dynamical systems approach
    • humanoid robot
    • maximum likelihood estimation
    • recurrent neural network

    ASJC Scopus subject areas

    • Artificial Intelligence
    • Software

    Cite this

    Learning to reproduce fluctuating time series by inferring their time-dependent stochastic properties : Application in Robot learning via tutoring. / Murata, Shingo; Namikawa, Jun; Arie, Hiroaki; Sugano, Shigeki; Tani, Jun.

    In: IEEE Transactions on Autonomous Mental Development, Vol. 5, No. 4, 6502665, 12.2013, p. 298-310.

    Research output: Contribution to journalArticle

    @article{22d6ae92fc4245dc83e6d646f54effb3,
    title = "Learning to reproduce fluctuating time series by inferring their time-dependent stochastic properties: Application in Robot learning via tutoring",
    abstract = "This study proposes a novel type of dynamic neural network model that can learn to extract stochastic or fluctuating structures hidden in time series data. The network learns to predict not only the mean of the next input state, but also its time-dependent variance. The training method is based on maximum likelihood estimation by using the gradient descent method and the likelihood function is expressed as a function of the estimated variance. Regarding the model evaluation, we present numerical experiments in which training data were generated in different ways utilizing Gaussian noise. Our analysis showed that the network can predict the time-dependent variance and the mean and it can also reproduce the target stochastic sequence data by utilizing the estimated variance. Furthermore, it was shown that a humanoid robot using the proposed network can learn to reproduce latent stochastic structures hidden in fluctuating tutoring trajectories. This learning scheme is essential for the acquisition of sensory-guided skilled behavior.",
    keywords = "Dynamical systems approach, humanoid robot, maximum likelihood estimation, recurrent neural network",
    author = "Shingo Murata and Jun Namikawa and Hiroaki Arie and Shigeki Sugano and Jun Tani",
    year = "2013",
    month = "12",
    doi = "10.1109/TAMD.2013.2258019",
    language = "English",
    volume = "5",
    pages = "298--310",
    journal = "IEEE Transactions on Cognitive and Developmental Systems",
    issn = "2379-8920",
    publisher = "Institute of Electrical and Electronics Engineers Inc.",
    number = "4",

    }

    TY - JOUR

    T1 - Learning to reproduce fluctuating time series by inferring their time-dependent stochastic properties

    T2 - Application in Robot learning via tutoring

    AU - Murata, Shingo

    AU - Namikawa, Jun

    AU - Arie, Hiroaki

    AU - Sugano, Shigeki

    AU - Tani, Jun

    PY - 2013/12

    Y1 - 2013/12

    N2 - This study proposes a novel type of dynamic neural network model that can learn to extract stochastic or fluctuating structures hidden in time series data. The network learns to predict not only the mean of the next input state, but also its time-dependent variance. The training method is based on maximum likelihood estimation by using the gradient descent method and the likelihood function is expressed as a function of the estimated variance. Regarding the model evaluation, we present numerical experiments in which training data were generated in different ways utilizing Gaussian noise. Our analysis showed that the network can predict the time-dependent variance and the mean and it can also reproduce the target stochastic sequence data by utilizing the estimated variance. Furthermore, it was shown that a humanoid robot using the proposed network can learn to reproduce latent stochastic structures hidden in fluctuating tutoring trajectories. This learning scheme is essential for the acquisition of sensory-guided skilled behavior.

    AB - This study proposes a novel type of dynamic neural network model that can learn to extract stochastic or fluctuating structures hidden in time series data. The network learns to predict not only the mean of the next input state, but also its time-dependent variance. The training method is based on maximum likelihood estimation by using the gradient descent method and the likelihood function is expressed as a function of the estimated variance. Regarding the model evaluation, we present numerical experiments in which training data were generated in different ways utilizing Gaussian noise. Our analysis showed that the network can predict the time-dependent variance and the mean and it can also reproduce the target stochastic sequence data by utilizing the estimated variance. Furthermore, it was shown that a humanoid robot using the proposed network can learn to reproduce latent stochastic structures hidden in fluctuating tutoring trajectories. This learning scheme is essential for the acquisition of sensory-guided skilled behavior.

    KW - Dynamical systems approach

    KW - humanoid robot

    KW - maximum likelihood estimation

    KW - recurrent neural network

    UR - http://www.scopus.com/inward/record.url?scp=84890921190&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=84890921190&partnerID=8YFLogxK

    U2 - 10.1109/TAMD.2013.2258019

    DO - 10.1109/TAMD.2013.2258019

    M3 - Article

    AN - SCOPUS:84890921190

    VL - 5

    SP - 298

    EP - 310

    JO - IEEE Transactions on Cognitive and Developmental Systems

    JF - IEEE Transactions on Cognitive and Developmental Systems

    SN - 2379-8920

    IS - 4

    M1 - 6502665

    ER -