Learning and recognition of multiple fluctuating temporal patterns using S-CTRNN

Shingo Murata, Hiroaki Arie, Tetsuya Ogata, Jun Tani, Shigeki Sugano

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    3 Citations (Scopus)

    Abstract

    In the present study, we demonstrate the learning and recognition capabilities of our recently proposed recurrent neural network (RNN) model called stochastic continuous-time RNN (S-CTRNN). S-CTRNN can learn to predict not only the mean but also the variance of the next state of the learning targets. The network parameters consisting of weights, biases, and initial states of context neurons are optimized through maximum likelihood estimation (MLE) using the gradient descent method. First, we clarify the essential difference between the learning capabilities of conventional CTRNN and S-CTRNN by analyzing the results of a numerical experiment in which multiple fluctuating temporal patterns were used as training data, where the variance of the Gaussian noise varied among the patterns. Furthermore, we also show that the trained S-CTRNN can recognize given fluctuating patterns by inferring the initial states that can reproduce the patterns through the same MLE scheme as that used for network training.

    Original languageEnglish
    Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    PublisherSpringer Verlag
    Pages9-16
    Number of pages8
    Volume8681 LNCS
    ISBN (Print)9783319111780
    DOIs
    Publication statusPublished - 2014
    Event24th International Conference on Artificial Neural Networks, ICANN 2014 - Hamburg, Germany
    Duration: 2014 Sep 152014 Sep 19

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Volume8681 LNCS
    ISSN (Print)03029743
    ISSN (Electronic)16113349

    Other

    Other24th International Conference on Artificial Neural Networks, ICANN 2014
    CountryGermany
    CityHamburg
    Period14/9/1514/9/19

    Fingerprint

    Recurrent neural networks
    Maximum likelihood estimation
    Continuous Time
    Recurrent Neural Networks
    Stochastic models
    Maximum Likelihood Estimation
    Neurons
    Gradient Descent Method
    Gaussian Noise
    Neural Network Model
    Neuron
    Experiments
    Numerical Experiment
    Predict
    Target
    Learning
    Demonstrate
    Training

    Keywords

    • recurrent neural network
    • S-CTRNN
    • variance estimation

    ASJC Scopus subject areas

    • Computer Science(all)
    • Theoretical Computer Science

    Cite this

    Murata, S., Arie, H., Ogata, T., Tani, J., & Sugano, S. (2014). Learning and recognition of multiple fluctuating temporal patterns using S-CTRNN. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 8681 LNCS, pp. 9-16). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8681 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-11179-7_2

    Learning and recognition of multiple fluctuating temporal patterns using S-CTRNN. / Murata, Shingo; Arie, Hiroaki; Ogata, Tetsuya; Tani, Jun; Sugano, Shigeki.

    Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 8681 LNCS Springer Verlag, 2014. p. 9-16 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8681 LNCS).

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Murata, S, Arie, H, Ogata, T, Tani, J & Sugano, S 2014, Learning and recognition of multiple fluctuating temporal patterns using S-CTRNN. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). vol. 8681 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 8681 LNCS, Springer Verlag, pp. 9-16, 24th International Conference on Artificial Neural Networks, ICANN 2014, Hamburg, Germany, 14/9/15. https://doi.org/10.1007/978-3-319-11179-7_2
    Murata S, Arie H, Ogata T, Tani J, Sugano S. Learning and recognition of multiple fluctuating temporal patterns using S-CTRNN. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 8681 LNCS. Springer Verlag. 2014. p. 9-16. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-11179-7_2
    Murata, Shingo ; Arie, Hiroaki ; Ogata, Tetsuya ; Tani, Jun ; Sugano, Shigeki. / Learning and recognition of multiple fluctuating temporal patterns using S-CTRNN. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 8681 LNCS Springer Verlag, 2014. pp. 9-16 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
    @inproceedings{ad50fa1ffbd34f6ca8e9e996b2ddcfb7,
    title = "Learning and recognition of multiple fluctuating temporal patterns using S-CTRNN",
    abstract = "In the present study, we demonstrate the learning and recognition capabilities of our recently proposed recurrent neural network (RNN) model called stochastic continuous-time RNN (S-CTRNN). S-CTRNN can learn to predict not only the mean but also the variance of the next state of the learning targets. The network parameters consisting of weights, biases, and initial states of context neurons are optimized through maximum likelihood estimation (MLE) using the gradient descent method. First, we clarify the essential difference between the learning capabilities of conventional CTRNN and S-CTRNN by analyzing the results of a numerical experiment in which multiple fluctuating temporal patterns were used as training data, where the variance of the Gaussian noise varied among the patterns. Furthermore, we also show that the trained S-CTRNN can recognize given fluctuating patterns by inferring the initial states that can reproduce the patterns through the same MLE scheme as that used for network training.",
    keywords = "recurrent neural network, S-CTRNN, variance estimation",
    author = "Shingo Murata and Hiroaki Arie and Tetsuya Ogata and Jun Tani and Shigeki Sugano",
    year = "2014",
    doi = "10.1007/978-3-319-11179-7_2",
    language = "English",
    isbn = "9783319111780",
    volume = "8681 LNCS",
    series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
    publisher = "Springer Verlag",
    pages = "9--16",
    booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

    }

    TY - GEN

    T1 - Learning and recognition of multiple fluctuating temporal patterns using S-CTRNN

    AU - Murata, Shingo

    AU - Arie, Hiroaki

    AU - Ogata, Tetsuya

    AU - Tani, Jun

    AU - Sugano, Shigeki

    PY - 2014

    Y1 - 2014

    N2 - In the present study, we demonstrate the learning and recognition capabilities of our recently proposed recurrent neural network (RNN) model called stochastic continuous-time RNN (S-CTRNN). S-CTRNN can learn to predict not only the mean but also the variance of the next state of the learning targets. The network parameters consisting of weights, biases, and initial states of context neurons are optimized through maximum likelihood estimation (MLE) using the gradient descent method. First, we clarify the essential difference between the learning capabilities of conventional CTRNN and S-CTRNN by analyzing the results of a numerical experiment in which multiple fluctuating temporal patterns were used as training data, where the variance of the Gaussian noise varied among the patterns. Furthermore, we also show that the trained S-CTRNN can recognize given fluctuating patterns by inferring the initial states that can reproduce the patterns through the same MLE scheme as that used for network training.

    AB - In the present study, we demonstrate the learning and recognition capabilities of our recently proposed recurrent neural network (RNN) model called stochastic continuous-time RNN (S-CTRNN). S-CTRNN can learn to predict not only the mean but also the variance of the next state of the learning targets. The network parameters consisting of weights, biases, and initial states of context neurons are optimized through maximum likelihood estimation (MLE) using the gradient descent method. First, we clarify the essential difference between the learning capabilities of conventional CTRNN and S-CTRNN by analyzing the results of a numerical experiment in which multiple fluctuating temporal patterns were used as training data, where the variance of the Gaussian noise varied among the patterns. Furthermore, we also show that the trained S-CTRNN can recognize given fluctuating patterns by inferring the initial states that can reproduce the patterns through the same MLE scheme as that used for network training.

    KW - recurrent neural network

    KW - S-CTRNN

    KW - variance estimation

    UR - http://www.scopus.com/inward/record.url?scp=84958521901&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=84958521901&partnerID=8YFLogxK

    U2 - 10.1007/978-3-319-11179-7_2

    DO - 10.1007/978-3-319-11179-7_2

    M3 - Conference contribution

    SN - 9783319111780

    VL - 8681 LNCS

    T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

    SP - 9

    EP - 16

    BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

    PB - Springer Verlag

    ER -