Comparison of Consolidation Methods for Predictive Learning of Time Series

Ryoichi Nakajo*, Tetsuya Ogata

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In environments where various tasks are sequentially given to deep neural networks (DNNs), training methods are needed that enable DNNs to learn the given tasks continuously. A DNN is typically trained by a single dataset, and continuous learning of subsequent datasets causes the problem of catastrophic forgetting. Previous studies have reported results for consolidation learning methods in recognition tasks and reinforcement learning problems. However, those methods were validated on only a few examples of predictive learning for time series. In this study, we applied elastic weight consolidation (EWC) and pseudo-rehearsal to the predictive learning of time series and compared their learning results. Evaluating the latent space after the consolidation learning revealed that the EWC method acquires properties of the pre-training and subsequent datasets with the same distribution, and the pseudo-rehearsal method distinguishes the properties and acquires them with different distributions.

Original languageEnglish
Title of host publicationAdvances and Trends in Artificial Intelligence. Artificial Intelligence Practices - 34th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2021, Proceedings
EditorsHamido Fujita, Ali Selamat, Jerry Chun-Wei Lin, Moonis Ali
PublisherSpringer Science and Business Media Deutschland GmbH
Pages113-120
Number of pages8
ISBN (Print)9783030794569
DOIs
Publication statusPublished - 2021
Event34th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2021 - Virtual, Online
Duration: 2021 Jul 262021 Jul 29

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12798 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference34th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2021
CityVirtual, Online
Period21/7/2621/7/29

Keywords

  • Consolidation learning
  • Predictive learning
  • Recurrent neural network

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'Comparison of Consolidation Methods for Predictive Learning of Time Series'. Together they form a unique fingerprint.

Cite this