A Meta Reinforcement Learning-based Approach for Self-Adaptive System

Mingyue Zhang, Jialong Li, Haiyan Zhao, Kenji Tei, Shinichi Honiden, Zhi Jin*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

A self-learning adaptive system (SLAS) uses machine learning to enable and enhance its adaptability. Such systems are expected to perform well in dynamic environments. For learning high-performance adaptation policy, some assumptions must be made on the environment-system dynamics when information about the real situation is incomplete. However, these assumptions cannot be expected to be always correct, and yet it is difficult to enumerate all possible assumptions. This leads to the problem of incomplete-information learning. We consider this problem as multiple model problem in terms of finding the adaptation policy that can cope with multiple models of environment-system dynamics. This paper proposes a novel approach to engineering the online adaptation of SLAS. It separates three concerns that are related to the adaptation policy and presents the modeling and synthesis process, with the goal of achieving higher model construction efficiency. In addition, it designs a meta-reinforcement learning algorithm for learning the meta policy over the multiple models, so that the meta policy can quickly adapt to the real environment-system dynamics. At last, it reports the case study on a robotic system to evaluate the adaptability of the approach.

Original languageEnglish
Title of host publicationProceedings - 2021 IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2021
EditorsEsam El-Araby, Vana Kalogeraki, Danilo Pianini, Frederic Lassabe, Barry Porter, Sona Ghahremani, Ingrid Nunes, Mohamed Bakhouya, Sven Tomforde
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1-10
Number of pages10
ISBN (Electronic)9781665412612
DOIs
Publication statusPublished - 2021
Event2nd IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2021 - Virtual, Online, United States
Duration: 2021 Sep 272021 Oct 1

Publication series

NameProceedings - 2021 IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2021

Conference

Conference2nd IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2021
Country/TerritoryUnited States
CityVirtual, Online
Period21/9/2721/10/1

Keywords

  • Meta Learning
  • Reinforcement Learning
  • Self-adaptation
  • Separation of Concerns

ASJC Scopus subject areas

  • Information Systems and Management
  • Control and Optimization
  • Artificial Intelligence
  • Computer Networks and Communications
  • Hardware and Architecture

Fingerprint

Dive into the research topics of 'A Meta Reinforcement Learning-based Approach for Self-Adaptive System'. Together they form a unique fingerprint.

Cite this