MAT-DQN: Toward Interpretable Multi-agent Deep Reinforcement Learning for Coordinated Activities

Yoshinari Motokawa*, Toshiharu Sugawara

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We propose an interpretable neural network architecture for multi-agent deep reinforcement learning to understand the rationale for learned cooperative behavior of the agents. Although the deep learning technology has contributed significantly to multi-agent systems to build coordination among agents, it is still unclear what information the agents depend on to behave cooperatively. Removing this ambiguity may further improve the efficiency and productivity of multi-agent systems. The main idea of our proposal is to adopt the transformer to deep Q-network for addressing the above-mentioned issue. By extracting multi-head attention weights from the transformer encoder, we propose a multi-agent transformer deep Q-network (MAT-DQN) and show that agents using attention mechanisms possess better coordination capability with other agents despite being trained individually for a cooperative patrolling task problem; thus, they can exhibit better performance results compared with the agents with vanilla DQN (which is a baseline method). Furthermore, we indicate that it is possible to visualize heatmaps of attentions, which indicate the influential input-information in agents’ decision-making process for their cooperative behaviors.

Original languageEnglish
Title of host publicationArtificial Neural Networks and Machine Learning – ICANN 2021 - 30th International Conference on Artificial Neural Networks, Proceedings
EditorsIgor Farkaš, Paolo Masulli, Sebastian Otte, Stefan Wermter
PublisherSpringer Science and Business Media Deutschland GmbH
Pages556-567
Number of pages12
ISBN (Print)9783030863791
DOIs
Publication statusPublished - 2021
Event30th International Conference on Artificial Neural Networks, ICANN 2021 - Virtual, Online
Duration: 2021 Sep 142021 Sep 17

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12894 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference30th International Conference on Artificial Neural Networks, ICANN 2021
CityVirtual, Online
Period21/9/1421/9/17

Keywords

  • Attention mechanism
  • Distributed autonomous system
  • Multi-agent deep reinforcement learning

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'MAT-DQN: Toward Interpretable Multi-agent Deep Reinforcement Learning for Coordinated Activities'. Together they form a unique fingerprint.

Cite this