MAT-DQN: Toward Interpretable Multi-agent Deep Reinforcement Learning for Coordinated Activities

Yoshinari Motokawa*, Toshiharu Sugawara

*この研究の対応する著者

研究成果: Conference contribution

抄録

We propose an interpretable neural network architecture for multi-agent deep reinforcement learning to understand the rationale for learned cooperative behavior of the agents. Although the deep learning technology has contributed significantly to multi-agent systems to build coordination among agents, it is still unclear what information the agents depend on to behave cooperatively. Removing this ambiguity may further improve the efficiency and productivity of multi-agent systems. The main idea of our proposal is to adopt the transformer to deep Q-network for addressing the above-mentioned issue. By extracting multi-head attention weights from the transformer encoder, we propose a multi-agent transformer deep Q-network (MAT-DQN) and show that agents using attention mechanisms possess better coordination capability with other agents despite being trained individually for a cooperative patrolling task problem; thus, they can exhibit better performance results compared with the agents with vanilla DQN (which is a baseline method). Furthermore, we indicate that it is possible to visualize heatmaps of attentions, which indicate the influential input-information in agents’ decision-making process for their cooperative behaviors.

本文言語English
ホスト出版物のタイトルArtificial Neural Networks and Machine Learning – ICANN 2021 - 30th International Conference on Artificial Neural Networks, Proceedings
編集者Igor Farkaš, Paolo Masulli, Sebastian Otte, Stefan Wermter
出版社Springer Science and Business Media Deutschland GmbH
ページ556-567
ページ数12
ISBN(印刷版)9783030863791
DOI
出版ステータスPublished - 2021
イベント30th International Conference on Artificial Neural Networks, ICANN 2021 - Virtual, Online
継続期間: 2021 9月 142021 9月 17

出版物シリーズ

名前Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
12894 LNCS
ISSN(印刷版)0302-9743
ISSN(電子版)1611-3349

Conference

Conference30th International Conference on Artificial Neural Networks, ICANN 2021
CityVirtual, Online
Period21/9/1421/9/17

ASJC Scopus subject areas

  • 理論的コンピュータサイエンス
  • コンピュータ サイエンス(全般)

フィンガープリント

「MAT-DQN: Toward Interpretable Multi-agent Deep Reinforcement Learning for Coordinated Activities」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル