Cooperation and Coordination Regimes by Deep Q-Learning in Multi-agent Task Executions

Yuki Miyashita, Toshiharu Sugawara

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

We investigate the coordination structures generated by deep Q-network (DQN) with various types of input by using a distributed task execution game. Although cooperation and coordination are mandatory for efficiency in multi-agent systems (MAS), they require sophisticated structures or regimes for effective behaviors. Recently, deep Q-learning has been applied to multi-agent systems to facilitate their coordinated behavior. However, the characteristics of the learned results have not yet been fully clarified. We investigate how information input to DQNs affect the resultant coordination and cooperation structures. We examine the inputs generated from local observations with and without the estimated location in the environment. Experimental results show that they form two types of coordination structures—the division of labor and the targeting of near tasks while avoiding conflicts—and that the latter is more efficient in our game. We clarify the mechanism behind and the characteristics of the generated coordination behaviors.

Original languageEnglish
Title of host publicationArtificial Neural Networks and Machine Learning – ICANN 2019
Subtitle of host publicationTheoretical Neural Computation - 28th International Conference on Artificial Neural Networks, 2019, Proceedings
EditorsIgor V. Tetko, Pavel Karpov, Fabian Theis, Vera Kurková
PublisherSpringer-Verlag
Pages541-554
Number of pages14
ISBN (Print)9783030304867
DOIs
Publication statusPublished - 2019 Jan 1
Event28th International Conference on Artificial Neural Networks, ICANN 2019 - Munich, Germany
Duration: 2019 Sep 172019 Sep 19

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11727 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference28th International Conference on Artificial Neural Networks, ICANN 2019
CountryGermany
CityMunich
Period19/9/1719/9/19

Keywords

  • Cooperation
  • Coordination
  • Divisional cooperation
  • Multi-agent deep reinforcement learning

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Cooperation and Coordination Regimes by Deep Q-Learning in Multi-agent Task Executions'. Together they form a unique fingerprint.

  • Cite this

    Miyashita, Y., & Sugawara, T. (2019). Cooperation and Coordination Regimes by Deep Q-Learning in Multi-agent Task Executions. In I. V. Tetko, P. Karpov, F. Theis, & V. Kurková (Eds.), Artificial Neural Networks and Machine Learning – ICANN 2019: Theoretical Neural Computation - 28th International Conference on Artificial Neural Networks, 2019, Proceedings (pp. 541-554). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11727 LNCS). Springer-Verlag. https://doi.org/10.1007/978-3-030-30487-4_42