Learning to coordinate with deep reinforcement learning in doubles pong game

Elhadji Amadou Oury Diallo, Ayumi Sugiyama, Toshiharu Sugawara

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    4 Citations (Scopus)

    Abstract

    This paper discusses the emergence of cooperative and coordinated behaviors between joint and concurrent learning agents using deep Q-learning. Multi-agent systems (MAS) arise in a variety of domains. The collective effort is one of the main building blocks of many fundamental systems that exist in the world, and thus, sequential decision making under uncertainty for collaborative work is one of the important and challenging issues for intelligent cooperative multiple agents. However, the decisions for cooperation are highly sophisticated and complicated because agents may have a certain shared goal or individual goals to achieve and their behavior is inevitably influenced by each other. Therefore, we attempt to explore whether agents using deep Q-networks (DQN) can learn cooperative behavior. We use doubles pong game as an example and we investigate how they learn to divide their works through iterated game executions. In our approach, agents jointly learn to divide their area of responsibility and each agent uses its own DQN to modify its behavior. We also investigate how learned behavior changes according to environmental characteristics including reward schemes and learning techniques. Our experiments indicate that effective cooperative behaviors with balanced division of workload emerge. These results help us to better understand how agents behave and interact with each other in complex environments and how they coherently choose their individual actions such that the resulting joint actions are optimal.

    Original languageEnglish
    Title of host publicationProceedings - 16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages14-19
    Number of pages6
    Volume2018-January
    ISBN (Electronic)9781538614174
    DOIs
    Publication statusPublished - 2018 Jan 16
    Event16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017 - Cancun, Mexico
    Duration: 2017 Dec 182017 Dec 21

    Other

    Other16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017
    CountryMexico
    CityCancun
    Period17/12/1817/12/21

      Fingerprint

    Keywords

    • Artificial Intelligence
    • Cooperative Systems
    • Deep Reinforcement Learning
    • Multi agent systems

    ASJC Scopus subject areas

    • Artificial Intelligence
    • Computer Science Applications

    Cite this

    Diallo, E. A. O., Sugiyama, A., & Sugawara, T. (2018). Learning to coordinate with deep reinforcement learning in doubles pong game. In Proceedings - 16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017 (Vol. 2018-January, pp. 14-19). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICMLA.2017.0-184