Learning to coordinate with deep reinforcement learning in doubles pong game

Elhadji Amadou Oury Diallo, Ayumi Sugiyama, Toshiharu Sugawara

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    3 Citations (Scopus)

    Abstract

    This paper discusses the emergence of cooperative and coordinated behaviors between joint and concurrent learning agents using deep Q-learning. Multi-agent systems (MAS) arise in a variety of domains. The collective effort is one of the main building blocks of many fundamental systems that exist in the world, and thus, sequential decision making under uncertainty for collaborative work is one of the important and challenging issues for intelligent cooperative multiple agents. However, the decisions for cooperation are highly sophisticated and complicated because agents may have a certain shared goal or individual goals to achieve and their behavior is inevitably influenced by each other. Therefore, we attempt to explore whether agents using deep Q-networks (DQN) can learn cooperative behavior. We use doubles pong game as an example and we investigate how they learn to divide their works through iterated game executions. In our approach, agents jointly learn to divide their area of responsibility and each agent uses its own DQN to modify its behavior. We also investigate how learned behavior changes according to environmental characteristics including reward schemes and learning techniques. Our experiments indicate that effective cooperative behaviors with balanced division of workload emerge. These results help us to better understand how agents behave and interact with each other in complex environments and how they coherently choose their individual actions such that the resulting joint actions are optimal.

    Original languageEnglish
    Title of host publicationProceedings - 16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages14-19
    Number of pages6
    Volume2018-January
    ISBN (Electronic)9781538614174
    DOIs
    Publication statusPublished - 2018 Jan 16
    Event16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017 - Cancun, Mexico
    Duration: 2017 Dec 182017 Dec 21

    Other

    Other16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017
    CountryMexico
    CityCancun
    Period17/12/1817/12/21

    Fingerprint

    Reinforcement learning
    Multi agent systems
    Decision making
    Experiments

    Keywords

    • Artificial Intelligence
    • Cooperative Systems
    • Deep Reinforcement Learning
    • Multi agent systems

    ASJC Scopus subject areas

    • Artificial Intelligence
    • Computer Science Applications

    Cite this

    Diallo, E. A. O., Sugiyama, A., & Sugawara, T. (2018). Learning to coordinate with deep reinforcement learning in doubles pong game. In Proceedings - 16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017 (Vol. 2018-January, pp. 14-19). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICMLA.2017.0-184

    Learning to coordinate with deep reinforcement learning in doubles pong game. / Diallo, Elhadji Amadou Oury; Sugiyama, Ayumi; Sugawara, Toshiharu.

    Proceedings - 16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017. Vol. 2018-January Institute of Electrical and Electronics Engineers Inc., 2018. p. 14-19.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Diallo, EAO, Sugiyama, A & Sugawara, T 2018, Learning to coordinate with deep reinforcement learning in doubles pong game. in Proceedings - 16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017. vol. 2018-January, Institute of Electrical and Electronics Engineers Inc., pp. 14-19, 16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017, Cancun, Mexico, 17/12/18. https://doi.org/10.1109/ICMLA.2017.0-184
    Diallo EAO, Sugiyama A, Sugawara T. Learning to coordinate with deep reinforcement learning in doubles pong game. In Proceedings - 16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017. Vol. 2018-January. Institute of Electrical and Electronics Engineers Inc. 2018. p. 14-19 https://doi.org/10.1109/ICMLA.2017.0-184
    Diallo, Elhadji Amadou Oury ; Sugiyama, Ayumi ; Sugawara, Toshiharu. / Learning to coordinate with deep reinforcement learning in doubles pong game. Proceedings - 16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017. Vol. 2018-January Institute of Electrical and Electronics Engineers Inc., 2018. pp. 14-19
    @inproceedings{a24569ba59574026b1f0fc9d69d82c24,
    title = "Learning to coordinate with deep reinforcement learning in doubles pong game",
    abstract = "This paper discusses the emergence of cooperative and coordinated behaviors between joint and concurrent learning agents using deep Q-learning. Multi-agent systems (MAS) arise in a variety of domains. The collective effort is one of the main building blocks of many fundamental systems that exist in the world, and thus, sequential decision making under uncertainty for collaborative work is one of the important and challenging issues for intelligent cooperative multiple agents. However, the decisions for cooperation are highly sophisticated and complicated because agents may have a certain shared goal or individual goals to achieve and their behavior is inevitably influenced by each other. Therefore, we attempt to explore whether agents using deep Q-networks (DQN) can learn cooperative behavior. We use doubles pong game as an example and we investigate how they learn to divide their works through iterated game executions. In our approach, agents jointly learn to divide their area of responsibility and each agent uses its own DQN to modify its behavior. We also investigate how learned behavior changes according to environmental characteristics including reward schemes and learning techniques. Our experiments indicate that effective cooperative behaviors with balanced division of workload emerge. These results help us to better understand how agents behave and interact with each other in complex environments and how they coherently choose their individual actions such that the resulting joint actions are optimal.",
    keywords = "Artificial Intelligence, Cooperative Systems, Deep Reinforcement Learning, Multi agent systems",
    author = "Diallo, {Elhadji Amadou Oury} and Ayumi Sugiyama and Toshiharu Sugawara",
    year = "2018",
    month = "1",
    day = "16",
    doi = "10.1109/ICMLA.2017.0-184",
    language = "English",
    volume = "2018-January",
    pages = "14--19",
    booktitle = "Proceedings - 16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017",
    publisher = "Institute of Electrical and Electronics Engineers Inc.",

    }

    TY - GEN

    T1 - Learning to coordinate with deep reinforcement learning in doubles pong game

    AU - Diallo, Elhadji Amadou Oury

    AU - Sugiyama, Ayumi

    AU - Sugawara, Toshiharu

    PY - 2018/1/16

    Y1 - 2018/1/16

    N2 - This paper discusses the emergence of cooperative and coordinated behaviors between joint and concurrent learning agents using deep Q-learning. Multi-agent systems (MAS) arise in a variety of domains. The collective effort is one of the main building blocks of many fundamental systems that exist in the world, and thus, sequential decision making under uncertainty for collaborative work is one of the important and challenging issues for intelligent cooperative multiple agents. However, the decisions for cooperation are highly sophisticated and complicated because agents may have a certain shared goal or individual goals to achieve and their behavior is inevitably influenced by each other. Therefore, we attempt to explore whether agents using deep Q-networks (DQN) can learn cooperative behavior. We use doubles pong game as an example and we investigate how they learn to divide their works through iterated game executions. In our approach, agents jointly learn to divide their area of responsibility and each agent uses its own DQN to modify its behavior. We also investigate how learned behavior changes according to environmental characteristics including reward schemes and learning techniques. Our experiments indicate that effective cooperative behaviors with balanced division of workload emerge. These results help us to better understand how agents behave and interact with each other in complex environments and how they coherently choose their individual actions such that the resulting joint actions are optimal.

    AB - This paper discusses the emergence of cooperative and coordinated behaviors between joint and concurrent learning agents using deep Q-learning. Multi-agent systems (MAS) arise in a variety of domains. The collective effort is one of the main building blocks of many fundamental systems that exist in the world, and thus, sequential decision making under uncertainty for collaborative work is one of the important and challenging issues for intelligent cooperative multiple agents. However, the decisions for cooperation are highly sophisticated and complicated because agents may have a certain shared goal or individual goals to achieve and their behavior is inevitably influenced by each other. Therefore, we attempt to explore whether agents using deep Q-networks (DQN) can learn cooperative behavior. We use doubles pong game as an example and we investigate how they learn to divide their works through iterated game executions. In our approach, agents jointly learn to divide their area of responsibility and each agent uses its own DQN to modify its behavior. We also investigate how learned behavior changes according to environmental characteristics including reward schemes and learning techniques. Our experiments indicate that effective cooperative behaviors with balanced division of workload emerge. These results help us to better understand how agents behave and interact with each other in complex environments and how they coherently choose their individual actions such that the resulting joint actions are optimal.

    KW - Artificial Intelligence

    KW - Cooperative Systems

    KW - Deep Reinforcement Learning

    KW - Multi agent systems

    UR - http://www.scopus.com/inward/record.url?scp=85048507667&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85048507667&partnerID=8YFLogxK

    U2 - 10.1109/ICMLA.2017.0-184

    DO - 10.1109/ICMLA.2017.0-184

    M3 - Conference contribution

    VL - 2018-January

    SP - 14

    EP - 19

    BT - Proceedings - 16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017

    PB - Institute of Electrical and Electronics Engineers Inc.

    ER -