Learning strategic group formation for coordinated behavior in adversarial multi-agent with double DQN

Elhadji Amadou Oury Diallo, Toshiharu Sugawara

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    We examine whether a team of agents can learn geometric and strategic group formations by using deep reinforcement learning in adversarial multi-agent systems. This is a significant point underlying the control and coordination of multiple autonomous and intelligent agents. While there are many possible approaches to solve this problem, we are interested in fully end-to-end learning method where agents do not have any prior knowledge of the environment and its dynamics. In this paper, we propose a scalable and distributed double DQN framework to train adversarial multi-agent systems. We show that a large number of agents can learn to cooperatively move, attack and defend themselves in various geometric formations and battle tactics like encirclement, guerrilla warfare, frontal attack, flanking maneuver, and so on. We finally show that agents create an emergent and collective flocking behaviors by using local views from the environment only.

    Original languageEnglish
    Title of host publicationPRIMA 2018
    Subtitle of host publicationPrinciples and Practice of Multi-Agent Systems - 21st International Conference, 2018, Proceedings
    EditorsNir Oren, Yuko Sakurai, Itsuki Noda, Tran Cao Son, Tim Miller, Bastin Tony Savarimuthu
    PublisherSpringer-Verlag
    Pages458-466
    Number of pages9
    ISBN (Print)9783030030971
    DOIs
    Publication statusPublished - 2018 Jan 1
    Event21st International Conference on Principles and Practice of Multi-Agent Systems, PRIMA 2018 - Tokyo, Japan
    Duration: 2018 Oct 292018 Nov 2

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Volume11224 LNAI
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349

    Other

    Other21st International Conference on Principles and Practice of Multi-Agent Systems, PRIMA 2018
    CountryJapan
    CityTokyo
    Period18/10/2918/11/2

    Fingerprint

    Multi agent systems
    Autonomous agents
    Intelligent agents
    Military operations
    Reinforcement learning
    Multi-agent Systems
    Attack
    Flocking
    Autonomous Agents
    Intelligent Agents
    Reinforcement Learning
    Prior Knowledge
    Learning

    Keywords

    • Collective intelligence
    • Cooperation
    • Coordination
    • Deep reinforcement learning
    • Multi-agent systems

    ASJC Scopus subject areas

    • Theoretical Computer Science
    • Computer Science(all)

    Cite this

    Diallo, E. A. O., & Sugawara, T. (2018). Learning strategic group formation for coordinated behavior in adversarial multi-agent with double DQN. In N. Oren, Y. Sakurai, I. Noda, T. Cao Son, T. Miller, & B. T. Savarimuthu (Eds.), PRIMA 2018: Principles and Practice of Multi-Agent Systems - 21st International Conference, 2018, Proceedings (pp. 458-466). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11224 LNAI). Springer-Verlag. https://doi.org/10.1007/978-3-030-03098-8_30

    Learning strategic group formation for coordinated behavior in adversarial multi-agent with double DQN. / Diallo, Elhadji Amadou Oury; Sugawara, Toshiharu.

    PRIMA 2018: Principles and Practice of Multi-Agent Systems - 21st International Conference, 2018, Proceedings. ed. / Nir Oren; Yuko Sakurai; Itsuki Noda; Tran Cao Son; Tim Miller; Bastin Tony Savarimuthu. Springer-Verlag, 2018. p. 458-466 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11224 LNAI).

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Diallo, EAO & Sugawara, T 2018, Learning strategic group formation for coordinated behavior in adversarial multi-agent with double DQN. in N Oren, Y Sakurai, I Noda, T Cao Son, T Miller & BT Savarimuthu (eds), PRIMA 2018: Principles and Practice of Multi-Agent Systems - 21st International Conference, 2018, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11224 LNAI, Springer-Verlag, pp. 458-466, 21st International Conference on Principles and Practice of Multi-Agent Systems, PRIMA 2018, Tokyo, Japan, 18/10/29. https://doi.org/10.1007/978-3-030-03098-8_30
    Diallo EAO, Sugawara T. Learning strategic group formation for coordinated behavior in adversarial multi-agent with double DQN. In Oren N, Sakurai Y, Noda I, Cao Son T, Miller T, Savarimuthu BT, editors, PRIMA 2018: Principles and Practice of Multi-Agent Systems - 21st International Conference, 2018, Proceedings. Springer-Verlag. 2018. p. 458-466. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-03098-8_30
    Diallo, Elhadji Amadou Oury ; Sugawara, Toshiharu. / Learning strategic group formation for coordinated behavior in adversarial multi-agent with double DQN. PRIMA 2018: Principles and Practice of Multi-Agent Systems - 21st International Conference, 2018, Proceedings. editor / Nir Oren ; Yuko Sakurai ; Itsuki Noda ; Tran Cao Son ; Tim Miller ; Bastin Tony Savarimuthu. Springer-Verlag, 2018. pp. 458-466 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
    @inproceedings{adc8b3b241d34d73b5bfe94575a66a05,
    title = "Learning strategic group formation for coordinated behavior in adversarial multi-agent with double DQN",
    abstract = "We examine whether a team of agents can learn geometric and strategic group formations by using deep reinforcement learning in adversarial multi-agent systems. This is a significant point underlying the control and coordination of multiple autonomous and intelligent agents. While there are many possible approaches to solve this problem, we are interested in fully end-to-end learning method where agents do not have any prior knowledge of the environment and its dynamics. In this paper, we propose a scalable and distributed double DQN framework to train adversarial multi-agent systems. We show that a large number of agents can learn to cooperatively move, attack and defend themselves in various geometric formations and battle tactics like encirclement, guerrilla warfare, frontal attack, flanking maneuver, and so on. We finally show that agents create an emergent and collective flocking behaviors by using local views from the environment only.",
    keywords = "Collective intelligence, Cooperation, Coordination, Deep reinforcement learning, Multi-agent systems",
    author = "Diallo, {Elhadji Amadou Oury} and Toshiharu Sugawara",
    year = "2018",
    month = "1",
    day = "1",
    doi = "10.1007/978-3-030-03098-8_30",
    language = "English",
    isbn = "9783030030971",
    series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
    publisher = "Springer-Verlag",
    pages = "458--466",
    editor = "Nir Oren and Yuko Sakurai and Itsuki Noda and {Cao Son}, Tran and Tim Miller and Savarimuthu, {Bastin Tony}",
    booktitle = "PRIMA 2018",

    }

    TY - GEN

    T1 - Learning strategic group formation for coordinated behavior in adversarial multi-agent with double DQN

    AU - Diallo, Elhadji Amadou Oury

    AU - Sugawara, Toshiharu

    PY - 2018/1/1

    Y1 - 2018/1/1

    N2 - We examine whether a team of agents can learn geometric and strategic group formations by using deep reinforcement learning in adversarial multi-agent systems. This is a significant point underlying the control and coordination of multiple autonomous and intelligent agents. While there are many possible approaches to solve this problem, we are interested in fully end-to-end learning method where agents do not have any prior knowledge of the environment and its dynamics. In this paper, we propose a scalable and distributed double DQN framework to train adversarial multi-agent systems. We show that a large number of agents can learn to cooperatively move, attack and defend themselves in various geometric formations and battle tactics like encirclement, guerrilla warfare, frontal attack, flanking maneuver, and so on. We finally show that agents create an emergent and collective flocking behaviors by using local views from the environment only.

    AB - We examine whether a team of agents can learn geometric and strategic group formations by using deep reinforcement learning in adversarial multi-agent systems. This is a significant point underlying the control and coordination of multiple autonomous and intelligent agents. While there are many possible approaches to solve this problem, we are interested in fully end-to-end learning method where agents do not have any prior knowledge of the environment and its dynamics. In this paper, we propose a scalable and distributed double DQN framework to train adversarial multi-agent systems. We show that a large number of agents can learn to cooperatively move, attack and defend themselves in various geometric formations and battle tactics like encirclement, guerrilla warfare, frontal attack, flanking maneuver, and so on. We finally show that agents create an emergent and collective flocking behaviors by using local views from the environment only.

    KW - Collective intelligence

    KW - Cooperation

    KW - Coordination

    KW - Deep reinforcement learning

    KW - Multi-agent systems

    UR - http://www.scopus.com/inward/record.url?scp=85056447233&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85056447233&partnerID=8YFLogxK

    U2 - 10.1007/978-3-030-03098-8_30

    DO - 10.1007/978-3-030-03098-8_30

    M3 - Conference contribution

    AN - SCOPUS:85056447233

    SN - 9783030030971

    T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

    SP - 458

    EP - 466

    BT - PRIMA 2018

    A2 - Oren, Nir

    A2 - Sakurai, Yuko

    A2 - Noda, Itsuki

    A2 - Cao Son, Tran

    A2 - Miller, Tim

    A2 - Savarimuthu, Bastin Tony

    PB - Springer-Verlag

    ER -