SvgAI - Training Methods Analysis of Artificial Intelligent Agent to use SVG Editor

Anh H. Dang, Wataru Kameyama

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Deep reinforcement learning has been successfully used to train artificial intelligent (AI) agents, which outperforms humans in many tasks. The objective of this research is to train an AI agent to draw SVG images by using scalable vector graphic (SVG) editor with deep reinforcement learning, where the AI agent is to draw SVG images that are similar as much as possible to the given target raster images. In this paper, we propose framework to train the AI agent by value-function based Q-learning and policy-gradient based learning methods. With Q-learning based method, we find that it is crucial to distinguish the action space into two sets to apply a different exploration policy on each set during the training process. Evaluations show that our proposed dual ϵ-greedy exploration policy greatly stabilizes the training process and increases the accuracy of the AI agent. On the other hand, policy-gradient based training does not depend on external reward function. However, it is hard to implement especially in the environment with a large action space. To overcome this difficulty, we propose a strategy similar to the dynamic programming method to allow the agent to generate training samples by itself. In our evaluation, the highest score is archived by the agent trained by this proposed method. SVG images produced by the proposed AI agent have also superior quality compared to popular raster-to-SVG conversion softwares.

Original languageEnglish
Title of host publication21st International Conference on Advanced Communication Technology
Subtitle of host publicationICT for 4th Industrial Revolution!, ICACT 2019 - Proceeding
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1159-1166
Number of pages8
ISBN (Electronic)9791188428021
DOIs
Publication statusPublished - 2019 Apr 29
Event21st International Conference on Advanced Communication Technology, ICACT 2019 - Pyeongchang, Korea, Republic of
Duration: 2019 Feb 172019 Feb 20

Publication series

NameInternational Conference on Advanced Communication Technology, ICACT
Volume2019-February
ISSN (Print)1738-9445

Conference

Conference21st International Conference on Advanced Communication Technology, ICACT 2019
CountryKorea, Republic of
CityPyeongchang
Period19/2/1719/2/20

Fingerprint

Intelligent agents
Reinforcement learning
Dynamic programming

Keywords

  • Exploration Policy
  • Q-learning
  • Reinforcement Learning
  • SVG

ASJC Scopus subject areas

  • Electrical and Electronic Engineering

Cite this

Dang, A. H., & Kameyama, W. (2019). SvgAI - Training Methods Analysis of Artificial Intelligent Agent to use SVG Editor. In 21st International Conference on Advanced Communication Technology: ICT for 4th Industrial Revolution!, ICACT 2019 - Proceeding (pp. 1159-1166). [8702041] (International Conference on Advanced Communication Technology, ICACT; Vol. 2019-February). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.23919/ICACT.2019.8702041

SvgAI - Training Methods Analysis of Artificial Intelligent Agent to use SVG Editor. / Dang, Anh H.; Kameyama, Wataru.

21st International Conference on Advanced Communication Technology: ICT for 4th Industrial Revolution!, ICACT 2019 - Proceeding. Institute of Electrical and Electronics Engineers Inc., 2019. p. 1159-1166 8702041 (International Conference on Advanced Communication Technology, ICACT; Vol. 2019-February).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Dang, AH & Kameyama, W 2019, SvgAI - Training Methods Analysis of Artificial Intelligent Agent to use SVG Editor. in 21st International Conference on Advanced Communication Technology: ICT for 4th Industrial Revolution!, ICACT 2019 - Proceeding., 8702041, International Conference on Advanced Communication Technology, ICACT, vol. 2019-February, Institute of Electrical and Electronics Engineers Inc., pp. 1159-1166, 21st International Conference on Advanced Communication Technology, ICACT 2019, Pyeongchang, Korea, Republic of, 19/2/17. https://doi.org/10.23919/ICACT.2019.8702041
Dang AH, Kameyama W. SvgAI - Training Methods Analysis of Artificial Intelligent Agent to use SVG Editor. In 21st International Conference on Advanced Communication Technology: ICT for 4th Industrial Revolution!, ICACT 2019 - Proceeding. Institute of Electrical and Electronics Engineers Inc. 2019. p. 1159-1166. 8702041. (International Conference on Advanced Communication Technology, ICACT). https://doi.org/10.23919/ICACT.2019.8702041
Dang, Anh H. ; Kameyama, Wataru. / SvgAI - Training Methods Analysis of Artificial Intelligent Agent to use SVG Editor. 21st International Conference on Advanced Communication Technology: ICT for 4th Industrial Revolution!, ICACT 2019 - Proceeding. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 1159-1166 (International Conference on Advanced Communication Technology, ICACT).
@inproceedings{a09f1d1c3a95470085967d9ef8c3d090,
title = "SvgAI - Training Methods Analysis of Artificial Intelligent Agent to use SVG Editor",
abstract = "Deep reinforcement learning has been successfully used to train artificial intelligent (AI) agents, which outperforms humans in many tasks. The objective of this research is to train an AI agent to draw SVG images by using scalable vector graphic (SVG) editor with deep reinforcement learning, where the AI agent is to draw SVG images that are similar as much as possible to the given target raster images. In this paper, we propose framework to train the AI agent by value-function based Q-learning and policy-gradient based learning methods. With Q-learning based method, we find that it is crucial to distinguish the action space into two sets to apply a different exploration policy on each set during the training process. Evaluations show that our proposed dual ϵ-greedy exploration policy greatly stabilizes the training process and increases the accuracy of the AI agent. On the other hand, policy-gradient based training does not depend on external reward function. However, it is hard to implement especially in the environment with a large action space. To overcome this difficulty, we propose a strategy similar to the dynamic programming method to allow the agent to generate training samples by itself. In our evaluation, the highest score is archived by the agent trained by this proposed method. SVG images produced by the proposed AI agent have also superior quality compared to popular raster-to-SVG conversion softwares.",
keywords = "Exploration Policy, Q-learning, Reinforcement Learning, SVG",
author = "Dang, {Anh H.} and Wataru Kameyama",
year = "2019",
month = "4",
day = "29",
doi = "10.23919/ICACT.2019.8702041",
language = "English",
series = "International Conference on Advanced Communication Technology, ICACT",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "1159--1166",
booktitle = "21st International Conference on Advanced Communication Technology",

}

TY - GEN

T1 - SvgAI - Training Methods Analysis of Artificial Intelligent Agent to use SVG Editor

AU - Dang, Anh H.

AU - Kameyama, Wataru

PY - 2019/4/29

Y1 - 2019/4/29

N2 - Deep reinforcement learning has been successfully used to train artificial intelligent (AI) agents, which outperforms humans in many tasks. The objective of this research is to train an AI agent to draw SVG images by using scalable vector graphic (SVG) editor with deep reinforcement learning, where the AI agent is to draw SVG images that are similar as much as possible to the given target raster images. In this paper, we propose framework to train the AI agent by value-function based Q-learning and policy-gradient based learning methods. With Q-learning based method, we find that it is crucial to distinguish the action space into two sets to apply a different exploration policy on each set during the training process. Evaluations show that our proposed dual ϵ-greedy exploration policy greatly stabilizes the training process and increases the accuracy of the AI agent. On the other hand, policy-gradient based training does not depend on external reward function. However, it is hard to implement especially in the environment with a large action space. To overcome this difficulty, we propose a strategy similar to the dynamic programming method to allow the agent to generate training samples by itself. In our evaluation, the highest score is archived by the agent trained by this proposed method. SVG images produced by the proposed AI agent have also superior quality compared to popular raster-to-SVG conversion softwares.

AB - Deep reinforcement learning has been successfully used to train artificial intelligent (AI) agents, which outperforms humans in many tasks. The objective of this research is to train an AI agent to draw SVG images by using scalable vector graphic (SVG) editor with deep reinforcement learning, where the AI agent is to draw SVG images that are similar as much as possible to the given target raster images. In this paper, we propose framework to train the AI agent by value-function based Q-learning and policy-gradient based learning methods. With Q-learning based method, we find that it is crucial to distinguish the action space into two sets to apply a different exploration policy on each set during the training process. Evaluations show that our proposed dual ϵ-greedy exploration policy greatly stabilizes the training process and increases the accuracy of the AI agent. On the other hand, policy-gradient based training does not depend on external reward function. However, it is hard to implement especially in the environment with a large action space. To overcome this difficulty, we propose a strategy similar to the dynamic programming method to allow the agent to generate training samples by itself. In our evaluation, the highest score is archived by the agent trained by this proposed method. SVG images produced by the proposed AI agent have also superior quality compared to popular raster-to-SVG conversion softwares.

KW - Exploration Policy

KW - Q-learning

KW - Reinforcement Learning

KW - SVG

UR - http://www.scopus.com/inward/record.url?scp=85065642084&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85065642084&partnerID=8YFLogxK

U2 - 10.23919/ICACT.2019.8702041

DO - 10.23919/ICACT.2019.8702041

M3 - Conference contribution

AN - SCOPUS:85065642084

T3 - International Conference on Advanced Communication Technology, ICACT

SP - 1159

EP - 1166

BT - 21st International Conference on Advanced Communication Technology

PB - Institute of Electrical and Electronics Engineers Inc.

ER -