A framework to co-optimize task and social dialogue policies using Reinforcement Learning

Florian Pecune, Stacy Marsella

研究成果: Conference contribution

1 被引用数 (Scopus)

抄録

One of the main challenges for conversational agents is to select the optimal dialogue policy based on the state of the interaction. This challenge becomes even harder when the conversational agent not only has to achieve a specific task, but also aims at building rapport. Although some work already tried to tackle this challenge using a Reinforcement Learning (RL) approach, they tend to consider one single optimal policy for all the users, regardless of their conversational goals. In this work, we describe a framework that allows us to build a RL-based agent able to adapt its dialogue policy depending on its user's conversational goals. After we build a rule-based agent and a user simulator communicating at the dialog-act level, we crowdsource the surface sentences authoring for both the simulated users and the agent, which allow us to generate a dataset of interactions in natural language. Then, we annotate each of these interactions with a single rapport score and analyze the links between simulated users' conversational goals, agent conversational policies, and rapport. Our results show that rapport was higher when both or none of the interlocutors tried to build rapport. We use this result to inform the design of a social reward function, and we rely on this social reward function to train a RL-based agent using an hybrid approach of supervised learning and reinforcement learning. We evaluate our approach by comparing two different versions of our RL-based agent: one that takes users' conversational goals into account and another that does not. The results show that an agent adapting its dialogue policy depending on users' conversational goals performs better.

本文言語English
ホスト出版物のタイトルProceedings of the 20th ACM International Conference on Intelligent Virtual Agents, IVA 2020
出版社Association for Computing Machinery, Inc
ISBN(電子版)9781450375863
DOI
出版ステータスPublished - 2020 10月 20
外部発表はい
イベント20th ACM International Conference on Intelligent Virtual Agents, IVA 2020 - Virtual, Online, United Kingdom
継続期間: 2020 10月 202020 10月 22

出版物シリーズ

名前Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents, IVA 2020

Conference

Conference20th ACM International Conference on Intelligent Virtual Agents, IVA 2020
国/地域United Kingdom
CityVirtual, Online
Period20/10/2020/10/22

ASJC Scopus subject areas

  • 人工知能
  • 人間とコンピュータの相互作用

フィンガープリント

「A framework to co-optimize task and social dialogue policies using Reinforcement Learning」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル