Fully automated generation of question-answer pairs for scripted virtual instruction

Pascal Kuyten, Timothy Bickmore, Svetlana Stoyanchev, Paul Piwek, Helmut Prendinger, Mitsuru Ishizuka

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

We introduce a novel approach for automatically generating a virtual instructor from textual input only. Our fully implemented system first analyzes the rhetorical structure of the input text and then creates various question-answer pairs using patterns. These patterns have been derived from correlations found between rhetorical structure of monologue texts and question-answer pairs in the corresponding dialogues. A selection of the candidate pairs is verbalized into a diverse collection of question-answer pairs. Finally the system compiles the collection of question-answer pairs into scripts for a virtual instructor. Our end-to-end system presents questions in pre-fixed order and the agent answers them. Our system was evaluated with a group of twenty-four subjects. The evaluation was conducted using three informed consent documents of clinical trials from the domain of colon cancer. Each of the documents was explained by a virtual instructor using 1) text, 2) text and agent monologue, and 3) text and agent performing question-answering. Results show that an agent explaining an informed consent document did not provide significantly better comprehension scores, but did score higher on satisfaction, compared to two control conditions.

Original languageEnglish
Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Pages1-14
Number of pages14
Volume7502 LNAI
DOIs
Publication statusPublished - 2012
Externally publishedYes
Event12th International Conference on Intelligent Virtual Agents, IVA 2012 - Santa Cruz, CA
Duration: 2012 Sep 122012 Sep 14

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume7502 LNAI
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other12th International Conference on Intelligent Virtual Agents, IVA 2012
CitySanta Cruz, CA
Period12/9/1212/9/14

Fingerprint

Question Answering
Clinical Trials
Cancer
Text
Evaluation
Dialogue

Keywords

  • Dialogue Generation
  • Medical Documents
  • Rhetorical Structure Theory

ASJC Scopus subject areas

  • Computer Science(all)
  • Theoretical Computer Science

Cite this

Kuyten, P., Bickmore, T., Stoyanchev, S., Piwek, P., Prendinger, H., & Ishizuka, M. (2012). Fully automated generation of question-answer pairs for scripted virtual instruction. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 7502 LNAI, pp. 1-14). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7502 LNAI). https://doi.org/10.1007/978-3-642-33197-8-1

Fully automated generation of question-answer pairs for scripted virtual instruction. / Kuyten, Pascal; Bickmore, Timothy; Stoyanchev, Svetlana; Piwek, Paul; Prendinger, Helmut; Ishizuka, Mitsuru.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 7502 LNAI 2012. p. 1-14 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7502 LNAI).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Kuyten, P, Bickmore, T, Stoyanchev, S, Piwek, P, Prendinger, H & Ishizuka, M 2012, Fully automated generation of question-answer pairs for scripted virtual instruction. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). vol. 7502 LNAI, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 7502 LNAI, pp. 1-14, 12th International Conference on Intelligent Virtual Agents, IVA 2012, Santa Cruz, CA, 12/9/12. https://doi.org/10.1007/978-3-642-33197-8-1
Kuyten P, Bickmore T, Stoyanchev S, Piwek P, Prendinger H, Ishizuka M. Fully automated generation of question-answer pairs for scripted virtual instruction. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 7502 LNAI. 2012. p. 1-14. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-642-33197-8-1
Kuyten, Pascal ; Bickmore, Timothy ; Stoyanchev, Svetlana ; Piwek, Paul ; Prendinger, Helmut ; Ishizuka, Mitsuru. / Fully automated generation of question-answer pairs for scripted virtual instruction. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 7502 LNAI 2012. pp. 1-14 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{a4008d6f680b46c5a51b398656ea8641,
title = "Fully automated generation of question-answer pairs for scripted virtual instruction",
abstract = "We introduce a novel approach for automatically generating a virtual instructor from textual input only. Our fully implemented system first analyzes the rhetorical structure of the input text and then creates various question-answer pairs using patterns. These patterns have been derived from correlations found between rhetorical structure of monologue texts and question-answer pairs in the corresponding dialogues. A selection of the candidate pairs is verbalized into a diverse collection of question-answer pairs. Finally the system compiles the collection of question-answer pairs into scripts for a virtual instructor. Our end-to-end system presents questions in pre-fixed order and the agent answers them. Our system was evaluated with a group of twenty-four subjects. The evaluation was conducted using three informed consent documents of clinical trials from the domain of colon cancer. Each of the documents was explained by a virtual instructor using 1) text, 2) text and agent monologue, and 3) text and agent performing question-answering. Results show that an agent explaining an informed consent document did not provide significantly better comprehension scores, but did score higher on satisfaction, compared to two control conditions.",
keywords = "Dialogue Generation, Medical Documents, Rhetorical Structure Theory",
author = "Pascal Kuyten and Timothy Bickmore and Svetlana Stoyanchev and Paul Piwek and Helmut Prendinger and Mitsuru Ishizuka",
year = "2012",
doi = "10.1007/978-3-642-33197-8-1",
language = "English",
isbn = "9783642331961",
volume = "7502 LNAI",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
pages = "1--14",
booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

}

TY - GEN

T1 - Fully automated generation of question-answer pairs for scripted virtual instruction

AU - Kuyten, Pascal

AU - Bickmore, Timothy

AU - Stoyanchev, Svetlana

AU - Piwek, Paul

AU - Prendinger, Helmut

AU - Ishizuka, Mitsuru

PY - 2012

Y1 - 2012

N2 - We introduce a novel approach for automatically generating a virtual instructor from textual input only. Our fully implemented system first analyzes the rhetorical structure of the input text and then creates various question-answer pairs using patterns. These patterns have been derived from correlations found between rhetorical structure of monologue texts and question-answer pairs in the corresponding dialogues. A selection of the candidate pairs is verbalized into a diverse collection of question-answer pairs. Finally the system compiles the collection of question-answer pairs into scripts for a virtual instructor. Our end-to-end system presents questions in pre-fixed order and the agent answers them. Our system was evaluated with a group of twenty-four subjects. The evaluation was conducted using three informed consent documents of clinical trials from the domain of colon cancer. Each of the documents was explained by a virtual instructor using 1) text, 2) text and agent monologue, and 3) text and agent performing question-answering. Results show that an agent explaining an informed consent document did not provide significantly better comprehension scores, but did score higher on satisfaction, compared to two control conditions.

AB - We introduce a novel approach for automatically generating a virtual instructor from textual input only. Our fully implemented system first analyzes the rhetorical structure of the input text and then creates various question-answer pairs using patterns. These patterns have been derived from correlations found between rhetorical structure of monologue texts and question-answer pairs in the corresponding dialogues. A selection of the candidate pairs is verbalized into a diverse collection of question-answer pairs. Finally the system compiles the collection of question-answer pairs into scripts for a virtual instructor. Our end-to-end system presents questions in pre-fixed order and the agent answers them. Our system was evaluated with a group of twenty-four subjects. The evaluation was conducted using three informed consent documents of clinical trials from the domain of colon cancer. Each of the documents was explained by a virtual instructor using 1) text, 2) text and agent monologue, and 3) text and agent performing question-answering. Results show that an agent explaining an informed consent document did not provide significantly better comprehension scores, but did score higher on satisfaction, compared to two control conditions.

KW - Dialogue Generation

KW - Medical Documents

KW - Rhetorical Structure Theory

UR - http://www.scopus.com/inward/record.url?scp=84867519134&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84867519134&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-33197-8-1

DO - 10.1007/978-3-642-33197-8-1

M3 - Conference contribution

AN - SCOPUS:84867519134

SN - 9783642331961

VL - 7502 LNAI

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 1

EP - 14

BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

ER -