Evaluating Image-Inspired Poetry Generation

Chao Chung Wu, Ruihua Song, Tetsuya Sakai, Wen Feng Cheng, Xing Xie, Shou De Lin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Creative natural language generation, such as poetry generation, writing lyrics, and storytelling, is appealing but difficult to evaluate. We take the application of image-inspired poetry generation as a showcase and investigate two problems in evaluation: (1) how to evaluate the generated text when there are no ground truths, and (2) how to evaluate nondeterministic systems that output different texts given the same input image. Regarding the first problem, we first design a judgment tool to collect ratings of a few poems for comparison with the inspiring image shown to assessors. We then propose a novelty measurement that quantifies how different a generated text is compared to a known corpus. Regarding the second problem, we experiment with different strategies to approximate evaluating multiple trials of output poems. We also use a measure for quantifying the diversity of different texts generated in response to the same input image, and discuss their merits.

Original languageEnglish
Title of host publicationNatural Language Processing and Chinese Computing - 8th CCF International Conference, NLPCC 2019, Proceedings
EditorsJie Tang, Min-Yen Kan, Dongyan Zhao, Sujian Li, Hongying Zan
PublisherSpringer
Pages539-551
Number of pages13
ISBN (Print)9783030322328
DOIs
Publication statusPublished - 2019
Event8th CCF International Conference on Natural Language Processing and Chinese Computing, NLPCC 2019 - Dunhuang, China
Duration: 2019 Oct 92019 Oct 14

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11838 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference8th CCF International Conference on Natural Language Processing and Chinese Computing, NLPCC 2019
CountryChina
CityDunhuang
Period19/10/919/10/14

Keywords

  • AI-based creation
  • Evaluation
  • Image
  • Natural language generation
  • Poetry generation

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Evaluating Image-Inspired Poetry Generation'. Together they form a unique fingerprint.

Cite this