Towards Automatic Evaluation of Reused Answers in Community Question Answering

Hsin Wen Liu, Sumio Fujita, Tetsuya Sakai

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We consider the problem of reused answer retrieval for community question answering (CQA): given a question q, retrieve answers (Formula presented) posted in response to other questions qi(≠ q), where (Formula presented) serves as an answer to q. While previous work evaluated this task by manually annotating the relationship between q and (Formula presented), this approach does not scale for large-scale CQA sites. We therefore explore an automatic evaluation method for reused answer retrieval, which computes nDCG by defining the gain value of each retrieved answer as a ROUGE score that treats the original answers to q as gold summaries. Our answer retrieval experiment suggests that effective reused answer retrieval systems may not be the same as effective gold answer retrieval systems. We provide case studies to discuss the benefits and limitations of our approach.

Original languageEnglish
Title of host publicationInformation Retrieval Technology - 15th Asia Information Retrieval Societies Conference, AIRS 2019, Proceedings
EditorsFu Lee Wang, Haoran Xie, Wai Lam, Aixin Sun, Lun-Wei Ku, Tianyong Hao, Wei Chen, Tak-Lam Wong, Xiaohui Tao
PublisherSpringer
Pages3-9
Number of pages7
ISBN (Print)9783030428341
DOIs
Publication statusPublished - 2020
Event15th Asia Information Retrieval Societies Conference, AIRS 2019 - Kowloon, Hong Kong
Duration: 2019 Nov 72019 Nov 9

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12004 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference15th Asia Information Retrieval Societies Conference, AIRS 2019
CountryHong Kong
CityKowloon
Period19/11/719/11/9

Keywords

  • Answer retrieval
  • Community question answering
  • ROUGE

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Towards Automatic Evaluation of Reused Answers in Community Question Answering'. Together they form a unique fingerprint.

  • Cite this

    Liu, H. W., Fujita, S., & Sakai, T. (2020). Towards Automatic Evaluation of Reused Answers in Community Question Answering. In F. L. Wang, H. Xie, W. Lam, A. Sun, L-W. Ku, T. Hao, W. Chen, T-L. Wong, & X. Tao (Eds.), Information Retrieval Technology - 15th Asia Information Retrieval Societies Conference, AIRS 2019, Proceedings (pp. 3-9). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12004 LNCS). Springer. https://doi.org/10.1007/978-3-030-42835-8_1