Evaluating search result diversity using intent hierarchies

Xiaojie Wang, Zhicheng Dou, Tetsuya Sakai, Ji Rong Wen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

21 Citations (Scopus)

Abstract

Search result diversification aims at returning diversified document lists to cover different user intents for ambiguous or broad queries. Existing diversity measures assume that user intents are independent or exclusive, and do not consider the relationships among the intents. In this paper, we introduce intent hierarchies to model the relationships among intents. Based on intent hierarchies, we propose several hierarchical measures that can consider the relationships among intents. We demonstrate the feasibility of hierarchical measures by using a new test collection based on TREC Web Track 2009-2013 diversity test collections. Our main experimental findings are: (1) Hierarchical measures are generally more discriminative and intuitive than existing measures using flat lists of intents; (2) When the queries have multilayer intent hierarchies, hierarchical measures are less correlated to existing measures, but can get more improvement in discriminative power; (3) Hierarchical measures are more intuitive in terms of diversity or relevance. The hierarchical measures using the whole intent hierarchies are more intuitive than only using the leaf nodes in terms of diversity and relevance.

Original languageEnglish
Title of host publicationSIGIR 2016 - Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval
PublisherAssociation for Computing Machinery, Inc
Pages415-424
Number of pages10
ISBN (Electronic)9781450342902
DOIs
Publication statusPublished - 2016 Jul 7
Event39th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2016 - Pisa, Italy
Duration: 2016 Jul 172016 Jul 21

Publication series

NameSIGIR 2016 - Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval

Other

Other39th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2016
Country/TerritoryItaly
CityPisa
Period16/7/1716/7/21

Keywords

  • Ambiguity
  • Diversity
  • Evaluation
  • Hierarchy
  • Novelty

ASJC Scopus subject areas

  • Information Systems
  • Software

Fingerprint

Dive into the research topics of 'Evaluating search result diversity using intent hierarchies'. Together they form a unique fingerprint.

Cite this