Evaluating search result diversity using intent hierarchies

Xiaojie Wang, Zhicheng Dou, Tetsuya Sakai, Ji Rong Wen

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    15 Citations (Scopus)

    Abstract

    Search result diversification aims at returning diversified document lists to cover different user intents for ambiguous or broad queries. Existing diversity measures assume that user intents are independent or exclusive, and do not consider the relationships among the intents. In this paper, we introduce intent hierarchies to model the relationships among intents. Based on intent hierarchies, we propose several hierarchical measures that can consider the relationships among intents. We demonstrate the feasibility of hierarchical measures by using a new test collection based on TREC Web Track 2009-2013 diversity test collections. Our main experimental findings are: (1) Hierarchical measures are generally more discriminative and intuitive than existing measures using flat lists of intents; (2) When the queries have multilayer intent hierarchies, hierarchical measures are less correlated to existing measures, but can get more improvement in discriminative power; (3) Hierarchical measures are more intuitive in terms of diversity or relevance. The hierarchical measures using the whole intent hierarchies are more intuitive than only using the leaf nodes in terms of diversity and relevance.

    Original languageEnglish
    Title of host publicationSIGIR 2016 - Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval
    PublisherAssociation for Computing Machinery, Inc
    Pages415-424
    Number of pages10
    ISBN (Electronic)9781450342902
    DOIs
    Publication statusPublished - 2016 Jul 7
    Event39th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2016 - Pisa, Italy
    Duration: 2016 Jul 172016 Jul 21

    Other

    Other39th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2016
    CountryItaly
    CityPisa
    Period16/7/1716/7/21

    Keywords

    • Ambiguity
    • Diversity
    • Evaluation
    • Hierarchy
    • Novelty

    ASJC Scopus subject areas

    • Information Systems
    • Software

    Fingerprint Dive into the research topics of 'Evaluating search result diversity using intent hierarchies'. Together they form a unique fingerprint.

  • Cite this

    Wang, X., Dou, Z., Sakai, T., & Wen, J. R. (2016). Evaluating search result diversity using intent hierarchies. In SIGIR 2016 - Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 415-424). Association for Computing Machinery, Inc. https://doi.org/10.1145/2911451.2911497