On labelling intent types for evaluating search result diversification

Tetsuya Sakai, Young In Song

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    1 Citation (Scopus)

    Abstract

    Search result diversification is important for accommodating different user needs by means of covering popular and diverse query intents within a single result page. To evaluate diversity, we believe that it is important to consider the distinction between informational and navigational intents, as users would not want redundant information especially for navigational intents. In this study, we conduct intent type-sensitive diversity evaluation based on both top-down labelling, which labels each intent as either navigational or informational a priori, and bottom-up labelling, which labels each intent based on whether a "navigational relevant" document has actually been identified in the document collection. Our results suggest that reliable type-sensitive diversity evaluation can be conducted using the top-down approach with a clear intent labelling guideline, while ensuring that the desired URLs for navigational intents make their way into relevance assessments.

    Original languageEnglish
    Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Pages38-49
    Number of pages12
    Volume8281 LNCS
    DOIs
    Publication statusPublished - 2013
    Event9th Asia Information Retrieval Societies Conference on Information Retrieval Technology, AIRS 2013 - Singapore
    Duration: 2013 Dec 92013 Dec 11

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Volume8281 LNCS
    ISSN (Print)03029743
    ISSN (Electronic)16113349

    Other

    Other9th Asia Information Retrieval Societies Conference on Information Retrieval Technology, AIRS 2013
    CitySingapore
    Period13/12/913/12/11

    Fingerprint

    Diversification
    Labeling
    Labels
    Evaluation
    Bottom-up
    Websites
    Covering
    Query
    Evaluate

    ASJC Scopus subject areas

    • Computer Science(all)
    • Theoretical Computer Science

    Cite this

    Sakai, T., & Song, Y. I. (2013). On labelling intent types for evaluating search result diversification. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 8281 LNCS, pp. 38-49). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8281 LNCS). https://doi.org/10.1007/978-3-642-45068-6_4

    On labelling intent types for evaluating search result diversification. / Sakai, Tetsuya; Song, Young In.

    Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 8281 LNCS 2013. p. 38-49 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8281 LNCS).

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Sakai, T & Song, YI 2013, On labelling intent types for evaluating search result diversification. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). vol. 8281 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 8281 LNCS, pp. 38-49, 9th Asia Information Retrieval Societies Conference on Information Retrieval Technology, AIRS 2013, Singapore, 13/12/9. https://doi.org/10.1007/978-3-642-45068-6_4
    Sakai T, Song YI. On labelling intent types for evaluating search result diversification. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 8281 LNCS. 2013. p. 38-49. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-642-45068-6_4
    Sakai, Tetsuya ; Song, Young In. / On labelling intent types for evaluating search result diversification. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 8281 LNCS 2013. pp. 38-49 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
    @inproceedings{a81936bdbdcc4513831bdbc04d10d7b9,
    title = "On labelling intent types for evaluating search result diversification",
    abstract = "Search result diversification is important for accommodating different user needs by means of covering popular and diverse query intents within a single result page. To evaluate diversity, we believe that it is important to consider the distinction between informational and navigational intents, as users would not want redundant information especially for navigational intents. In this study, we conduct intent type-sensitive diversity evaluation based on both top-down labelling, which labels each intent as either navigational or informational a priori, and bottom-up labelling, which labels each intent based on whether a {"}navigational relevant{"} document has actually been identified in the document collection. Our results suggest that reliable type-sensitive diversity evaluation can be conducted using the top-down approach with a clear intent labelling guideline, while ensuring that the desired URLs for navigational intents make their way into relevance assessments.",
    author = "Tetsuya Sakai and Song, {Young In}",
    year = "2013",
    doi = "10.1007/978-3-642-45068-6_4",
    language = "English",
    isbn = "9783642450679",
    volume = "8281 LNCS",
    series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
    pages = "38--49",
    booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

    }

    TY - GEN

    T1 - On labelling intent types for evaluating search result diversification

    AU - Sakai, Tetsuya

    AU - Song, Young In

    PY - 2013

    Y1 - 2013

    N2 - Search result diversification is important for accommodating different user needs by means of covering popular and diverse query intents within a single result page. To evaluate diversity, we believe that it is important to consider the distinction between informational and navigational intents, as users would not want redundant information especially for navigational intents. In this study, we conduct intent type-sensitive diversity evaluation based on both top-down labelling, which labels each intent as either navigational or informational a priori, and bottom-up labelling, which labels each intent based on whether a "navigational relevant" document has actually been identified in the document collection. Our results suggest that reliable type-sensitive diversity evaluation can be conducted using the top-down approach with a clear intent labelling guideline, while ensuring that the desired URLs for navigational intents make their way into relevance assessments.

    AB - Search result diversification is important for accommodating different user needs by means of covering popular and diverse query intents within a single result page. To evaluate diversity, we believe that it is important to consider the distinction between informational and navigational intents, as users would not want redundant information especially for navigational intents. In this study, we conduct intent type-sensitive diversity evaluation based on both top-down labelling, which labels each intent as either navigational or informational a priori, and bottom-up labelling, which labels each intent based on whether a "navigational relevant" document has actually been identified in the document collection. Our results suggest that reliable type-sensitive diversity evaluation can be conducted using the top-down approach with a clear intent labelling guideline, while ensuring that the desired URLs for navigational intents make their way into relevance assessments.

    UR - http://www.scopus.com/inward/record.url?scp=84893249083&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=84893249083&partnerID=8YFLogxK

    U2 - 10.1007/978-3-642-45068-6_4

    DO - 10.1007/978-3-642-45068-6_4

    M3 - Conference contribution

    SN - 9783642450679

    VL - 8281 LNCS

    T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

    SP - 38

    EP - 49

    BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

    ER -