On the reliability and intuitiveness of aggregated search metrics

Ke Zhou, Mounia Lalmas, Tetsuya Sakai, Ronan Cummins, Joemon M. Jose

研究成果: Conference contribution

7 被引用数 (Scopus)

抄録

Aggregating search results from a variety of diverse verticals such as news, images, videos and Wikipedia into a single interface is a popular web search presentation paradigm. Although several aggregated search (AS) metrics have been proposed to evaluate AS result pages, their properties remain poorly understood. In this paper, we compare the properties of existing AS metrics under the assumptions that (1) queries may have multiple preferred verticals; (2) the likelihood of each vertical preference is available; and (3) the topical relevance assessments of results returned from each vertical is available. We compare a wide range of AS metrics on two test collections. Our main criteria of comparison are (1) discriminative power, which represents the reliability of a metric in comparing the performance of systems, and (2) intuitiveness, which represents how well a metric captures the various key aspects to be measured (i.e. various aspects of a user's perception of AS result pages). Our study shows that the AS metrics that capture key AS components (e.g., vertical selection) have several advantages over other metrics. This work sheds new lights on the further developments and applications of AS metrics.

本文言語English
ホスト出版物のタイトルCIKM 2013 - Proceedings of the 22nd ACM International Conference on Information and Knowledge Management
ページ689-698
ページ数10
DOI
出版ステータスPublished - 2013 12 11
イベント22nd ACM International Conference on Information and Knowledge Management, CIKM 2013 - San Francisco, CA, United States
継続期間: 2013 10 272013 11 1

出版物シリーズ

名前International Conference on Information and Knowledge Management, Proceedings

Conference

Conference22nd ACM International Conference on Information and Knowledge Management, CIKM 2013
CountryUnited States
CitySan Francisco, CA
Period13/10/2713/11/1

ASJC Scopus subject areas

  • Decision Sciences(all)
  • Business, Management and Accounting(all)

フィンガープリント 「On the reliability and intuitiveness of aggregated search metrics」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル