Bootstrap-based comparisons of IR metrics for finding one relevant document

Tetsuya Sakai*

*この研究の対応する著者

研究成果: Conference contribution

11 被引用数 (Scopus)

抄録

This paper compares the sensitivity of IR metrics designed for the task of finding one relevant document, using a method recently proposed at SIGIR 2006. The metrics are: P+-measure, P-measure, O-measure, Normalised Weighted Reciprocal Rank (NWRR) and Reciprocal Rank (RR). All of them except for RR can handle graded relevance. Unlike the ad hoc (but nevertheless useful) "swap" method proposed by Voorhees and Buckley, the new method derives the sensitivity and the performance difference required to guarantee a given significance level directly from Bootstrap Hypothesis Tests. We use four data sets from NTCIR to show that, according to this method, "P( +)-measure ≥ O-measure ≥ NWRR ≥ RR" generally holds, where "≥" means "is at least as sensitive as". These results generalise and reinforce previously reported ones based on the swap method. Therefore, we recommend the use of P(+)-measure and O-measure for practical tasks such as known-item search where recall is either unimportant or immeasurable.

本文言語English
ホスト出版物のタイトルInformation Retrieval Technology - Third Asia Information Retrieval Symposium, AIRS 2006, Proceedings
出版社Springer Verlag
ページ374-389
ページ数16
ISBN(印刷版)3540457801, 9783540457800
出版ステータスPublished - 2006 1 1
外部発表はい
イベント3rd Asia Information Retrieval Symposium, AIRS 2006 - Singapore, Singapore
継続期間: 2006 10 162006 10 18

出版物シリーズ

名前Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
4182 LNCS
ISSN(印刷版)0302-9743
ISSN(電子版)1611-3349

Conference

Conference3rd Asia Information Retrieval Symposium, AIRS 2006
国/地域Singapore
CitySingapore
Period06/10/1606/10/18

ASJC Scopus subject areas

  • 理論的コンピュータサイエンス
  • コンピュータ サイエンス(全般)

フィンガープリント

「Bootstrap-based comparisons of IR metrics for finding one relevant document」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル