Ranking the NTCIR systems based on multigrade relevance

研究成果: Conference contribution

9 引用 (Scopus)

抄録

At NTCIR-4, new retrieval effectiveness metrics called Q-measure and R-measure were proposed for evaluation based on multigrade relevance. This paper shows that Q-measure inherits both the reliability of noninterpolated Average Precision and the multigrade relevance capability of Average Weighted Precision through a theoretical analysis, and then verify the above claim through experiments by actually ranking the systems submitted to the NTCIR-3 CLIR Task. Our experiments confirm that the Q-measure ranking is very highly correlated with the Average Precision ranking and that it is more reliable than Average Weighted Precision.

元の言語English
ホスト出版物のタイトルLecture Notes in Computer Science
編集者S.H. Myaeng, M. Zhou, H.J. Zhang, K.-F. Wong
ページ251-262
ページ数12
3411
出版物ステータスPublished - 2005
外部発表Yes
イベントAsia Information Retrieval Symposium, AIRS 2004 - Beijing, China
継続期間: 2004 10 182004 10 20

Other

OtherAsia Information Retrieval Symposium, AIRS 2004
China
Beijing
期間04/10/1804/10/20

Fingerprint

Experiments

ASJC Scopus subject areas

  • Computer Science (miscellaneous)

これを引用

Sakai, T. (2005). Ranking the NTCIR systems based on multigrade relevance. : S. H. Myaeng, M. Zhou, H. J. Zhang, & K-F. Wong (版), Lecture Notes in Computer Science (巻 3411, pp. 251-262)

Ranking the NTCIR systems based on multigrade relevance. / Sakai, Tetsuya.

Lecture Notes in Computer Science. 版 / S.H. Myaeng; M. Zhou; H.J. Zhang; K.-F. Wong. 巻 3411 2005. p. 251-262.

研究成果: Conference contribution

Sakai, T 2005, Ranking the NTCIR systems based on multigrade relevance. : SH Myaeng, M Zhou, HJ Zhang & K-F Wong (版), Lecture Notes in Computer Science. 巻. 3411, pp. 251-262, Asia Information Retrieval Symposium, AIRS 2004, Beijing, China, 04/10/18.
Sakai T. Ranking the NTCIR systems based on multigrade relevance. : Myaeng SH, Zhou M, Zhang HJ, Wong K-F, 編集者, Lecture Notes in Computer Science. 巻 3411. 2005. p. 251-262
Sakai, Tetsuya. / Ranking the NTCIR systems based on multigrade relevance. Lecture Notes in Computer Science. 編集者 / S.H. Myaeng ; M. Zhou ; H.J. Zhang ; K.-F. Wong. 巻 3411 2005. pp. 251-262
@inproceedings{e3a29070688347bd890deca27c3d764a,
title = "Ranking the NTCIR systems based on multigrade relevance",
abstract = "At NTCIR-4, new retrieval effectiveness metrics called Q-measure and R-measure were proposed for evaluation based on multigrade relevance. This paper shows that Q-measure inherits both the reliability of noninterpolated Average Precision and the multigrade relevance capability of Average Weighted Precision through a theoretical analysis, and then verify the above claim through experiments by actually ranking the systems submitted to the NTCIR-3 CLIR Task. Our experiments confirm that the Q-measure ranking is very highly correlated with the Average Precision ranking and that it is more reliable than Average Weighted Precision.",
author = "Tetsuya Sakai",
year = "2005",
language = "English",
volume = "3411",
pages = "251--262",
editor = "S.H. Myaeng and M. Zhou and H.J. Zhang and K.-F. Wong",
booktitle = "Lecture Notes in Computer Science",

}

TY - GEN

T1 - Ranking the NTCIR systems based on multigrade relevance

AU - Sakai, Tetsuya

PY - 2005

Y1 - 2005

N2 - At NTCIR-4, new retrieval effectiveness metrics called Q-measure and R-measure were proposed for evaluation based on multigrade relevance. This paper shows that Q-measure inherits both the reliability of noninterpolated Average Precision and the multigrade relevance capability of Average Weighted Precision through a theoretical analysis, and then verify the above claim through experiments by actually ranking the systems submitted to the NTCIR-3 CLIR Task. Our experiments confirm that the Q-measure ranking is very highly correlated with the Average Precision ranking and that it is more reliable than Average Weighted Precision.

AB - At NTCIR-4, new retrieval effectiveness metrics called Q-measure and R-measure were proposed for evaluation based on multigrade relevance. This paper shows that Q-measure inherits both the reliability of noninterpolated Average Precision and the multigrade relevance capability of Average Weighted Precision through a theoretical analysis, and then verify the above claim through experiments by actually ranking the systems submitted to the NTCIR-3 CLIR Task. Our experiments confirm that the Q-measure ranking is very highly correlated with the Average Precision ranking and that it is more reliable than Average Weighted Precision.

UR - http://www.scopus.com/inward/record.url?scp=24344471313&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=24344471313&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:24344471313

VL - 3411

SP - 251

EP - 262

BT - Lecture Notes in Computer Science

A2 - Myaeng, S.H.

A2 - Zhou, M.

A2 - Zhang, H.J.

A2 - Wong, K.-F.

ER -