抄録
At NTCIR-4, new retrieval effectiveness metrics called Q-measure and R-measure were proposed for evaluation based on multigrade relevance. This paper shows that Q-measure inherits both the reliability of noninterpolated Average Precision and the multigrade relevance capability of Average Weighted Precision through a theoretical analysis, and then verify the above claim through experiments by actually ranking the systems submitted to the NTCIR-3 CLIR Task. Our experiments confirm that the Q-measure ranking is very highly correlated with the Average Precision ranking and that it is more reliable than Average Weighted Precision.
| 元の言語 | English |
|---|---|
| ホスト出版物のタイトル | Lecture Notes in Computer Science |
| 編集者 | S.H. Myaeng, M. Zhou, H.J. Zhang, K.-F. Wong |
| ページ | 251-262 |
| ページ数 | 12 |
| 巻 | 3411 |
| 出版物ステータス | Published - 2005 |
| 外部発表 | Yes |
| イベント | Asia Information Retrieval Symposium, AIRS 2004 - Beijing, China 継続期間: 2004 10 18 → 2004 10 20 |
Other
| Other | Asia Information Retrieval Symposium, AIRS 2004 |
|---|---|
| 国 | China |
| 市 | Beijing |
| 期間 | 04/10/18 → 04/10/20 |
Fingerprint
ASJC Scopus subject areas
- Computer Science (miscellaneous)
これを引用
Ranking the NTCIR systems based on multigrade relevance. / Sakai, Tetsuya.
Lecture Notes in Computer Science. 版 / S.H. Myaeng; M. Zhou; H.J. Zhang; K.-F. Wong. 巻 3411 2005. p. 251-262.研究成果: Conference contribution
}
TY - GEN
T1 - Ranking the NTCIR systems based on multigrade relevance
AU - Sakai, Tetsuya
PY - 2005
Y1 - 2005
N2 - At NTCIR-4, new retrieval effectiveness metrics called Q-measure and R-measure were proposed for evaluation based on multigrade relevance. This paper shows that Q-measure inherits both the reliability of noninterpolated Average Precision and the multigrade relevance capability of Average Weighted Precision through a theoretical analysis, and then verify the above claim through experiments by actually ranking the systems submitted to the NTCIR-3 CLIR Task. Our experiments confirm that the Q-measure ranking is very highly correlated with the Average Precision ranking and that it is more reliable than Average Weighted Precision.
AB - At NTCIR-4, new retrieval effectiveness metrics called Q-measure and R-measure were proposed for evaluation based on multigrade relevance. This paper shows that Q-measure inherits both the reliability of noninterpolated Average Precision and the multigrade relevance capability of Average Weighted Precision through a theoretical analysis, and then verify the above claim through experiments by actually ranking the systems submitted to the NTCIR-3 CLIR Task. Our experiments confirm that the Q-measure ranking is very highly correlated with the Average Precision ranking and that it is more reliable than Average Weighted Precision.
UR - http://www.scopus.com/inward/record.url?scp=24344471313&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=24344471313&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:24344471313
VL - 3411
SP - 251
EP - 262
BT - Lecture Notes in Computer Science
A2 - Myaeng, S.H.
A2 - Zhou, M.
A2 - Zhang, H.J.
A2 - Wong, K.-F.
ER -