Precision-at-ten considered redundant

William Webber, Alistair Moffat, Justin Zobel, Tetsuya Sakai

Research output: Chapter in Book/Report/Conference proceedingConference contribution

20 Citations (Scopus)

Abstract

Information retrieval systems are compared using evaluation metrics, with researchers commonly reporting results for simple metrics such as precision-at-10 or reciprocal rank together with more complex ones such as average precision or discounted cumulative gain. In this paper, we demonstrate that complex metrics are as good as or better than simple metrics at predicting the performance of the simple metrics on other topics. Therefore, reporting of results from simple metrics alongside complex ones is redundant.

Original languageEnglish
Title of host publicationACM SIGIR 2008 - 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Proceedings
Pages695-696
Number of pages2
DOIs
Publication statusPublished - 2008
Externally publishedYes
Event31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM SIGIR 2008 - Singapore
Duration: 2008 Jul 202008 Jul 24

Other

Other31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM SIGIR 2008
CitySingapore
Period08/7/2008/7/24

Fingerprint

Information retrieval systems

Keywords

  • Experimentation
  • Measurement
  • Performance

ASJC Scopus subject areas

  • Information Systems
  • Software

Cite this

Webber, W., Moffat, A., Zobel, J., & Sakai, T. (2008). Precision-at-ten considered redundant. In ACM SIGIR 2008 - 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Proceedings (pp. 695-696) https://doi.org/10.1145/1390334.1390456

Precision-at-ten considered redundant. / Webber, William; Moffat, Alistair; Zobel, Justin; Sakai, Tetsuya.

ACM SIGIR 2008 - 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Proceedings. 2008. p. 695-696.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Webber, W, Moffat, A, Zobel, J & Sakai, T 2008, Precision-at-ten considered redundant. in ACM SIGIR 2008 - 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Proceedings. pp. 695-696, 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM SIGIR 2008, Singapore, 08/7/20. https://doi.org/10.1145/1390334.1390456
Webber W, Moffat A, Zobel J, Sakai T. Precision-at-ten considered redundant. In ACM SIGIR 2008 - 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Proceedings. 2008. p. 695-696 https://doi.org/10.1145/1390334.1390456
Webber, William ; Moffat, Alistair ; Zobel, Justin ; Sakai, Tetsuya. / Precision-at-ten considered redundant. ACM SIGIR 2008 - 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Proceedings. 2008. pp. 695-696
@inproceedings{34dbfe3a28ff419d89a695af9588011a,
title = "Precision-at-ten considered redundant",
abstract = "Information retrieval systems are compared using evaluation metrics, with researchers commonly reporting results for simple metrics such as precision-at-10 or reciprocal rank together with more complex ones such as average precision or discounted cumulative gain. In this paper, we demonstrate that complex metrics are as good as or better than simple metrics at predicting the performance of the simple metrics on other topics. Therefore, reporting of results from simple metrics alongside complex ones is redundant.",
keywords = "Experimentation, Measurement, Performance",
author = "William Webber and Alistair Moffat and Justin Zobel and Tetsuya Sakai",
year = "2008",
doi = "10.1145/1390334.1390456",
language = "English",
isbn = "9781605581644",
pages = "695--696",
booktitle = "ACM SIGIR 2008 - 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Proceedings",

}

TY - GEN

T1 - Precision-at-ten considered redundant

AU - Webber, William

AU - Moffat, Alistair

AU - Zobel, Justin

AU - Sakai, Tetsuya

PY - 2008

Y1 - 2008

N2 - Information retrieval systems are compared using evaluation metrics, with researchers commonly reporting results for simple metrics such as precision-at-10 or reciprocal rank together with more complex ones such as average precision or discounted cumulative gain. In this paper, we demonstrate that complex metrics are as good as or better than simple metrics at predicting the performance of the simple metrics on other topics. Therefore, reporting of results from simple metrics alongside complex ones is redundant.

AB - Information retrieval systems are compared using evaluation metrics, with researchers commonly reporting results for simple metrics such as precision-at-10 or reciprocal rank together with more complex ones such as average precision or discounted cumulative gain. In this paper, we demonstrate that complex metrics are as good as or better than simple metrics at predicting the performance of the simple metrics on other topics. Therefore, reporting of results from simple metrics alongside complex ones is redundant.

KW - Experimentation

KW - Measurement

KW - Performance

UR - http://www.scopus.com/inward/record.url?scp=57349093460&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=57349093460&partnerID=8YFLogxK

U2 - 10.1145/1390334.1390456

DO - 10.1145/1390334.1390456

M3 - Conference contribution

AN - SCOPUS:57349093460

SN - 9781605581644

SP - 695

EP - 696

BT - ACM SIGIR 2008 - 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Proceedings

ER -