Flexible Pseudo-Relevance Feedback via Selective Sampling

Tetsuya Sakai, Toshihiko Manabe, Makoto Koyama

Research output: Contribution to journalArticle

55 Citations (Scopus)

Abstract

Although Pseudo-Relevance Feedback (PRF) is a widely used technique for enhancing average retrieval performance, it may actually hurt performance for around one-third of a given set of topics. To enhance the reliability of PRF, Flexible PRF has been proposed, which adjusts the number of pseudo-relevant documents and/or the number of expansion terms for each topic. This paper explores a new, inexpensive Flexible PRF method, called Selective Sampling, which is unique in that it can skip documents in the initial ranked output to look for more “novel” pseudo-relevant documents. While Selective Sampling is only comparable to Traditional PRF in terms of average performance and reliability, per-topic analyses show that Selective Sampling outperforms Traditional PRF almost as often as Traditional PRF outperforms Selective Sampling. Thus, treating the top P documents as relevant is often not the best strategy. However, predicting when Selective Sampling outperforms Traditional PRF appears to be as difficult as predicting when a PRF method fails. For example, our per-topic analyses show that even the proportion of truly relevant documents in the pseudo-relevant set is not necessarily a good performance predictor.

Original languageEnglish
Pages (from-to)111-135
Number of pages25
JournalACM Transactions on Asian Language Information Processing
Volume4
Issue number2
DOIs
Publication statusPublished - 2005 Jun 1
Externally publishedYes

Keywords

  • Experimentation
  • Performance
  • Pseudo-relevance feedback
  • flexible pseudo-relevance feedback
  • selective sampling

ASJC Scopus subject areas

  • Computer Science(all)

Fingerprint Dive into the research topics of 'Flexible Pseudo-Relevance Feedback via Selective Sampling'. Together they form a unique fingerprint.

  • Cite this