Report from the NTCIR-10 1CLICK-2 Japanese subtask: Baselines, upperbounds and evaluation robustness

Makoto P. Kato, Tetsuya Sakai, Takehiro Yamamoto, Mayu Iwata

研究成果: Conference contribution

4 被引用数 (Scopus)

抄録

The One Click Access Task (1CLICK) of NTCIR requires systems to return a concise multi-document summary of web pages in response to a query which is assumed to have been submitted in a mobile context. Systems are evaluated based on information units (or iUnits), and are required to present important pieces of information first and to minimise the amount of text the user has to read. Using the official Japanese results of the second round of the 1CLICK task from NTCIR-10, we discuss our task setting and evaluation framework. Our analyses show that: (1) Simple baseline methods that leverage search engine snippets or Wikipedia are effective for "lookup" type queries but not necessarily for other query types; (2) There is still a substantial gap between manual and automatic runs; and (3) Our evaluation metrics are relatively robust to the incompleteness of iUnits.

本文言語English
ホスト出版物のタイトルSIGIR 2013 - Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval
ページ753-756
ページ数4
DOI
出版ステータスPublished - 2013
外部発表はい
イベント36th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2013 - Dublin, Ireland
継続期間: 2013 7 282013 8 1

出版物シリーズ

名前SIGIR 2013 - Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval

Conference

Conference36th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2013
国/地域Ireland
CityDublin
Period13/7/2813/8/1

ASJC Scopus subject areas

  • コンピュータ グラフィックスおよびコンピュータ支援設計
  • 情報システム

フィンガープリント

「Report from the NTCIR-10 1CLICK-2 Japanese subtask: Baselines, upperbounds and evaluation robustness」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル