Lessons from BMIR-J2: A test collection for Japanese IR systems

Tsuyoshi Kitani, Haruo Kimoto, Toshikazu Fukushima, Tetsuya Sakai, Hiroshi Tsuruoka, Yasushi Ogawa, Ikuo Keshi, Kunio Matsui, Takenobu Tokunaga, Hidekazu Nakawatase, Tetsuya Ishikawa, Jun Toyoura, Yoshihiro Ueda, Teru Agata

Research output: Contribution to journalConference article

9 Citations (Scopus)

Abstract

BMIR-J2 is the first complete Japanese test collection available for use in evaluating information retrieval systems. It contains sixty queries and the IDs of 5080 newspaper articles in the fields of economics and engineering. The queries are classified into five categories, based on the functions the system is likely to use to interpret them correctly and retrieve relevant texts. This collection has two levels of relevance, topically relevant and partially relevant. Also discussed are design issues such as collection types and size. This collection and the principles derived in designing it should be helpful in the future development of new test collections.

Original languageEnglish
Pages (from-to)345-346
Number of pages2
JournalSIGIR Forum (ACM Special Interest Group on Information Retrieval)
Publication statusPublished - 1998 Dec 1
EventProceedings of the 1998 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'98) - Melbourne, Vic., Aust
Duration: 1998 Aug 241998 Aug 28

    Fingerprint

ASJC Scopus subject areas

  • Management Information Systems
  • Hardware and Architecture

Cite this

Kitani, T., Kimoto, H., Fukushima, T., Sakai, T., Tsuruoka, H., Ogawa, Y., Keshi, I., Matsui, K., Tokunaga, T., Nakawatase, H., Ishikawa, T., Toyoura, J., Ueda, Y., & Agata, T. (1998). Lessons from BMIR-J2: A test collection for Japanese IR systems. SIGIR Forum (ACM Special Interest Group on Information Retrieval), 345-346.