Lessons from BMIR-J2

A test collection for Japanese IR systems

Tsuyoshi Kitani, Haruo Kimoto, Toshikazu Fukushima, Tetsuya Sakai, Hiroshi Tsuruoka, Yasushi Ogawa, Ikuo Keshi, Kunio Matsui, Takenobu Tokunaga, Hidekazu Nakawatase, Tetsuya Ishikawa, Jun Toyoura, Yoshihiro Ueda, Teru Agata

Research output: Contribution to journalArticle

9 Citations (Scopus)

Abstract

BMIR-J2 is the first complete Japanese test collection available for use in evaluating information retrieval systems. It contains sixty queries and the IDs of 5080 newspaper articles in the fields of economics and engineering. The queries are classified into five categories, based on the functions the system is likely to use to interpret them correctly and retrieve relevant texts. This collection has two levels of relevance, topically relevant and partially relevant. Also discussed are design issues such as collection types and size. This collection and the principles derived in designing it should be helpful in the future development of new test collections.

Original languageEnglish
Pages (from-to)345-346
Number of pages2
JournalSIGIR Forum (ACM Special Interest Group on Information Retrieval)
Publication statusPublished - 1998
Externally publishedYes

Fingerprint

Information retrieval systems
Economics
Test collections
Query

ASJC Scopus subject areas

  • Hardware and Architecture
  • Management Information Systems

Cite this

Kitani, T., Kimoto, H., Fukushima, T., Sakai, T., Tsuruoka, H., Ogawa, Y., ... Agata, T. (1998). Lessons from BMIR-J2: A test collection for Japanese IR systems. SIGIR Forum (ACM Special Interest Group on Information Retrieval), 345-346.

Lessons from BMIR-J2 : A test collection for Japanese IR systems. / Kitani, Tsuyoshi; Kimoto, Haruo; Fukushima, Toshikazu; Sakai, Tetsuya; Tsuruoka, Hiroshi; Ogawa, Yasushi; Keshi, Ikuo; Matsui, Kunio; Tokunaga, Takenobu; Nakawatase, Hidekazu; Ishikawa, Tetsuya; Toyoura, Jun; Ueda, Yoshihiro; Agata, Teru.

In: SIGIR Forum (ACM Special Interest Group on Information Retrieval), 1998, p. 345-346.

Research output: Contribution to journalArticle

Kitani, T, Kimoto, H, Fukushima, T, Sakai, T, Tsuruoka, H, Ogawa, Y, Keshi, I, Matsui, K, Tokunaga, T, Nakawatase, H, Ishikawa, T, Toyoura, J, Ueda, Y & Agata, T 1998, 'Lessons from BMIR-J2: A test collection for Japanese IR systems', SIGIR Forum (ACM Special Interest Group on Information Retrieval), pp. 345-346.
Kitani, Tsuyoshi ; Kimoto, Haruo ; Fukushima, Toshikazu ; Sakai, Tetsuya ; Tsuruoka, Hiroshi ; Ogawa, Yasushi ; Keshi, Ikuo ; Matsui, Kunio ; Tokunaga, Takenobu ; Nakawatase, Hidekazu ; Ishikawa, Tetsuya ; Toyoura, Jun ; Ueda, Yoshihiro ; Agata, Teru. / Lessons from BMIR-J2 : A test collection for Japanese IR systems. In: SIGIR Forum (ACM Special Interest Group on Information Retrieval). 1998 ; pp. 345-346.
@article{34ca3332b5d4468c955dc86b20eaa13f,
title = "Lessons from BMIR-J2: A test collection for Japanese IR systems",
abstract = "BMIR-J2 is the first complete Japanese test collection available for use in evaluating information retrieval systems. It contains sixty queries and the IDs of 5080 newspaper articles in the fields of economics and engineering. The queries are classified into five categories, based on the functions the system is likely to use to interpret them correctly and retrieve relevant texts. This collection has two levels of relevance, topically relevant and partially relevant. Also discussed are design issues such as collection types and size. This collection and the principles derived in designing it should be helpful in the future development of new test collections.",
author = "Tsuyoshi Kitani and Haruo Kimoto and Toshikazu Fukushima and Tetsuya Sakai and Hiroshi Tsuruoka and Yasushi Ogawa and Ikuo Keshi and Kunio Matsui and Takenobu Tokunaga and Hidekazu Nakawatase and Tetsuya Ishikawa and Jun Toyoura and Yoshihiro Ueda and Teru Agata",
year = "1998",
language = "English",
pages = "345--346",
journal = "SIGIR Forum (ACM Special Interest Group on Information Retrieval)",
issn = "0163-5840",
publisher = "Association for Computing Machinery (ACM)",

}

TY - JOUR

T1 - Lessons from BMIR-J2

T2 - A test collection for Japanese IR systems

AU - Kitani, Tsuyoshi

AU - Kimoto, Haruo

AU - Fukushima, Toshikazu

AU - Sakai, Tetsuya

AU - Tsuruoka, Hiroshi

AU - Ogawa, Yasushi

AU - Keshi, Ikuo

AU - Matsui, Kunio

AU - Tokunaga, Takenobu

AU - Nakawatase, Hidekazu

AU - Ishikawa, Tetsuya

AU - Toyoura, Jun

AU - Ueda, Yoshihiro

AU - Agata, Teru

PY - 1998

Y1 - 1998

N2 - BMIR-J2 is the first complete Japanese test collection available for use in evaluating information retrieval systems. It contains sixty queries and the IDs of 5080 newspaper articles in the fields of economics and engineering. The queries are classified into five categories, based on the functions the system is likely to use to interpret them correctly and retrieve relevant texts. This collection has two levels of relevance, topically relevant and partially relevant. Also discussed are design issues such as collection types and size. This collection and the principles derived in designing it should be helpful in the future development of new test collections.

AB - BMIR-J2 is the first complete Japanese test collection available for use in evaluating information retrieval systems. It contains sixty queries and the IDs of 5080 newspaper articles in the fields of economics and engineering. The queries are classified into five categories, based on the functions the system is likely to use to interpret them correctly and retrieve relevant texts. This collection has two levels of relevance, topically relevant and partially relevant. Also discussed are design issues such as collection types and size. This collection and the principles derived in designing it should be helpful in the future development of new test collections.

UR - http://www.scopus.com/inward/record.url?scp=0032272513&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0032272513&partnerID=8YFLogxK

M3 - Article

SP - 345

EP - 346

JO - SIGIR Forum (ACM Special Interest Group on Information Retrieval)

JF - SIGIR Forum (ACM Special Interest Group on Information Retrieval)

SN - 0163-5840

ER -