MT-based Japanese-English cross-language IR experiments using the TREC test collections

This paper evaluates the effectiveness of MT-based Japanese-English CLIR using a subcollection of the TREC test collections and two bilingual researchers to separately translate the TREC requests into Japanese. Our main findings are as follows: (1)With the aid of pseudo-relevance feedback, MT-based J-E CLIR can be as effective as "best-case" monolingual IR. In particular, although poor MT quality often leads to poor initial CLIR performance, pseudo-relevance feedback is useful for alleviating the harm; (2)The manual request translation process that is inherent in conventional CLIR performance evaluation can be a more dominant factor than query length and overall MT quality.