Abstract
The effect of back-formulating questions in question answering evaluation is discussed. QA evaluation using back-formulated questions is not equivalent to that using questions devised without access to the document collection. QA test collection constructors should be careful when back-formulating questions based on some document passages. The users of back-formulated QA test collections should bear in mind that the absolute QA performance values obtained through experiments may not represent operational QA performance.
| Original language | English |
|---|---|
| Title of host publication | Proceedings of Sheffield SIGIR - Twenty-Seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval |
| Editors | K. Jarvelin, J. Allen, P. Bruza, M. Sanderson |
| Pages | 474-475 |
| Number of pages | 2 |
| Publication status | Published - 2004 Nov 25 |
| Event | Proceedings of Sheffield SIGIR - Twenty-Seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval - Sheffield, United Kingdom Duration: 2004 Jul 25 → 2004 Jul 29 |
Publication series
| Name | Proceedings of Sheffield SIGIR - Twenty-Seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval |
|---|
Conference
| Conference | Proceedings of Sheffield SIGIR - Twenty-Seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval |
|---|---|
| Country | United Kingdom |
| City | Sheffield |
| Period | 04/7/25 → 04/7/29 |
Fingerprint
Keywords
- Evaluation
- Question answering
- Test collection
ASJC Scopus subject areas
- Engineering(all)
Cite this
Sakai, T., Saito, Y., Ichimura, Y., Kokubu, T., & Koyama, M. (2004). The effect of back-formulating questions in question answering evaluation. In K. Jarvelin, J. Allen, P. Bruza, & M. Sanderson (Eds.), Proceedings of Sheffield SIGIR - Twenty-Seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 474-475). (Proceedings of Sheffield SIGIR - Twenty-Seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval).