Abstract
The effect of back-formulating questions in question answering evaluation is discussed. QA evaluation using back-formulated questions is not equivalent to that using questions devised without access to the document collection. QA test collection constructors should be careful when back-formulating questions based on some document passages. The users of back-formulated QA test collections should bear in mind that the absolute QA performance values obtained through experiments may not represent operational QA performance.
| Original language | English |
|---|---|
| Title of host publication | Proceedings of Sheffield SIGIR - Twenty-Seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval |
| Editors | K. Jarvelin, J. Allen, P. Bruza, M. Sanderson |
| Pages | 474-475 |
| Number of pages | 2 |
| Publication status | Published - 2004 |
| Externally published | Yes |
| Event | Proceedings of Sheffield SIGIR - Twenty-Seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval - Sheffield Duration: 2004 Jul 25 → 2004 Jul 29 |
Other
| Other | Proceedings of Sheffield SIGIR - Twenty-Seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval |
|---|---|
| City | Sheffield |
| Period | 04/7/25 → 04/7/29 |
Fingerprint
Keywords
- Evaluation
- Question answering
- Test collection
ASJC Scopus subject areas
- Engineering(all)
Cite this
The effect of back-formulating questions in question answering evaluation. / Sakai, Tetsuya; Saito, Yoshimi; Ichimura, Yumi; Kokubu, Tomoharu; Koyama, Makoto.
Proceedings of Sheffield SIGIR - Twenty-Seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ed. / K. Jarvelin; J. Allen; P. Bruza; M. Sanderson. 2004. p. 474-475.Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - The effect of back-formulating questions in question answering evaluation
AU - Sakai, Tetsuya
AU - Saito, Yoshimi
AU - Ichimura, Yumi
AU - Kokubu, Tomoharu
AU - Koyama, Makoto
PY - 2004
Y1 - 2004
N2 - The effect of back-formulating questions in question answering evaluation is discussed. QA evaluation using back-formulated questions is not equivalent to that using questions devised without access to the document collection. QA test collection constructors should be careful when back-formulating questions based on some document passages. The users of back-formulated QA test collections should bear in mind that the absolute QA performance values obtained through experiments may not represent operational QA performance.
AB - The effect of back-formulating questions in question answering evaluation is discussed. QA evaluation using back-formulated questions is not equivalent to that using questions devised without access to the document collection. QA test collection constructors should be careful when back-formulating questions based on some document passages. The users of back-formulated QA test collections should bear in mind that the absolute QA performance values obtained through experiments may not represent operational QA performance.
KW - Evaluation
KW - Question answering
KW - Test collection
UR - http://www.scopus.com/inward/record.url?scp=8644283799&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=8644283799&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:8644283799
SN - 1581138814
SP - 474
EP - 475
BT - Proceedings of Sheffield SIGIR - Twenty-Seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval
A2 - Jarvelin, K.
A2 - Allen, J.
A2 - Bruza, P.
A2 - Sanderson, M.
ER -