Abstract
In this paper, we present a simple protocol to evaluate word aligners on bilingual lexicon induction tasks from parallel corpora. Rather than resorting to gold standards, it relies on a comparison of the outputs of word aligners against a reference bilingual lexicon. The quality of this reference bilingual lexicon does not need to be particularly high, because evaluation quality is ensured by systematically filtering this reference lexicon with the parallel corpus the word aligners are trained on. We perform a comparison of three freely available word aligners on numerous language pairs from the Bible parallel corpus (Resnik et al., 1999): MGIZA++ (Gao and Vogel, 2008), BerkeleyAligner (Liang et al., 2006), and Anymalign (Lardilleux and Lepage, 2009). We then select the most appropriate one to produce bilingual lexicons for all language pairs of this corpus. These involve Cebuano, Chinese, Danish, English, Finnish, French, Greek, Indonesian, Latin, Spanish, Swedish, and Vietnamese. The 66 resulting lexicons are made freely available.
| Original language | English |
|---|---|
| Title of host publication | Proceedings of the 7th International Conference on Language Resources and Evaluation, LREC 2010 |
| Publisher | European Language Resources Association (ELRA) |
| Pages | 252-256 |
| Number of pages | 5 |
| ISBN (Electronic) | 2951740867, 9782951740860 |
| Publication status | Published - 2010 Jan 1 |
| Externally published | Yes |
| Event | 7th International Conference on Language Resources and Evaluation, LREC 2010 - Valletta, Malta Duration: 2010 May 17 → 2010 May 23 |
Other
| Other | 7th International Conference on Language Resources and Evaluation, LREC 2010 |
|---|---|
| Country | Malta |
| City | Valletta |
| Period | 10/5/17 → 10/5/23 |
Fingerprint
ASJC Scopus subject areas
- Education
- Library and Information Sciences
- Linguistics and Language
- Language and Linguistics
Cite this
Bilingual lexicon induction : Effortless evaluation of word alignment tools and production of resources for improbable language pairs. / Lardilleux, Adrien; Gosme, Julien; Lepage, Yves.
Proceedings of the 7th International Conference on Language Resources and Evaluation, LREC 2010. European Language Resources Association (ELRA), 2010. p. 252-256.Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - Bilingual lexicon induction
T2 - Effortless evaluation of word alignment tools and production of resources for improbable language pairs
AU - Lardilleux, Adrien
AU - Gosme, Julien
AU - Lepage, Yves
PY - 2010/1/1
Y1 - 2010/1/1
N2 - In this paper, we present a simple protocol to evaluate word aligners on bilingual lexicon induction tasks from parallel corpora. Rather than resorting to gold standards, it relies on a comparison of the outputs of word aligners against a reference bilingual lexicon. The quality of this reference bilingual lexicon does not need to be particularly high, because evaluation quality is ensured by systematically filtering this reference lexicon with the parallel corpus the word aligners are trained on. We perform a comparison of three freely available word aligners on numerous language pairs from the Bible parallel corpus (Resnik et al., 1999): MGIZA++ (Gao and Vogel, 2008), BerkeleyAligner (Liang et al., 2006), and Anymalign (Lardilleux and Lepage, 2009). We then select the most appropriate one to produce bilingual lexicons for all language pairs of this corpus. These involve Cebuano, Chinese, Danish, English, Finnish, French, Greek, Indonesian, Latin, Spanish, Swedish, and Vietnamese. The 66 resulting lexicons are made freely available.
AB - In this paper, we present a simple protocol to evaluate word aligners on bilingual lexicon induction tasks from parallel corpora. Rather than resorting to gold standards, it relies on a comparison of the outputs of word aligners against a reference bilingual lexicon. The quality of this reference bilingual lexicon does not need to be particularly high, because evaluation quality is ensured by systematically filtering this reference lexicon with the parallel corpus the word aligners are trained on. We perform a comparison of three freely available word aligners on numerous language pairs from the Bible parallel corpus (Resnik et al., 1999): MGIZA++ (Gao and Vogel, 2008), BerkeleyAligner (Liang et al., 2006), and Anymalign (Lardilleux and Lepage, 2009). We then select the most appropriate one to produce bilingual lexicons for all language pairs of this corpus. These involve Cebuano, Chinese, Danish, English, Finnish, French, Greek, Indonesian, Latin, Spanish, Swedish, and Vietnamese. The 66 resulting lexicons are made freely available.
UR - http://www.scopus.com/inward/record.url?scp=85037066948&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85037066948&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85037066948
SP - 252
EP - 256
BT - Proceedings of the 7th International Conference on Language Resources and Evaluation, LREC 2010
PB - European Language Resources Association (ELRA)
ER -