TY - GEN
T1 - JGLUE
T2 - 13th International Conference on Language Resources and Evaluation Conference, LREC 2022
AU - Kurihara, Kentaro
AU - Kawahara, Daisuke
AU - Shibata, Tomohide
N1 - Funding Information:
This work was carried out in a joint research project between Yahoo Japan Corporation and Waseda University.
Publisher Copyright:
© European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0.
PY - 2022
Y1 - 2022
N2 - To develop high-performance natural language understanding (NLU) models, it is necessary to have a benchmark to evaluate and analyze NLU ability from various perspectives. While the English NLU benchmark, GLUE (Wang et al., 2018), has been the forerunner, benchmarks are now being released for languages other than English, such as CLUE (Xu et al., 2020) for Chinese and FLUE (Le et al., 2020) for French; but there is no such benchmark for Japanese. We build a Japanese NLU benchmark, JGLUE, from scratch without translation to measure the general NLU ability in Japanese. We hope that JGLUE will facilitate NLU research in Japanese.
AB - To develop high-performance natural language understanding (NLU) models, it is necessary to have a benchmark to evaluate and analyze NLU ability from various perspectives. While the English NLU benchmark, GLUE (Wang et al., 2018), has been the forerunner, benchmarks are now being released for languages other than English, such as CLUE (Xu et al., 2020) for Chinese and FLUE (Le et al., 2020) for French; but there is no such benchmark for Japanese. We build a Japanese NLU benchmark, JGLUE, from scratch without translation to measure the general NLU ability in Japanese. We hope that JGLUE will facilitate NLU research in Japanese.
KW - GLUE
KW - Japanese
KW - NLU benchmark
KW - QA
KW - sentence pair classification
KW - text classification
UR - http://www.scopus.com/inward/record.url?scp=85144418989&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85144418989&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85144418989
T3 - 2022 Language Resources and Evaluation Conference, LREC 2022
SP - 2957
EP - 2966
BT - 2022 Language Resources and Evaluation Conference, LREC 2022
A2 - Calzolari, Nicoletta
A2 - Bechet, Frederic
A2 - Blache, Philippe
A2 - Choukri, Khalid
A2 - Cieri, Christopher
A2 - Declerck, Thierry
A2 - Goggi, Sara
A2 - Isahara, Hitoshi
A2 - Maegaard, Bente
A2 - Mariani, Joseph
A2 - Mazo, Helene
A2 - Odijk, Jan
A2 - Piperidis, Stelios
PB - European Language Resources Association (ELRA)
Y2 - 20 June 2022 through 25 June 2022
ER -