Juman++: A morphological analysis toolkit for scriptio continua

Arseny Tolmachev, Daisuke Kawahara, Sadao Kurohashi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

We present a three-part toolkit for developing morphological analyzers for languages without natural word boundaries. The first part is a lattice-based morphological analysis library that uses a combination of linear and recurrent neural net language models for analysis. The other parts are a tool for exposing problems in the trained model and a partial annotation tool. Our morphological analyzer for Japanese achieves new SOTA on Jumandic-based corpora while being 250 times faster than the previous one. We also perform a small experiment and quantitive analysis of using our toolkit.

Original languageEnglish
Title of host publicationEMNLP 2018 - Conference on Empirical Methods in Natural Language Processing
Subtitle of host publicationSystem Demonstrations, Proceedings
PublisherAssociation for Computational Linguistics (ACL)
Pages54-59
Number of pages6
ISBN (Electronic)9781948087858
DOIs
Publication statusPublished - 2018
Externally publishedYes
Event2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, EMNLP 2018 - Brussels, Belgium
Duration: 2018 Oct 312018 Nov 4

Publication series

NameEMNLP 2018 - Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Proceedings

Conference

Conference2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, EMNLP 2018
CountryBelgium
CityBrussels
Period18/10/3118/11/4

ASJC Scopus subject areas

  • Information Systems
  • Computational Theory and Mathematics
  • Computer Science Applications

Fingerprint Dive into the research topics of 'Juman++: A morphological analysis toolkit for scriptio continua'. Together they form a unique fingerprint.

  • Cite this

    Tolmachev, A., Kawahara, D., & Kurohashi, S. (2018). Juman++: A morphological analysis toolkit for scriptio continua. In EMNLP 2018 - Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Proceedings (pp. 54-59). (EMNLP 2018 - Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Proceedings). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/d18-2010