Low-Latency Real-Time Meeting Recognition and Understanding Using Distant Microphones and Omni-Directional Camera

Takaaki Hori, Shoko Araki, Takuya Yoshioka, Masakiyo Fujimoto, Shinji Watanabe, Takanobu Oba, Atsunori Ogawa, Keisuke Kinoshita, Tomohiro Nakatani, Atsushi Nakamura, Kazuhiro Otsuka, Dan Mikami, Junji Yamato

Research output: Contribution to journalArticlepeer-review

69 Citations (Scopus)


This paper presents our real-time meeting analyzer for monitoring conversations in an ongoing group meeting. The goal of the system is to recognize automatically “who is speaking what” in an online manner for meeting assistance. Our system continuously captures the utterances and face poses of each speaker using a microphone array and an omni-directional camera positioned at the center of the meeting table. Through a series of advanced audio processing operations, an overlapping speech signal is enhanced and the components are separated into individual speaker's channels. Then the utterances are sequentially transcribed by our speech recognizer with low latency. In parallel with speech recognition, the activity of each participant (e.g., speaking, laughing, watching someone) and the circumstances of the meeting (e.g., topic, activeness, casualness) are detected and displayed on a browser together with the transcripts. In this paper, we describe our techniques and our attempt to achieve the low-la-tency monitoring of meetings, and we show our experimental results for real-time meeting transcription.

Original languageEnglish
Pages (from-to)499-513
Number of pages15
JournalIEEE Transactions on Audio, Speech and Language Processing
Issue number2
Publication statusPublished - 2012 Feb
Externally publishedYes


  • Distant microphones
  • meeting analysis
  • speaker diarization
  • speech enhancement
  • speech recognition
  • topic tracking

ASJC Scopus subject areas

  • Acoustics and Ultrasonics
  • Electrical and Electronic Engineering


Dive into the research topics of 'Low-Latency Real-Time Meeting Recognition and Understanding Using Distant Microphones and Omni-Directional Camera'. Together they form a unique fingerprint.

Cite this