Low-Latency Real-Time Meeting Recognition and Understanding Using Distant Microphones and Omni-Directional Camera

Takaaki Hori, Shoko Araki, Takuya Yoshioka, Masakiyo Fujimoto, Shinji Watanabe, Takanobu Oba, Atsunori Ogawa, Keisuke Kinoshita, Tomohiro Nakatani, Atsushi Nakamura, Kazuhiro Otsuka, Dan Mikami, Junji Yamato

Research output: Contribution to journalArticlepeer-review

62 Citations (Scopus)

Abstract

This paper presents our real-time meeting analyzer for monitoring conversations in an ongoing group meeting. The goal of the system is to recognize automatically “who is speaking what” in an online manner for meeting assistance. Our system continuously captures the utterances and face poses of each speaker using a microphone array and an omni-directional camera positioned at the center of the meeting table. Through a series of advanced audio processing operations, an overlapping speech signal is enhanced and the components are separated into individual speaker's channels. Then the utterances are sequentially transcribed by our speech recognizer with low latency. In parallel with speech recognition, the activity of each participant (e.g., speaking, laughing, watching someone) and the circumstances of the meeting (e.g., topic, activeness, casualness) are detected and displayed on a browser together with the transcripts. In this paper, we describe our techniques and our attempt to achieve the low-la-tency monitoring of meetings, and we show our experimental results for real-time meeting transcription.

Original languageEnglish
Pages (from-to)499-513
Number of pages15
JournalIEEE Transactions on Audio, Speech and Language Processing
Volume20
Issue number2
DOIs
Publication statusPublished - 2012
Externally publishedYes

Keywords

  • Distant microphones
  • meeting analysis
  • speaker diarization
  • speech enhancement
  • speech recognition
  • topic tracking

ASJC Scopus subject areas

  • Acoustics and Ultrasonics
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Low-Latency Real-Time Meeting Recognition and Understanding Using Distant Microphones and Omni-Directional Camera'. Together they form a unique fingerprint.

Cite this