Separating three simultaneous speeches with two microphones by integrating auditory and visual processing

Hiroshi G. Okuno; Kazuhiro Nakadai; Tino Lourens; Hiroaki Kitano

Separating three simultaneous speeches with two microphones by integrating auditory and visual processing

Hiroshi G. Okuno, Kazuhiro Nakadai, Tino Lourens, Hiroaki Kitano

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

This paper addresses the problem of automatic recognition of three simultaneous speeches with two microphones, that is, that of sound source separation where the number of sound sources is greater than that of microphones. The approach used is the direction-pass filter, which is implemented by hypothetical reasoning on the interaural phase difference (IPD) and interaural intensity difference (IID). Auditory processing calculates IPD and IID for each subband, and generates hypotheses for precalculated IPD and IID for every direction including one obtained by visual processing. Then the system calculates the belief factor of hypothesis by Dempster-Shafer theory and determines the direction of each subband. Subbands of the specific direction are collected and then converted to a wave form by inverse FFT. With 200 benchmarks of three simultaneous utterances of Japanese words, the average 1-best and 10-best recognition rates of extracted speeches are 60% and 81%, respectively.

Original language	English
Title of host publication	EUROSPEECH 2001 - SCANDINAVIA - 7th European Conference on Speech Communication and Technology
Publisher	International Speech Communication Association
Pages	2643-2646
Number of pages	4
ISBN (Electronic)	8790834100, 9788790834104
Publication status	Published - 2001
Externally published	Yes
Event	7th European Conference on Speech Communication and Technology - Scandinavia, EUROSPEECH 2001 - Aalborg, Denmark Duration: 2001 Sep 3 → 2001 Sep 7

Other

Other	7th European Conference on Speech Communication and Technology - Scandinavia, EUROSPEECH 2001
Country/Territory	Denmark
City	Aalborg
Period	01/9/3 → 01/9/7

ASJC Scopus subject areas

Communication
Linguistics and Language
Computer Science Applications
Software

Cite this

Separating three simultaneous speeches with two microphones by integrating auditory and visual processing. / Okuno, Hiroshi G.; Nakadai, Kazuhiro; Lourens, Tino; Kitano, Hiroaki.

EUROSPEECH 2001 - SCANDINAVIA - 7th European Conference on Speech Communication and Technology. International Speech Communication Association, 2001. p. 2643-2646.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Okuno, HG, Nakadai, K, Lourens, T & Kitano, H 2001, Separating three simultaneous speeches with two microphones by integrating auditory and visual processing. in EUROSPEECH 2001 - SCANDINAVIA - 7th European Conference on Speech Communication and Technology. International Speech Communication Association, pp. 2643-2646, 7th European Conference on Speech Communication and Technology - Scandinavia, EUROSPEECH 2001, Aalborg, Denmark, 01/9/3.

@inproceedings{17ed961555c84b359975307c8ebc2991,

title = "Separating three simultaneous speeches with two microphones by integrating auditory and visual processing",

abstract = "This paper addresses the problem of automatic recognition of three simultaneous speeches with two microphones, that is, that of sound source separation where the number of sound sources is greater than that of microphones. The approach used is the direction-pass filter, which is implemented by hypothetical reasoning on the interaural phase difference (IPD) and interaural intensity difference (IID). Auditory processing calculates IPD and IID for each subband, and generates hypotheses for precalculated IPD and IID for every direction including one obtained by visual processing. Then the system calculates the belief factor of hypothesis by Dempster-Shafer theory and determines the direction of each subband. Subbands of the specific direction are collected and then converted to a wave form by inverse FFT. With 200 benchmarks of three simultaneous utterances of Japanese words, the average 1-best and 10-best recognition rates of extracted speeches are 60% and 81%, respectively.",

author = "Okuno, {Hiroshi G.} and Kazuhiro Nakadai and Tino Lourens and Hiroaki Kitano",

year = "2001",

language = "English",

pages = "2643--2646",

booktitle = "EUROSPEECH 2001 - SCANDINAVIA - 7th European Conference on Speech Communication and Technology",

publisher = "International Speech Communication Association",

note = "7th European Conference on Speech Communication and Technology - Scandinavia, EUROSPEECH 2001 ; Conference date: 03-09-2001 Through 07-09-2001",

}

TY - GEN

T1 - Separating three simultaneous speeches with two microphones by integrating auditory and visual processing

AU - Okuno, Hiroshi G.

AU - Nakadai, Kazuhiro

AU - Lourens, Tino

AU - Kitano, Hiroaki

PY - 2001

Y1 - 2001

N2 - This paper addresses the problem of automatic recognition of three simultaneous speeches with two microphones, that is, that of sound source separation where the number of sound sources is greater than that of microphones. The approach used is the direction-pass filter, which is implemented by hypothetical reasoning on the interaural phase difference (IPD) and interaural intensity difference (IID). Auditory processing calculates IPD and IID for each subband, and generates hypotheses for precalculated IPD and IID for every direction including one obtained by visual processing. Then the system calculates the belief factor of hypothesis by Dempster-Shafer theory and determines the direction of each subband. Subbands of the specific direction are collected and then converted to a wave form by inverse FFT. With 200 benchmarks of three simultaneous utterances of Japanese words, the average 1-best and 10-best recognition rates of extracted speeches are 60% and 81%, respectively.

AB - This paper addresses the problem of automatic recognition of three simultaneous speeches with two microphones, that is, that of sound source separation where the number of sound sources is greater than that of microphones. The approach used is the direction-pass filter, which is implemented by hypothetical reasoning on the interaural phase difference (IPD) and interaural intensity difference (IID). Auditory processing calculates IPD and IID for each subband, and generates hypotheses for precalculated IPD and IID for every direction including one obtained by visual processing. Then the system calculates the belief factor of hypothesis by Dempster-Shafer theory and determines the direction of each subband. Subbands of the specific direction are collected and then converted to a wave form by inverse FFT. With 200 benchmarks of three simultaneous utterances of Japanese words, the average 1-best and 10-best recognition rates of extracted speeches are 60% and 81%, respectively.

UR - http://www.scopus.com/inward/record.url?scp=85009151501&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85009151501&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85009151501

SP - 2643

EP - 2646

BT - EUROSPEECH 2001 - SCANDINAVIA - 7th European Conference on Speech Communication and Technology

PB - International Speech Communication Association

T2 - 7th European Conference on Speech Communication and Technology - Scandinavia, EUROSPEECH 2001

Y2 - 3 September 2001 through 7 September 2001

ER -

Separating three simultaneous speeches with two microphones by integrating auditory and visual processing

Abstract

Other

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this