Audio-visual interaction in model adaptation for multi-modal speech recognition

Satoshi Tamura; Masanao Oonishi; Satoru Hayamizu

Audio-visual interaction in model adaptation for multi-modal speech recognition

Satoshi Tamura^*, Masanao Oonishi, Satoru Hayamizu

^*Corresponding author for this work

Research output: Contribution to conference › Paper › peer-review

7 Citations (Scopus)

Abstract

This paper investigates audio-visual interaction, i.e. inter-modal influences, in linear-regressive model adaptation for multi-modal speech recognition. In the multi-modal adaptation, inter-modal information may contribute the performance of speech recognition. Thus the influence and advantage of intermodal elements should be examined. Experiments were conducted to evaluate several transformation matrices including or excluding inter-modal and intra-modal elements, using noisy data in an audio-visual corpus. From the experimental results, the importance of effective use of audio-visual interaction is clarified.

Original language	English
Pages	875-878
Number of pages	4
Publication status	Published - 2011
Externally published	Yes
Event	Asia-Pacific Signal and Information Processing Association Annual Summit and Conference 2011, APSIPA ASC 2011 - Xi'an, China Duration: 2011 Oct 18 → 2011 Oct 21

Conference

Conference	Asia-Pacific Signal and Information Processing Association Annual Summit and Conference 2011, APSIPA ASC 2011
Country/Territory	China
City	Xi'an
Period	11/10/18 → 11/10/21

ASJC Scopus subject areas

Information Systems
Signal Processing

Cite this

@conference{cd3e3031ee8f4f89a08b580b4038b3fc,

title = "Audio-visual interaction in model adaptation for multi-modal speech recognition",

abstract = "This paper investigates audio-visual interaction, i.e. inter-modal influences, in linear-regressive model adaptation for multi-modal speech recognition. In the multi-modal adaptation, inter-modal information may contribute the performance of speech recognition. Thus the influence and advantage of intermodal elements should be examined. Experiments were conducted to evaluate several transformation matrices including or excluding inter-modal and intra-modal elements, using noisy data in an audio-visual corpus. From the experimental results, the importance of effective use of audio-visual interaction is clarified.",

author = "Satoshi Tamura and Masanao Oonishi and Satoru Hayamizu",

year = "2011",

language = "English",

pages = "875--878",

note = "Asia-Pacific Signal and Information Processing Association Annual Summit and Conference 2011, APSIPA ASC 2011 ; Conference date: 18-10-2011 Through 21-10-2011",

}

TY - CONF

T1 - Audio-visual interaction in model adaptation for multi-modal speech recognition

AU - Tamura, Satoshi

AU - Oonishi, Masanao

AU - Hayamizu, Satoru

PY - 2011

Y1 - 2011

N2 - This paper investigates audio-visual interaction, i.e. inter-modal influences, in linear-regressive model adaptation for multi-modal speech recognition. In the multi-modal adaptation, inter-modal information may contribute the performance of speech recognition. Thus the influence and advantage of intermodal elements should be examined. Experiments were conducted to evaluate several transformation matrices including or excluding inter-modal and intra-modal elements, using noisy data in an audio-visual corpus. From the experimental results, the importance of effective use of audio-visual interaction is clarified.

AB - This paper investigates audio-visual interaction, i.e. inter-modal influences, in linear-regressive model adaptation for multi-modal speech recognition. In the multi-modal adaptation, inter-modal information may contribute the performance of speech recognition. Thus the influence and advantage of intermodal elements should be examined. Experiments were conducted to evaluate several transformation matrices including or excluding inter-modal and intra-modal elements, using noisy data in an audio-visual corpus. From the experimental results, the importance of effective use of audio-visual interaction is clarified.

UR - http://www.scopus.com/inward/record.url?scp=84866851099&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84866851099&partnerID=8YFLogxK

M3 - Paper

AN - SCOPUS:84866851099

SP - 875

EP - 878

T2 - Asia-Pacific Signal and Information Processing Association Annual Summit and Conference 2011, APSIPA ASC 2011

Y2 - 18 October 2011 through 21 October 2011

ER -

Audio-visual interaction in model adaptation for multi-modal speech recognition

Abstract

Conference

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this