Using speech data to recognize emotion in human gait

Angelica Lim, Hiroshi G. Okuno

Research output: Chapter in Book/Report/Conference proceedingConference contribution

7 Citations (Scopus)

Abstract

Robots that can recognize emotions can improve humans' mental health by providing empathy and social communication. Emotion recognition by robots is challenging because unlike in human-computer environments, facial information is not always available. Instead, our method proposes using speech and gait analysis to recognize human emotion. Previous research suggests that the dynamics of emotional human speech also underlie emotional gait (walking). We investigate the possibility of combining these two modalities via perceptually common parameters: Speed, Intensity, irRegularity, and Extent (SIRE). We map low-level features to this 4D cross-modal emotion space and train a Gaussian Mixture Model using independent samples from both voice and gait. Our results show that a single, modality-mixed trained model can perform emotion recognition for both modalities. Most interestingly, recognition of emotion in gait using a model trained uniquely on speech data gives comparable results to a model trained on gait data alone, providing evidence for a common underlying model for emotion across modalities.

Original languageEnglish
Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Pages52-64
Number of pages13
Volume7559 LNCS
DOIs
Publication statusPublished - 2012
Externally publishedYes
Event3rd International Workshop on Human Behavior Understanding, HBU 2012 - Vilamoura
Duration: 2012 Oct 72012 Oct 7

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume7559 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other3rd International Workshop on Human Behavior Understanding, HBU 2012
CityVilamoura
Period12/10/712/10/7

Fingerprint

Gait
Modality
Emotion Recognition
Robots
Gait analysis
Speech analysis
Robot
Speech Analysis
Gait Analysis
Gaussian Mixture Model
Mixed Model
Irregularity
Emotion
Speech
Human
Health
Communication
Model

Keywords

  • affect recognition
  • emotional gait
  • emotional voice
  • robot emotions

ASJC Scopus subject areas

  • Computer Science(all)
  • Theoretical Computer Science

Cite this

Lim, A., & Okuno, H. G. (2012). Using speech data to recognize emotion in human gait. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 7559 LNCS, pp. 52-64). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7559 LNCS). https://doi.org/10.1007/978-3-642-34014-7_5

Using speech data to recognize emotion in human gait. / Lim, Angelica; Okuno, Hiroshi G.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 7559 LNCS 2012. p. 52-64 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7559 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Lim, A & Okuno, HG 2012, Using speech data to recognize emotion in human gait. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). vol. 7559 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 7559 LNCS, pp. 52-64, 3rd International Workshop on Human Behavior Understanding, HBU 2012, Vilamoura, 12/10/7. https://doi.org/10.1007/978-3-642-34014-7_5
Lim A, Okuno HG. Using speech data to recognize emotion in human gait. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 7559 LNCS. 2012. p. 52-64. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-642-34014-7_5
Lim, Angelica ; Okuno, Hiroshi G. / Using speech data to recognize emotion in human gait. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 7559 LNCS 2012. pp. 52-64 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{0106d5708c284536a9bcaae36273ec3e,
title = "Using speech data to recognize emotion in human gait",
abstract = "Robots that can recognize emotions can improve humans' mental health by providing empathy and social communication. Emotion recognition by robots is challenging because unlike in human-computer environments, facial information is not always available. Instead, our method proposes using speech and gait analysis to recognize human emotion. Previous research suggests that the dynamics of emotional human speech also underlie emotional gait (walking). We investigate the possibility of combining these two modalities via perceptually common parameters: Speed, Intensity, irRegularity, and Extent (SIRE). We map low-level features to this 4D cross-modal emotion space and train a Gaussian Mixture Model using independent samples from both voice and gait. Our results show that a single, modality-mixed trained model can perform emotion recognition for both modalities. Most interestingly, recognition of emotion in gait using a model trained uniquely on speech data gives comparable results to a model trained on gait data alone, providing evidence for a common underlying model for emotion across modalities.",
keywords = "affect recognition, emotional gait, emotional voice, robot emotions",
author = "Angelica Lim and Okuno, {Hiroshi G.}",
year = "2012",
doi = "10.1007/978-3-642-34014-7_5",
language = "English",
isbn = "9783642340130",
volume = "7559 LNCS",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
pages = "52--64",
booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

}

TY - GEN

T1 - Using speech data to recognize emotion in human gait

AU - Lim, Angelica

AU - Okuno, Hiroshi G.

PY - 2012

Y1 - 2012

N2 - Robots that can recognize emotions can improve humans' mental health by providing empathy and social communication. Emotion recognition by robots is challenging because unlike in human-computer environments, facial information is not always available. Instead, our method proposes using speech and gait analysis to recognize human emotion. Previous research suggests that the dynamics of emotional human speech also underlie emotional gait (walking). We investigate the possibility of combining these two modalities via perceptually common parameters: Speed, Intensity, irRegularity, and Extent (SIRE). We map low-level features to this 4D cross-modal emotion space and train a Gaussian Mixture Model using independent samples from both voice and gait. Our results show that a single, modality-mixed trained model can perform emotion recognition for both modalities. Most interestingly, recognition of emotion in gait using a model trained uniquely on speech data gives comparable results to a model trained on gait data alone, providing evidence for a common underlying model for emotion across modalities.

AB - Robots that can recognize emotions can improve humans' mental health by providing empathy and social communication. Emotion recognition by robots is challenging because unlike in human-computer environments, facial information is not always available. Instead, our method proposes using speech and gait analysis to recognize human emotion. Previous research suggests that the dynamics of emotional human speech also underlie emotional gait (walking). We investigate the possibility of combining these two modalities via perceptually common parameters: Speed, Intensity, irRegularity, and Extent (SIRE). We map low-level features to this 4D cross-modal emotion space and train a Gaussian Mixture Model using independent samples from both voice and gait. Our results show that a single, modality-mixed trained model can perform emotion recognition for both modalities. Most interestingly, recognition of emotion in gait using a model trained uniquely on speech data gives comparable results to a model trained on gait data alone, providing evidence for a common underlying model for emotion across modalities.

KW - affect recognition

KW - emotional gait

KW - emotional voice

KW - robot emotions

UR - http://www.scopus.com/inward/record.url?scp=84867628984&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84867628984&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-34014-7_5

DO - 10.1007/978-3-642-34014-7_5

M3 - Conference contribution

SN - 9783642340130

VL - 7559 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 52

EP - 64

BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

ER -