Human gesture analysis using multimodal features

Luo Dan*, Hazim Kemal Ekenel, Ohya Jun

*この研究の対応する著者

研究成果: Conference contribution

6 被引用数 (Scopus)

抄録

Human gesture as a natural interface plays an utmost important role for achieving intelligent Human Computer Interaction (HCI). Human gestures include different components of visual actions such as motion of hands, facial expression, and torso, to convey meaning. So far, in the field of gesture recognition, most previous works have focused on the manual component of gestures. In this paper, we present an appearance-based multimodal gesture recognition framework, which combines the different groups of features such as facial expression features and hand motion features which are extracted from image frames captured by a single web camera. We refer 12 classes of human gestures with facial expression including neutral, negative and positive meanings from American Sign Languages (ASL). We combine the features in two levels by employing two fusion strategies. At the feature level, an early feature combination can be performed by concatenating and weighting different feature groups, and PLS is used to choose the most discriminative elements by projecting the feature on a discriminative expression space. The second strategy is applied on decision level. Weighted decisions from single modalities are fused in a later stage. A condensation-based algorithm is adopted for classification. We collected a data set with three to seven recording sessions and conducted experiments with the combination techniques. Experimental results showed that facial analysis improve hand gesture recognition, decision level fusion performs better than feature level fusion.

本文言語English
ホスト出版物のタイトルProceedings of the 2012 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2012
ページ471-476
ページ数6
DOI
出版ステータスPublished - 2012 10月 4
イベント2012 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2012 - Melbourne, VIC, Australia
継続期間: 2012 7月 92012 7月 13

出版物シリーズ

名前Proceedings of the 2012 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2012

Conference

Conference2012 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2012
国/地域Australia
CityMelbourne, VIC
Period12/7/912/7/13

ASJC Scopus subject areas

  • コンピュータ グラフィックスおよびコンピュータ支援設計
  • コンピュータ ビジョンおよびパターン認識
  • 人間とコンピュータの相互作用

フィンガープリント

「Human gesture analysis using multimodal features」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル