Recognizing multiple persons' facial expressions using HMM based on automatic extraction of significant frames from image sequences

Takahiro Otsuka, Jun Ohya

Research output: Contribution to conferencePaper

Abstract

A method that can be used for for recognizing facial expressions of multiple persons is proposed. In this method, the condition of facial muscles is assigned to a hidden state of a HMM for each expression. Then, the probability of the state is updated according to a feature vector obtained from image processing. Image processing is performed in two steps. First, a velocity vector is estimated from every two successive frames by using an optical flow algorithm. Then, a two-dimensional Fourier transform is applied to a velocity vector field at the regions around an eye and the mouth. The coefficients for lower frequencies are selected to form a feature vector. A mixture density is used for approximating the output probability of the HMM so as to represent a variation in facial expressions among persons. To cope with the case when two expressions are displayed contiguously, the HMM computation is modified such that when the peak of a facial motion is detected, a new sequence of facial expressions is assumed to start from the previous frame with minimal facial motion. Experiments show that a mixture density is effective because recognition accuracy improves as the number of mixtures increases. In addition, the method correctly recognizes a facial expression that contiguously follows another one.

Original languageEnglish
Pages546-549
Number of pages4
Publication statusPublished - 1997 Dec 1
EventProceedings of the 1997 International Conference on Image Processing. Part 2 (of 3) - Santa Barbara, CA, USA
Duration: 1997 Oct 261997 Oct 29

Other

OtherProceedings of the 1997 International Conference on Image Processing. Part 2 (of 3)
CitySanta Barbara, CA, USA
Period97/10/2697/10/29

    Fingerprint

ASJC Scopus subject areas

  • Hardware and Architecture
  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering

Cite this

Otsuka, T., & Ohya, J. (1997). Recognizing multiple persons' facial expressions using HMM based on automatic extraction of significant frames from image sequences. 546-549. Paper presented at Proceedings of the 1997 International Conference on Image Processing. Part 2 (of 3), Santa Barbara, CA, USA, .