Realizing audio-visually triggered Eliza-like non-verbal behaviors

Hiroshi G. Okuno, Kazuhiro Nakadai, Hiroaki Kitano

Research output: Chapter in Book/Report/Conference proceedingConference contribution

9 Citations (Scopus)

Abstract

We are studying how to create social physical agents, i.e., humanoids, that perform actions empowered by real-time audio-visual tracking of multiple talkers. Social skills require complex perceptual and motor capabilities as well as communicating ones. It is critical to identify primary features in designing building blocks for social skills, because performance of social interaction is usually evaluated as a whole system but not as each component.We investigate the minimum functionalities for social interaction, supposed that a humanoid is equipped with auditory and visual perception and simple motor control but not with sound output. Real-time audio-visual multiple-talker tracking system is implemented on the humanoid, SIG, by using sound source localization, stereo vision, face recognition, and motor control. It extracts either auditory or visual streams and associates audio and visual streams by the proximity in localization. Socially oriented attention control makes the best use of personality variations classified by the Interpersonal Theory of psychology. It also provides task-oriented funcitons with decaying factor of belief for each stream. We demonstrate that the resulting behavior of SIG invites the users’ participation in interaction and encourages the users to explore SIG’s behaviors. These demonstrations show that SIG behaves like a physical non-verbal Eliza.

Original languageEnglish
Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
PublisherSpringer Verlag
Pages552-562
Number of pages11
Volume2417
ISBN (Print)3540440380, 9783540440383
Publication statusPublished - 2002
Externally publishedYes
Event7th Pacific Rim International Conference on Artificial Intelligence, PRICAI 2002 - Tokyo, Japan
Duration: 2002 Aug 182002 Aug 22

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume2417
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other7th Pacific Rim International Conference on Artificial Intelligence, PRICAI 2002
CountryJapan
CityTokyo
Period02/8/1802/8/22

Fingerprint

Motor Control
Social Interaction
Acoustic waves
Real-time
Source Localization
Visual Tracking
Visual Perception
Stereo vision
Stereo Vision
Tracking System
Face recognition
Face Recognition
Building Blocks
Proximity
Demonstrations
Output
Interaction
Demonstrate
Vision
Skills

ASJC Scopus subject areas

  • Computer Science(all)
  • Theoretical Computer Science

Cite this

Okuno, H. G., Nakadai, K., & Kitano, H. (2002). Realizing audio-visually triggered Eliza-like non-verbal behaviors. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 2417, pp. 552-562). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 2417). Springer Verlag.

Realizing audio-visually triggered Eliza-like non-verbal behaviors. / Okuno, Hiroshi G.; Nakadai, Kazuhiro; Kitano, Hiroaki.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 2417 Springer Verlag, 2002. p. 552-562 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 2417).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Okuno, HG, Nakadai, K & Kitano, H 2002, Realizing audio-visually triggered Eliza-like non-verbal behaviors. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). vol. 2417, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 2417, Springer Verlag, pp. 552-562, 7th Pacific Rim International Conference on Artificial Intelligence, PRICAI 2002, Tokyo, Japan, 02/8/18.
Okuno HG, Nakadai K, Kitano H. Realizing audio-visually triggered Eliza-like non-verbal behaviors. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 2417. Springer Verlag. 2002. p. 552-562. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
Okuno, Hiroshi G. ; Nakadai, Kazuhiro ; Kitano, Hiroaki. / Realizing audio-visually triggered Eliza-like non-verbal behaviors. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 2417 Springer Verlag, 2002. pp. 552-562 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{7f4b4958c3594a3a976c66364a40330b,
title = "Realizing audio-visually triggered Eliza-like non-verbal behaviors",
abstract = "We are studying how to create social physical agents, i.e., humanoids, that perform actions empowered by real-time audio-visual tracking of multiple talkers. Social skills require complex perceptual and motor capabilities as well as communicating ones. It is critical to identify primary features in designing building blocks for social skills, because performance of social interaction is usually evaluated as a whole system but not as each component.We investigate the minimum functionalities for social interaction, supposed that a humanoid is equipped with auditory and visual perception and simple motor control but not with sound output. Real-time audio-visual multiple-talker tracking system is implemented on the humanoid, SIG, by using sound source localization, stereo vision, face recognition, and motor control. It extracts either auditory or visual streams and associates audio and visual streams by the proximity in localization. Socially oriented attention control makes the best use of personality variations classified by the Interpersonal Theory of psychology. It also provides task-oriented funcitons with decaying factor of belief for each stream. We demonstrate that the resulting behavior of SIG invites the users’ participation in interaction and encourages the users to explore SIG’s behaviors. These demonstrations show that SIG behaves like a physical non-verbal Eliza.",
author = "Okuno, {Hiroshi G.} and Kazuhiro Nakadai and Hiroaki Kitano",
year = "2002",
language = "English",
isbn = "3540440380",
volume = "2417",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "552--562",
booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

}

TY - GEN

T1 - Realizing audio-visually triggered Eliza-like non-verbal behaviors

AU - Okuno, Hiroshi G.

AU - Nakadai, Kazuhiro

AU - Kitano, Hiroaki

PY - 2002

Y1 - 2002

N2 - We are studying how to create social physical agents, i.e., humanoids, that perform actions empowered by real-time audio-visual tracking of multiple talkers. Social skills require complex perceptual and motor capabilities as well as communicating ones. It is critical to identify primary features in designing building blocks for social skills, because performance of social interaction is usually evaluated as a whole system but not as each component.We investigate the minimum functionalities for social interaction, supposed that a humanoid is equipped with auditory and visual perception and simple motor control but not with sound output. Real-time audio-visual multiple-talker tracking system is implemented on the humanoid, SIG, by using sound source localization, stereo vision, face recognition, and motor control. It extracts either auditory or visual streams and associates audio and visual streams by the proximity in localization. Socially oriented attention control makes the best use of personality variations classified by the Interpersonal Theory of psychology. It also provides task-oriented funcitons with decaying factor of belief for each stream. We demonstrate that the resulting behavior of SIG invites the users’ participation in interaction and encourages the users to explore SIG’s behaviors. These demonstrations show that SIG behaves like a physical non-verbal Eliza.

AB - We are studying how to create social physical agents, i.e., humanoids, that perform actions empowered by real-time audio-visual tracking of multiple talkers. Social skills require complex perceptual and motor capabilities as well as communicating ones. It is critical to identify primary features in designing building blocks for social skills, because performance of social interaction is usually evaluated as a whole system but not as each component.We investigate the minimum functionalities for social interaction, supposed that a humanoid is equipped with auditory and visual perception and simple motor control but not with sound output. Real-time audio-visual multiple-talker tracking system is implemented on the humanoid, SIG, by using sound source localization, stereo vision, face recognition, and motor control. It extracts either auditory or visual streams and associates audio and visual streams by the proximity in localization. Socially oriented attention control makes the best use of personality variations classified by the Interpersonal Theory of psychology. It also provides task-oriented funcitons with decaying factor of belief for each stream. We demonstrate that the resulting behavior of SIG invites the users’ participation in interaction and encourages the users to explore SIG’s behaviors. These demonstrations show that SIG behaves like a physical non-verbal Eliza.

UR - http://www.scopus.com/inward/record.url?scp=77954272906&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77954272906&partnerID=8YFLogxK

M3 - Conference contribution

SN - 3540440380

SN - 9783540440383

VL - 2417

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 552

EP - 562

BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

PB - Springer Verlag

ER -