3D Auditory Scene Visualizer with face tracking: Design and implementation for auditory awareness compensation

Yuji Kubota, Shun Shiramatsu, Masatoshi Yoshida, Kazunori Komatani, Tetsuya Ogata, Hiroshi G. Okuno

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

This paper presents the design and implementation of 3D Auditory Scene Visualizer based on the visual information seeking mantra, "overview first, zoom and filter, then details on demand". The machine audition system called HARK captures 3D sounds with a microphone array. The natural language processing called SalienceGraph visualizes topic transition by using discourse salience. The 3D visualizer implemented in Java 3D displays topic transition and each sound stream as a beam originating from the microphones (overview mode), shows temporal snapshots with/without specifying focusing areas (zoom-andfilter mode), and shows detailed information about a particular sound stream (details-on-demand mode). This threemode visualization will give the user auditory awareness enhanced by HARK and SalienceGraph. In addition, a facetracking system automatically determines the user's intention by tracking the user's face. The resulting system will enable users to manage and browse auditory scene files effectively, so it should acceleration and support the information explosion to compensate the lack of auditory awareness.

Original languageEnglish
Title of host publicationProceedings of the 2nd International Symposium on Universal Communication, ISUC 2008
Pages42-49
Number of pages8
DOIs
Publication statusPublished - 2008
Externally publishedYes
Event2nd International Symposium on Universal Communication, ISUC 2008 - Osaka
Duration: 2008 Dec 152008 Dec 16

Other

Other2nd International Symposium on Universal Communication, ISUC 2008
CityOsaka
Period08/12/1508/12/16

Fingerprint

Acoustic waves
Microphones
Audition
Explosions
Visualization
Processing
Compensation and Redress

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Science Applications
  • Information Systems
  • Software

Cite this

Kubota, Y., Shiramatsu, S., Yoshida, M., Komatani, K., Ogata, T., & Okuno, H. G. (2008). 3D Auditory Scene Visualizer with face tracking: Design and implementation for auditory awareness compensation. In Proceedings of the 2nd International Symposium on Universal Communication, ISUC 2008 (pp. 42-49). [4724440] https://doi.org/10.1109/ISUC.2008.59

3D Auditory Scene Visualizer with face tracking : Design and implementation for auditory awareness compensation. / Kubota, Yuji; Shiramatsu, Shun; Yoshida, Masatoshi; Komatani, Kazunori; Ogata, Tetsuya; Okuno, Hiroshi G.

Proceedings of the 2nd International Symposium on Universal Communication, ISUC 2008. 2008. p. 42-49 4724440.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Kubota, Y, Shiramatsu, S, Yoshida, M, Komatani, K, Ogata, T & Okuno, HG 2008, 3D Auditory Scene Visualizer with face tracking: Design and implementation for auditory awareness compensation. in Proceedings of the 2nd International Symposium on Universal Communication, ISUC 2008., 4724440, pp. 42-49, 2nd International Symposium on Universal Communication, ISUC 2008, Osaka, 08/12/15. https://doi.org/10.1109/ISUC.2008.59
Kubota Y, Shiramatsu S, Yoshida M, Komatani K, Ogata T, Okuno HG. 3D Auditory Scene Visualizer with face tracking: Design and implementation for auditory awareness compensation. In Proceedings of the 2nd International Symposium on Universal Communication, ISUC 2008. 2008. p. 42-49. 4724440 https://doi.org/10.1109/ISUC.2008.59
Kubota, Yuji ; Shiramatsu, Shun ; Yoshida, Masatoshi ; Komatani, Kazunori ; Ogata, Tetsuya ; Okuno, Hiroshi G. / 3D Auditory Scene Visualizer with face tracking : Design and implementation for auditory awareness compensation. Proceedings of the 2nd International Symposium on Universal Communication, ISUC 2008. 2008. pp. 42-49
@inproceedings{538ce35b2bfa4f6591e49b0dc9991554,
title = "3D Auditory Scene Visualizer with face tracking: Design and implementation for auditory awareness compensation",
abstract = "This paper presents the design and implementation of 3D Auditory Scene Visualizer based on the visual information seeking mantra, {"}overview first, zoom and filter, then details on demand{"}. The machine audition system called HARK captures 3D sounds with a microphone array. The natural language processing called SalienceGraph visualizes topic transition by using discourse salience. The 3D visualizer implemented in Java 3D displays topic transition and each sound stream as a beam originating from the microphones (overview mode), shows temporal snapshots with/without specifying focusing areas (zoom-andfilter mode), and shows detailed information about a particular sound stream (details-on-demand mode). This threemode visualization will give the user auditory awareness enhanced by HARK and SalienceGraph. In addition, a facetracking system automatically determines the user's intention by tracking the user's face. The resulting system will enable users to manage and browse auditory scene files effectively, so it should acceleration and support the information explosion to compensate the lack of auditory awareness.",
author = "Yuji Kubota and Shun Shiramatsu and Masatoshi Yoshida and Kazunori Komatani and Tetsuya Ogata and Okuno, {Hiroshi G.}",
year = "2008",
doi = "10.1109/ISUC.2008.59",
language = "English",
isbn = "9780769534336",
pages = "42--49",
booktitle = "Proceedings of the 2nd International Symposium on Universal Communication, ISUC 2008",

}

TY - GEN

T1 - 3D Auditory Scene Visualizer with face tracking

T2 - Design and implementation for auditory awareness compensation

AU - Kubota, Yuji

AU - Shiramatsu, Shun

AU - Yoshida, Masatoshi

AU - Komatani, Kazunori

AU - Ogata, Tetsuya

AU - Okuno, Hiroshi G.

PY - 2008

Y1 - 2008

N2 - This paper presents the design and implementation of 3D Auditory Scene Visualizer based on the visual information seeking mantra, "overview first, zoom and filter, then details on demand". The machine audition system called HARK captures 3D sounds with a microphone array. The natural language processing called SalienceGraph visualizes topic transition by using discourse salience. The 3D visualizer implemented in Java 3D displays topic transition and each sound stream as a beam originating from the microphones (overview mode), shows temporal snapshots with/without specifying focusing areas (zoom-andfilter mode), and shows detailed information about a particular sound stream (details-on-demand mode). This threemode visualization will give the user auditory awareness enhanced by HARK and SalienceGraph. In addition, a facetracking system automatically determines the user's intention by tracking the user's face. The resulting system will enable users to manage and browse auditory scene files effectively, so it should acceleration and support the information explosion to compensate the lack of auditory awareness.

AB - This paper presents the design and implementation of 3D Auditory Scene Visualizer based on the visual information seeking mantra, "overview first, zoom and filter, then details on demand". The machine audition system called HARK captures 3D sounds with a microphone array. The natural language processing called SalienceGraph visualizes topic transition by using discourse salience. The 3D visualizer implemented in Java 3D displays topic transition and each sound stream as a beam originating from the microphones (overview mode), shows temporal snapshots with/without specifying focusing areas (zoom-andfilter mode), and shows detailed information about a particular sound stream (details-on-demand mode). This threemode visualization will give the user auditory awareness enhanced by HARK and SalienceGraph. In addition, a facetracking system automatically determines the user's intention by tracking the user's face. The resulting system will enable users to manage and browse auditory scene files effectively, so it should acceleration and support the information explosion to compensate the lack of auditory awareness.

UR - http://www.scopus.com/inward/record.url?scp=78650759446&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=78650759446&partnerID=8YFLogxK

U2 - 10.1109/ISUC.2008.59

DO - 10.1109/ISUC.2008.59

M3 - Conference contribution

AN - SCOPUS:78650759446

SN - 9780769534336

SP - 42

EP - 49

BT - Proceedings of the 2nd International Symposium on Universal Communication, ISUC 2008

ER -