Abstract
This paper presents the design and implementation of 3D Auditory Scene Visualizer based on the visual information seeking mantra, "overview first, zoom and filter, then details on demand". The machine audition system called HARK captures 3D sounds with a microphone array. The natural language processing called SalienceGraph visualizes topic transition by using discourse salience. The 3D visualizer implemented in Java 3D displays topic transition and each sound stream as a beam originating from the microphones (overview mode), shows temporal snapshots with/without specifying focusing areas (zoom-andfilter mode), and shows detailed information about a particular sound stream (details-on-demand mode). This threemode visualization will give the user auditory awareness enhanced by HARK and SalienceGraph. In addition, a facetracking system automatically determines the user's intention by tracking the user's face. The resulting system will enable users to manage and browse auditory scene files effectively, so it should acceleration and support the information explosion to compensate the lack of auditory awareness.
| Original language | English |
|---|---|
| Title of host publication | Proceedings of the 2nd International Symposium on Universal Communication, ISUC 2008 |
| Pages | 42-49 |
| Number of pages | 8 |
| DOIs | |
| Publication status | Published - 2008 |
| Externally published | Yes |
| Event | 2nd International Symposium on Universal Communication, ISUC 2008 - Osaka Duration: 2008 Dec 15 → 2008 Dec 16 |
Other
| Other | 2nd International Symposium on Universal Communication, ISUC 2008 |
|---|---|
| City | Osaka |
| Period | 08/12/15 → 08/12/16 |
Fingerprint
ASJC Scopus subject areas
- Artificial Intelligence
- Computer Science Applications
- Information Systems
- Software
Cite this
3D Auditory Scene Visualizer with face tracking : Design and implementation for auditory awareness compensation. / Kubota, Yuji; Shiramatsu, Shun; Yoshida, Masatoshi; Komatani, Kazunori; Ogata, Tetsuya; Okuno, Hiroshi G.
Proceedings of the 2nd International Symposium on Universal Communication, ISUC 2008. 2008. p. 42-49 4724440.Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - 3D Auditory Scene Visualizer with face tracking
T2 - Design and implementation for auditory awareness compensation
AU - Kubota, Yuji
AU - Shiramatsu, Shun
AU - Yoshida, Masatoshi
AU - Komatani, Kazunori
AU - Ogata, Tetsuya
AU - Okuno, Hiroshi G.
PY - 2008
Y1 - 2008
N2 - This paper presents the design and implementation of 3D Auditory Scene Visualizer based on the visual information seeking mantra, "overview first, zoom and filter, then details on demand". The machine audition system called HARK captures 3D sounds with a microphone array. The natural language processing called SalienceGraph visualizes topic transition by using discourse salience. The 3D visualizer implemented in Java 3D displays topic transition and each sound stream as a beam originating from the microphones (overview mode), shows temporal snapshots with/without specifying focusing areas (zoom-andfilter mode), and shows detailed information about a particular sound stream (details-on-demand mode). This threemode visualization will give the user auditory awareness enhanced by HARK and SalienceGraph. In addition, a facetracking system automatically determines the user's intention by tracking the user's face. The resulting system will enable users to manage and browse auditory scene files effectively, so it should acceleration and support the information explosion to compensate the lack of auditory awareness.
AB - This paper presents the design and implementation of 3D Auditory Scene Visualizer based on the visual information seeking mantra, "overview first, zoom and filter, then details on demand". The machine audition system called HARK captures 3D sounds with a microphone array. The natural language processing called SalienceGraph visualizes topic transition by using discourse salience. The 3D visualizer implemented in Java 3D displays topic transition and each sound stream as a beam originating from the microphones (overview mode), shows temporal snapshots with/without specifying focusing areas (zoom-andfilter mode), and shows detailed information about a particular sound stream (details-on-demand mode). This threemode visualization will give the user auditory awareness enhanced by HARK and SalienceGraph. In addition, a facetracking system automatically determines the user's intention by tracking the user's face. The resulting system will enable users to manage and browse auditory scene files effectively, so it should acceleration and support the information explosion to compensate the lack of auditory awareness.
UR - http://www.scopus.com/inward/record.url?scp=78650759446&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=78650759446&partnerID=8YFLogxK
U2 - 10.1109/ISUC.2008.59
DO - 10.1109/ISUC.2008.59
M3 - Conference contribution
AN - SCOPUS:78650759446
SN - 9780769534336
SP - 42
EP - 49
BT - Proceedings of the 2nd International Symposium on Universal Communication, ISUC 2008
ER -