Abstract
Vowel-color association characteristics have been studied in the field of phonetics and perception. Though it has been reported that selected color categories after listening vowel categories have similar trends in multiple languages, their sentiment correlations have not yet been thoroughly studied from the viewpoint of speech features. We tried to find sentiment association characteristics between color parameters and speech features directly to have better understanding of cross-modal correlations and to find underlying principles for multimodal applications. Vowel samples uttered by 4 male and 3 female speakers were employed to associate colors after listening them by 34 subjects. Statistical analyses showed the advantage of employing RGB color parameters and speech formants directly to conventional color category to vowel category mapping. The selected color distributions in the F1- F2 plane clearly show that the acoustic speech resonance (i.e. F1 and F2) -RGB correlations can more consistently explain their sentiment correlations. Moreover, by incorporating our sentiment association experiment results using formant-synthesized speech, their correlations can be attributed to F1 and F2 rather than vowel categories. We believe that this finding in cross-modal correlations will serve for not only scientific understanding but also further studies and applications using cross-modal information mapping.
| Original language | English |
|---|---|
| Title of host publication | 2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018 - Proceedings |
| Publisher | Institute of Electrical and Electronics Engineers Inc. |
| ISBN (Electronic) | 9781728101644 |
| DOIs | |
| Publication status | Published - 2019 Apr 16 |
| Event | 2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018 - Pattaya, Thailand Duration: 2018 Nov 15 → 2018 Nov 17 |
Publication series
| Name | 2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018 - Proceedings |
|---|
Conference
| Conference | 2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018 |
|---|---|
| Country | Thailand |
| City | Pattaya |
| Period | 18/11/15 → 18/11/17 |
Fingerprint
Keywords
- Cross-modal information expression
- Cross-modal perception
- Sentiment information
- Speech-color correlation
ASJC Scopus subject areas
- Artificial Intelligence
- Computer Science Applications
- Computer Vision and Pattern Recognition
- Software
- Health Informatics
Cite this
Cross-modal Correlation Analysis between Vowel Sounds and Color. / Kyaw, Win Thuzar; Suzuki, Atsuya; Sagisaka, Yoshinori.
2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2019. 8692957 (2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018 - Proceedings).Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - Cross-modal Correlation Analysis between Vowel Sounds and Color
AU - Kyaw, Win Thuzar
AU - Suzuki, Atsuya
AU - Sagisaka, Yoshinori
PY - 2019/4/16
Y1 - 2019/4/16
N2 - Vowel-color association characteristics have been studied in the field of phonetics and perception. Though it has been reported that selected color categories after listening vowel categories have similar trends in multiple languages, their sentiment correlations have not yet been thoroughly studied from the viewpoint of speech features. We tried to find sentiment association characteristics between color parameters and speech features directly to have better understanding of cross-modal correlations and to find underlying principles for multimodal applications. Vowel samples uttered by 4 male and 3 female speakers were employed to associate colors after listening them by 34 subjects. Statistical analyses showed the advantage of employing RGB color parameters and speech formants directly to conventional color category to vowel category mapping. The selected color distributions in the F1- F2 plane clearly show that the acoustic speech resonance (i.e. F1 and F2) -RGB correlations can more consistently explain their sentiment correlations. Moreover, by incorporating our sentiment association experiment results using formant-synthesized speech, their correlations can be attributed to F1 and F2 rather than vowel categories. We believe that this finding in cross-modal correlations will serve for not only scientific understanding but also further studies and applications using cross-modal information mapping.
AB - Vowel-color association characteristics have been studied in the field of phonetics and perception. Though it has been reported that selected color categories after listening vowel categories have similar trends in multiple languages, their sentiment correlations have not yet been thoroughly studied from the viewpoint of speech features. We tried to find sentiment association characteristics between color parameters and speech features directly to have better understanding of cross-modal correlations and to find underlying principles for multimodal applications. Vowel samples uttered by 4 male and 3 female speakers were employed to associate colors after listening them by 34 subjects. Statistical analyses showed the advantage of employing RGB color parameters and speech formants directly to conventional color category to vowel category mapping. The selected color distributions in the F1- F2 plane clearly show that the acoustic speech resonance (i.e. F1 and F2) -RGB correlations can more consistently explain their sentiment correlations. Moreover, by incorporating our sentiment association experiment results using formant-synthesized speech, their correlations can be attributed to F1 and F2 rather than vowel categories. We believe that this finding in cross-modal correlations will serve for not only scientific understanding but also further studies and applications using cross-modal information mapping.
KW - Cross-modal information expression
KW - Cross-modal perception
KW - Sentiment information
KW - Speech-color correlation
UR - http://www.scopus.com/inward/record.url?scp=85065060923&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85065060923&partnerID=8YFLogxK
U2 - 10.1109/iSAI-NLP.2018.8692957
DO - 10.1109/iSAI-NLP.2018.8692957
M3 - Conference contribution
AN - SCOPUS:85065060923
T3 - 2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018 - Proceedings
BT - 2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018 - Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
ER -