Cross-modal Correlation Analysis between Vowel Sounds and Color

Win Thuzar Kyaw, Atsuya Suzuki, Yoshinori Sagisaka

研究成果: Conference contribution

抄録

Vowel-color association characteristics have been studied in the field of phonetics and perception. Though it has been reported that selected color categories after listening vowel categories have similar trends in multiple languages, their sentiment correlations have not yet been thoroughly studied from the viewpoint of speech features. We tried to find sentiment association characteristics between color parameters and speech features directly to have better understanding of cross-modal correlations and to find underlying principles for multimodal applications. Vowel samples uttered by 4 male and 3 female speakers were employed to associate colors after listening them by 34 subjects. Statistical analyses showed the advantage of employing RGB color parameters and speech formants directly to conventional color category to vowel category mapping. The selected color distributions in the F1- F2 plane clearly show that the acoustic speech resonance (i.e. F1 and F2) -RGB correlations can more consistently explain their sentiment correlations. Moreover, by incorporating our sentiment association experiment results using formant-synthesized speech, their correlations can be attributed to F1 and F2 rather than vowel categories. We believe that this finding in cross-modal correlations will serve for not only scientific understanding but also further studies and applications using cross-modal information mapping.

元の言語English
ホスト出版物のタイトル2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018 - Proceedings
出版者Institute of Electrical and Electronics Engineers Inc.
ISBN(電子版)9781728101644
DOI
出版物ステータスPublished - 2019 4 16
イベント2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018 - Pattaya, Thailand
継続期間: 2018 11 152018 11 17

出版物シリーズ

名前2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018 - Proceedings

Conference

Conference2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018
Thailand
Pattaya
期間18/11/1518/11/17

Fingerprint

Color
Acoustic waves
Speech Acoustics
Phonetics
Speech analysis
Language
Acoustics
Experiments

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Science Applications
  • Computer Vision and Pattern Recognition
  • Software
  • Health Informatics

これを引用

Kyaw, W. T., Suzuki, A., & Sagisaka, Y. (2019). Cross-modal Correlation Analysis between Vowel Sounds and Color. : 2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018 - Proceedings [8692957] (2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018 - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/iSAI-NLP.2018.8692957

Cross-modal Correlation Analysis between Vowel Sounds and Color. / Kyaw, Win Thuzar; Suzuki, Atsuya; Sagisaka, Yoshinori.

2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2019. 8692957 (2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018 - Proceedings).

研究成果: Conference contribution

Kyaw, WT, Suzuki, A & Sagisaka, Y 2019, Cross-modal Correlation Analysis between Vowel Sounds and Color. : 2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018 - Proceedings., 8692957, 2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018 - Proceedings, Institute of Electrical and Electronics Engineers Inc., 2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018, Pattaya, Thailand, 18/11/15. https://doi.org/10.1109/iSAI-NLP.2018.8692957
Kyaw WT, Suzuki A, Sagisaka Y. Cross-modal Correlation Analysis between Vowel Sounds and Color. : 2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2019. 8692957. (2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018 - Proceedings). https://doi.org/10.1109/iSAI-NLP.2018.8692957
Kyaw, Win Thuzar ; Suzuki, Atsuya ; Sagisaka, Yoshinori. / Cross-modal Correlation Analysis between Vowel Sounds and Color. 2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2019. (2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018 - Proceedings).
@inproceedings{60f81632d46e4cb5969c875efbb44de3,
title = "Cross-modal Correlation Analysis between Vowel Sounds and Color",
abstract = "Vowel-color association characteristics have been studied in the field of phonetics and perception. Though it has been reported that selected color categories after listening vowel categories have similar trends in multiple languages, their sentiment correlations have not yet been thoroughly studied from the viewpoint of speech features. We tried to find sentiment association characteristics between color parameters and speech features directly to have better understanding of cross-modal correlations and to find underlying principles for multimodal applications. Vowel samples uttered by 4 male and 3 female speakers were employed to associate colors after listening them by 34 subjects. Statistical analyses showed the advantage of employing RGB color parameters and speech formants directly to conventional color category to vowel category mapping. The selected color distributions in the F1- F2 plane clearly show that the acoustic speech resonance (i.e. F1 and F2) -RGB correlations can more consistently explain their sentiment correlations. Moreover, by incorporating our sentiment association experiment results using formant-synthesized speech, their correlations can be attributed to F1 and F2 rather than vowel categories. We believe that this finding in cross-modal correlations will serve for not only scientific understanding but also further studies and applications using cross-modal information mapping.",
keywords = "Cross-modal information expression, Cross-modal perception, Sentiment information, Speech-color correlation",
author = "Kyaw, {Win Thuzar} and Atsuya Suzuki and Yoshinori Sagisaka",
year = "2019",
month = "4",
day = "16",
doi = "10.1109/iSAI-NLP.2018.8692957",
language = "English",
series = "2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018 - Proceedings",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
booktitle = "2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018 - Proceedings",

}

TY - GEN

T1 - Cross-modal Correlation Analysis between Vowel Sounds and Color

AU - Kyaw, Win Thuzar

AU - Suzuki, Atsuya

AU - Sagisaka, Yoshinori

PY - 2019/4/16

Y1 - 2019/4/16

N2 - Vowel-color association characteristics have been studied in the field of phonetics and perception. Though it has been reported that selected color categories after listening vowel categories have similar trends in multiple languages, their sentiment correlations have not yet been thoroughly studied from the viewpoint of speech features. We tried to find sentiment association characteristics between color parameters and speech features directly to have better understanding of cross-modal correlations and to find underlying principles for multimodal applications. Vowel samples uttered by 4 male and 3 female speakers were employed to associate colors after listening them by 34 subjects. Statistical analyses showed the advantage of employing RGB color parameters and speech formants directly to conventional color category to vowel category mapping. The selected color distributions in the F1- F2 plane clearly show that the acoustic speech resonance (i.e. F1 and F2) -RGB correlations can more consistently explain their sentiment correlations. Moreover, by incorporating our sentiment association experiment results using formant-synthesized speech, their correlations can be attributed to F1 and F2 rather than vowel categories. We believe that this finding in cross-modal correlations will serve for not only scientific understanding but also further studies and applications using cross-modal information mapping.

AB - Vowel-color association characteristics have been studied in the field of phonetics and perception. Though it has been reported that selected color categories after listening vowel categories have similar trends in multiple languages, their sentiment correlations have not yet been thoroughly studied from the viewpoint of speech features. We tried to find sentiment association characteristics between color parameters and speech features directly to have better understanding of cross-modal correlations and to find underlying principles for multimodal applications. Vowel samples uttered by 4 male and 3 female speakers were employed to associate colors after listening them by 34 subjects. Statistical analyses showed the advantage of employing RGB color parameters and speech formants directly to conventional color category to vowel category mapping. The selected color distributions in the F1- F2 plane clearly show that the acoustic speech resonance (i.e. F1 and F2) -RGB correlations can more consistently explain their sentiment correlations. Moreover, by incorporating our sentiment association experiment results using formant-synthesized speech, their correlations can be attributed to F1 and F2 rather than vowel categories. We believe that this finding in cross-modal correlations will serve for not only scientific understanding but also further studies and applications using cross-modal information mapping.

KW - Cross-modal information expression

KW - Cross-modal perception

KW - Sentiment information

KW - Speech-color correlation

UR - http://www.scopus.com/inward/record.url?scp=85065060923&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85065060923&partnerID=8YFLogxK

U2 - 10.1109/iSAI-NLP.2018.8692957

DO - 10.1109/iSAI-NLP.2018.8692957

M3 - Conference contribution

T3 - 2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018 - Proceedings

BT - 2018 International Joint Symposium on Artificial Intelligence and Natural Language Processing, iSAI-NLP 2018 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

ER -