TY - GEN
T1 - Real-time estimation of learners' mental states from learners' physiological information using deep learning
AU - Tawatsuji, Yoshimasa
AU - Uno, Tatsuro
AU - Fang, Siyuan
AU - Matsui, Tatsunori
N1 - Publisher Copyright:
© 2018 Asia-Pacific Society for Computers in Education. All rights reserved.
Copyright:
Copyright 2019 Elsevier B.V., All rights reserved.
PY - 2018/11/24
Y1 - 2018/11/24
N2 - It is important to know the mental states of learners during the learning process to improve the effectiveness of teaching and learning. In this study, we first extracted the relationships between learners' mental states and teachers' speech acts, as well as learners' physiological information, by constructing a deep learning system. The physiological indexes were near infrared spectroscopy (NIRS), electroencephalography (EEG), respiration intensity, skin conductance, and pulse volume. Learners' mental states were divided into nine categories in accordance with the Achievement Emotions Questionnaire. In our experiment, the system achieved a high accuracy in predicting the learner's mental states from the teacher's speech acts and the learner's physiological information. A mock-up experiment was then conducted, which revealed that the system's interface was able to support teaching and learning in real time.
AB - It is important to know the mental states of learners during the learning process to improve the effectiveness of teaching and learning. In this study, we first extracted the relationships between learners' mental states and teachers' speech acts, as well as learners' physiological information, by constructing a deep learning system. The physiological indexes were near infrared spectroscopy (NIRS), electroencephalography (EEG), respiration intensity, skin conductance, and pulse volume. Learners' mental states were divided into nine categories in accordance with the Achievement Emotions Questionnaire. In our experiment, the system achieved a high accuracy in predicting the learner's mental states from the teacher's speech acts and the learner's physiological information. A mock-up experiment was then conducted, which revealed that the system's interface was able to support teaching and learning in real time.
KW - Achievement emotions questionnaire
KW - Deep learning
KW - Emotion estimation
KW - Learning support
KW - Physiological information
UR - http://www.scopus.com/inward/record.url?scp=85060050473&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85060050473&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85060050473
T3 - ICCE 2018 - 26th International Conference on Computers in Education, Main Conference Proceedings
SP - 107
EP - 109
BT - ICCE 2018 - 26th International Conference on Computers in Education, Main Conference Proceedings
A2 - Rodrigo, Ma. Mercedes T.
A2 - Yang, Jie-Chi
A2 - Wong, Lung-Hsiang
A2 - Chang, Maiga
PB - Asia-Pacific Society for Computers in Education
T2 - 26th International Conference on Computers in Education, ICCE 2018
Y2 - 26 November 2018 through 30 November 2018
ER -