Real-time estimation of learners' mental states from learners' physiological information using deep learning

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

It is important to know the mental states of learners during the learning process to improve the effectiveness of teaching and learning. In this study, we first extracted the relationships between learners' mental states and teachers' speech acts, as well as learners' physiological information, by constructing a deep learning system. The physiological indexes were near infrared spectroscopy (NIRS), electroencephalography (EEG), respiration intensity, skin conductance, and pulse volume. Learners' mental states were divided into nine categories in accordance with the Achievement Emotions Questionnaire. In our experiment, the system achieved a high accuracy in predicting the learner's mental states from the teacher's speech acts and the learner's physiological information. A mock-up experiment was then conducted, which revealed that the system's interface was able to support teaching and learning in real time.

Original languageEnglish
Title of host publicationICCE 2018 - 26th International Conference on Computers in Education, Main Conference Proceedings
EditorsMa. Mercedes T. Rodrigo, Jie-Chi Yang, Lung-Hsiang Wong, Maiga Chang
PublisherAsia-Pacific Society for Computers in Education
Pages107-109
Number of pages3
ISBN (Electronic)9789869401289
Publication statusPublished - 2018 Nov 24
Event26th International Conference on Computers in Education, ICCE 2018 - Metro Manila, Philippines
Duration: 2018 Nov 262018 Nov 30

Other

Other26th International Conference on Computers in Education, ICCE 2018
CountryPhilippines
CityMetro Manila
Period18/11/2618/11/30

Fingerprint

Teaching
speech act
Near infrared spectroscopy
Electroencephalography
learning
Learning systems
effectiveness of teaching
Skin
Experiments
experiment
teacher
learning process
emotion
questionnaire
time
Deep learning

Keywords

  • Achievement emotions questionnaire
  • Deep learning
  • Emotion estimation
  • Learning support
  • Physiological information

ASJC Scopus subject areas

  • Computer Science (miscellaneous)
  • Computer Science Applications
  • Education

Cite this

Tawatsuji, Y., Uno, T., Fang, S., & Matsui, T. (2018). Real-time estimation of learners' mental states from learners' physiological information using deep learning. In M. M. T. Rodrigo, J-C. Yang, L-H. Wong, & M. Chang (Eds.), ICCE 2018 - 26th International Conference on Computers in Education, Main Conference Proceedings (pp. 107-109). Asia-Pacific Society for Computers in Education.

Real-time estimation of learners' mental states from learners' physiological information using deep learning. / Tawatsuji, Yoshimasa; Uno, Tatsuro; Fang, Siyuan; Matsui, Tatsunori.

ICCE 2018 - 26th International Conference on Computers in Education, Main Conference Proceedings. ed. / Ma. Mercedes T. Rodrigo; Jie-Chi Yang; Lung-Hsiang Wong; Maiga Chang. Asia-Pacific Society for Computers in Education, 2018. p. 107-109.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Tawatsuji, Y, Uno, T, Fang, S & Matsui, T 2018, Real-time estimation of learners' mental states from learners' physiological information using deep learning. in MMT Rodrigo, J-C Yang, L-H Wong & M Chang (eds), ICCE 2018 - 26th International Conference on Computers in Education, Main Conference Proceedings. Asia-Pacific Society for Computers in Education, pp. 107-109, 26th International Conference on Computers in Education, ICCE 2018, Metro Manila, Philippines, 18/11/26.
Tawatsuji Y, Uno T, Fang S, Matsui T. Real-time estimation of learners' mental states from learners' physiological information using deep learning. In Rodrigo MMT, Yang J-C, Wong L-H, Chang M, editors, ICCE 2018 - 26th International Conference on Computers in Education, Main Conference Proceedings. Asia-Pacific Society for Computers in Education. 2018. p. 107-109
Tawatsuji, Yoshimasa ; Uno, Tatsuro ; Fang, Siyuan ; Matsui, Tatsunori. / Real-time estimation of learners' mental states from learners' physiological information using deep learning. ICCE 2018 - 26th International Conference on Computers in Education, Main Conference Proceedings. editor / Ma. Mercedes T. Rodrigo ; Jie-Chi Yang ; Lung-Hsiang Wong ; Maiga Chang. Asia-Pacific Society for Computers in Education, 2018. pp. 107-109
@inproceedings{9cacad90143944909db1fb528bb3801c,
title = "Real-time estimation of learners' mental states from learners' physiological information using deep learning",
abstract = "It is important to know the mental states of learners during the learning process to improve the effectiveness of teaching and learning. In this study, we first extracted the relationships between learners' mental states and teachers' speech acts, as well as learners' physiological information, by constructing a deep learning system. The physiological indexes were near infrared spectroscopy (NIRS), electroencephalography (EEG), respiration intensity, skin conductance, and pulse volume. Learners' mental states were divided into nine categories in accordance with the Achievement Emotions Questionnaire. In our experiment, the system achieved a high accuracy in predicting the learner's mental states from the teacher's speech acts and the learner's physiological information. A mock-up experiment was then conducted, which revealed that the system's interface was able to support teaching and learning in real time.",
keywords = "Achievement emotions questionnaire, Deep learning, Emotion estimation, Learning support, Physiological information",
author = "Yoshimasa Tawatsuji and Tatsuro Uno and Siyuan Fang and Tatsunori Matsui",
year = "2018",
month = "11",
day = "24",
language = "English",
pages = "107--109",
editor = "Rodrigo, {Ma. Mercedes T.} and Jie-Chi Yang and Lung-Hsiang Wong and Maiga Chang",
booktitle = "ICCE 2018 - 26th International Conference on Computers in Education, Main Conference Proceedings",
publisher = "Asia-Pacific Society for Computers in Education",

}

TY - GEN

T1 - Real-time estimation of learners' mental states from learners' physiological information using deep learning

AU - Tawatsuji, Yoshimasa

AU - Uno, Tatsuro

AU - Fang, Siyuan

AU - Matsui, Tatsunori

PY - 2018/11/24

Y1 - 2018/11/24

N2 - It is important to know the mental states of learners during the learning process to improve the effectiveness of teaching and learning. In this study, we first extracted the relationships between learners' mental states and teachers' speech acts, as well as learners' physiological information, by constructing a deep learning system. The physiological indexes were near infrared spectroscopy (NIRS), electroencephalography (EEG), respiration intensity, skin conductance, and pulse volume. Learners' mental states were divided into nine categories in accordance with the Achievement Emotions Questionnaire. In our experiment, the system achieved a high accuracy in predicting the learner's mental states from the teacher's speech acts and the learner's physiological information. A mock-up experiment was then conducted, which revealed that the system's interface was able to support teaching and learning in real time.

AB - It is important to know the mental states of learners during the learning process to improve the effectiveness of teaching and learning. In this study, we first extracted the relationships between learners' mental states and teachers' speech acts, as well as learners' physiological information, by constructing a deep learning system. The physiological indexes were near infrared spectroscopy (NIRS), electroencephalography (EEG), respiration intensity, skin conductance, and pulse volume. Learners' mental states were divided into nine categories in accordance with the Achievement Emotions Questionnaire. In our experiment, the system achieved a high accuracy in predicting the learner's mental states from the teacher's speech acts and the learner's physiological information. A mock-up experiment was then conducted, which revealed that the system's interface was able to support teaching and learning in real time.

KW - Achievement emotions questionnaire

KW - Deep learning

KW - Emotion estimation

KW - Learning support

KW - Physiological information

UR - http://www.scopus.com/inward/record.url?scp=85060050473&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85060050473&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85060050473

SP - 107

EP - 109

BT - ICCE 2018 - 26th International Conference on Computers in Education, Main Conference Proceedings

A2 - Rodrigo, Ma. Mercedes T.

A2 - Yang, Jie-Chi

A2 - Wong, Lung-Hsiang

A2 - Chang, Maiga

PB - Asia-Pacific Society for Computers in Education

ER -