Extraction of relationships between learners' physiological information and learners' mental states by machine learning

Yoshimasa Tawatsuji, Tatsuro Uno, Keita Okazaki, Siyuan Fang, Tatsunori Matsui

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The estimation of learners' mental states during the interaction between teachers and learners is a very important problem in improving the quality of teaching and learning. In this experimental study, we developed a deep learning neural network (DLNN) system that extracted the relationships between a learner's mental states and a teacher's utterances plus the learner's physiological information. The learner's physiological information consisted of the NIRS signals, the EEG signals, respiration intensity, skin conductance, and pulse volume. The learner's mental states were elicited through the learner's introspective reports using the Achievement Emotions Questionnaire (AEQ). According to the AEQ, the learner's mental states were divided into nine categories: Enjoy, Hope, Pride, Anger, Anxiety, Shame, Hopelessness, Boredom, and Others. In a simulation, the DLNN system exhibited the ability to estimate the learner's mental states from the learner's physiological information with high accuracy.

Original languageEnglish
Title of host publicationProceedings of the 25th International Conference on Computers in Education, ICCE 2017 - Main Conference Proceedings
EditorsAhmad Fauzi Mohd Ayub, Antonija Mitrovic, Jie-Chi Yang, Su Luan Wong, Wenli Chen
PublisherAsia-Pacific Society for Computers in Education
Pages56-61
Number of pages6
ISBN (Print)9789869401265
Publication statusPublished - 2017 Jan 1
Event25th International Conference on Computers in Education, ICCE 2017 - Christchurch, New Zealand
Duration: 2017 Dec 42017 Dec 8

Other

Other25th International Conference on Computers in Education, ICCE 2017
CountryNew Zealand
CityChristchurch
Period17/12/417/12/8

Keywords

  • Deep learning
  • Intelligent Mentoring System
  • Mental state estimation
  • Physiological data

ASJC Scopus subject areas

  • Computer Science (miscellaneous)
  • Computer Science Applications
  • Information Systems
  • Hardware and Architecture
  • Education

Fingerprint Dive into the research topics of 'Extraction of relationships between learners' physiological information and learners' mental states by machine learning'. Together they form a unique fingerprint.

  • Cite this

    Tawatsuji, Y., Uno, T., Okazaki, K., Fang, S., & Matsui, T. (2017). Extraction of relationships between learners' physiological information and learners' mental states by machine learning. In A. F. Mohd Ayub, A. Mitrovic, J-C. Yang, S. L. Wong, & W. Chen (Eds.), Proceedings of the 25th International Conference on Computers in Education, ICCE 2017 - Main Conference Proceedings (pp. 56-61). Asia-Pacific Society for Computers in Education.