Extraction of relationships between learners' physiological information and learners' mental states by machine learning

Yoshimasa Tawatsuji, Tatsuro Uno, Keita Okazaki, Siyuan Fang, Tatsunori Matsui

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The estimation of learners' mental states during the interaction between teachers and learners is a very important problem in improving the quality of teaching and learning. In this experimental study, we developed a deep learning neural network (DLNN) system that extracted the relationships between a learner's mental states and a teacher's utterances plus the learner's physiological information. The learner's physiological information consisted of the NIRS signals, the EEG signals, respiration intensity, skin conductance, and pulse volume. The learner's mental states were elicited through the learner's introspective reports using the Achievement Emotions Questionnaire (AEQ). According to the AEQ, the learner's mental states were divided into nine categories: Enjoy, Hope, Pride, Anger, Anxiety, Shame, Hopelessness, Boredom, and Others. In a simulation, the DLNN system exhibited the ability to estimate the learner's mental states from the learner's physiological information with high accuracy.

Original languageEnglish
Title of host publicationProceedings of the 25th International Conference on Computers in Education, ICCE 2017 - Main Conference Proceedings
EditorsAhmad Fauzi Mohd Ayub, Antonija Mitrovic, Jie-Chi Yang, Su Luan Wong, Wenli Chen
PublisherAsia-Pacific Society for Computers in Education
Pages56-61
Number of pages6
ISBN (Print)9789869401265
Publication statusPublished - 2017 Jan 1
Event25th International Conference on Computers in Education, ICCE 2017 - Christchurch, New Zealand
Duration: 2017 Dec 42017 Dec 8

Other

Other25th International Conference on Computers in Education, ICCE 2017
CountryNew Zealand
CityChristchurch
Period17/12/417/12/8

Fingerprint

Learning systems
Neural networks
neural network
emotion
Electroencephalography
learning
boredom
Skin
Teaching
questionnaire
shame
teacher
anger
anxiety
simulation
ability
interaction
Deep learning

Keywords

  • Deep learning
  • Intelligent Mentoring System
  • Mental state estimation
  • Physiological data

ASJC Scopus subject areas

  • Computer Science (miscellaneous)
  • Computer Science Applications
  • Information Systems
  • Hardware and Architecture
  • Education

Cite this

Tawatsuji, Y., Uno, T., Okazaki, K., Fang, S., & Matsui, T. (2017). Extraction of relationships between learners' physiological information and learners' mental states by machine learning. In A. F. Mohd Ayub, A. Mitrovic, J-C. Yang, S. L. Wong, & W. Chen (Eds.), Proceedings of the 25th International Conference on Computers in Education, ICCE 2017 - Main Conference Proceedings (pp. 56-61). Asia-Pacific Society for Computers in Education.

Extraction of relationships between learners' physiological information and learners' mental states by machine learning. / Tawatsuji, Yoshimasa; Uno, Tatsuro; Okazaki, Keita; Fang, Siyuan; Matsui, Tatsunori.

Proceedings of the 25th International Conference on Computers in Education, ICCE 2017 - Main Conference Proceedings. ed. / Ahmad Fauzi Mohd Ayub; Antonija Mitrovic; Jie-Chi Yang; Su Luan Wong; Wenli Chen. Asia-Pacific Society for Computers in Education, 2017. p. 56-61.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Tawatsuji, Y, Uno, T, Okazaki, K, Fang, S & Matsui, T 2017, Extraction of relationships between learners' physiological information and learners' mental states by machine learning. in AF Mohd Ayub, A Mitrovic, J-C Yang, SL Wong & W Chen (eds), Proceedings of the 25th International Conference on Computers in Education, ICCE 2017 - Main Conference Proceedings. Asia-Pacific Society for Computers in Education, pp. 56-61, 25th International Conference on Computers in Education, ICCE 2017, Christchurch, New Zealand, 17/12/4.
Tawatsuji Y, Uno T, Okazaki K, Fang S, Matsui T. Extraction of relationships between learners' physiological information and learners' mental states by machine learning. In Mohd Ayub AF, Mitrovic A, Yang J-C, Wong SL, Chen W, editors, Proceedings of the 25th International Conference on Computers in Education, ICCE 2017 - Main Conference Proceedings. Asia-Pacific Society for Computers in Education. 2017. p. 56-61
Tawatsuji, Yoshimasa ; Uno, Tatsuro ; Okazaki, Keita ; Fang, Siyuan ; Matsui, Tatsunori. / Extraction of relationships between learners' physiological information and learners' mental states by machine learning. Proceedings of the 25th International Conference on Computers in Education, ICCE 2017 - Main Conference Proceedings. editor / Ahmad Fauzi Mohd Ayub ; Antonija Mitrovic ; Jie-Chi Yang ; Su Luan Wong ; Wenli Chen. Asia-Pacific Society for Computers in Education, 2017. pp. 56-61
@inproceedings{7649715f2de840138c141389c164ddb4,
title = "Extraction of relationships between learners' physiological information and learners' mental states by machine learning",
abstract = "The estimation of learners' mental states during the interaction between teachers and learners is a very important problem in improving the quality of teaching and learning. In this experimental study, we developed a deep learning neural network (DLNN) system that extracted the relationships between a learner's mental states and a teacher's utterances plus the learner's physiological information. The learner's physiological information consisted of the NIRS signals, the EEG signals, respiration intensity, skin conductance, and pulse volume. The learner's mental states were elicited through the learner's introspective reports using the Achievement Emotions Questionnaire (AEQ). According to the AEQ, the learner's mental states were divided into nine categories: Enjoy, Hope, Pride, Anger, Anxiety, Shame, Hopelessness, Boredom, and Others. In a simulation, the DLNN system exhibited the ability to estimate the learner's mental states from the learner's physiological information with high accuracy.",
keywords = "Deep learning, Intelligent Mentoring System, Mental state estimation, Physiological data",
author = "Yoshimasa Tawatsuji and Tatsuro Uno and Keita Okazaki and Siyuan Fang and Tatsunori Matsui",
year = "2017",
month = "1",
day = "1",
language = "English",
isbn = "9789869401265",
pages = "56--61",
editor = "{Mohd Ayub}, {Ahmad Fauzi} and Antonija Mitrovic and Jie-Chi Yang and Wong, {Su Luan} and Wenli Chen",
booktitle = "Proceedings of the 25th International Conference on Computers in Education, ICCE 2017 - Main Conference Proceedings",
publisher = "Asia-Pacific Society for Computers in Education",

}

TY - GEN

T1 - Extraction of relationships between learners' physiological information and learners' mental states by machine learning

AU - Tawatsuji, Yoshimasa

AU - Uno, Tatsuro

AU - Okazaki, Keita

AU - Fang, Siyuan

AU - Matsui, Tatsunori

PY - 2017/1/1

Y1 - 2017/1/1

N2 - The estimation of learners' mental states during the interaction between teachers and learners is a very important problem in improving the quality of teaching and learning. In this experimental study, we developed a deep learning neural network (DLNN) system that extracted the relationships between a learner's mental states and a teacher's utterances plus the learner's physiological information. The learner's physiological information consisted of the NIRS signals, the EEG signals, respiration intensity, skin conductance, and pulse volume. The learner's mental states were elicited through the learner's introspective reports using the Achievement Emotions Questionnaire (AEQ). According to the AEQ, the learner's mental states were divided into nine categories: Enjoy, Hope, Pride, Anger, Anxiety, Shame, Hopelessness, Boredom, and Others. In a simulation, the DLNN system exhibited the ability to estimate the learner's mental states from the learner's physiological information with high accuracy.

AB - The estimation of learners' mental states during the interaction between teachers and learners is a very important problem in improving the quality of teaching and learning. In this experimental study, we developed a deep learning neural network (DLNN) system that extracted the relationships between a learner's mental states and a teacher's utterances plus the learner's physiological information. The learner's physiological information consisted of the NIRS signals, the EEG signals, respiration intensity, skin conductance, and pulse volume. The learner's mental states were elicited through the learner's introspective reports using the Achievement Emotions Questionnaire (AEQ). According to the AEQ, the learner's mental states were divided into nine categories: Enjoy, Hope, Pride, Anger, Anxiety, Shame, Hopelessness, Boredom, and Others. In a simulation, the DLNN system exhibited the ability to estimate the learner's mental states from the learner's physiological information with high accuracy.

KW - Deep learning

KW - Intelligent Mentoring System

KW - Mental state estimation

KW - Physiological data

UR - http://www.scopus.com/inward/record.url?scp=85053881748&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85053881748&partnerID=8YFLogxK

M3 - Conference contribution

SN - 9789869401265

SP - 56

EP - 61

BT - Proceedings of the 25th International Conference on Computers in Education, ICCE 2017 - Main Conference Proceedings

A2 - Mohd Ayub, Ahmad Fauzi

A2 - Mitrovic, Antonija

A2 - Yang, Jie-Chi

A2 - Wong, Su Luan

A2 - Chen, Wenli

PB - Asia-Pacific Society for Computers in Education

ER -