Driver confusion status detection using recurrent neural networks

Chiori Hori; Shinji Watanabe; Takaaki Hori; Bret A. Harsham; Johnr Hershey; Yusuke Koji; Yoichi Fujii; Yuki Furumoto

doi:10.1109/ICME.2016.7552966

Driver confusion status detection using recurrent neural networks

Chiori Hori, Shinji Watanabe, Takaaki Hori, Bret A. Harsham, Johnr Hershey, Yusuke Koji, Yoichi Fujii, Yuki Furumoto

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

7 Citations (Scopus)

Abstract

In this paper, we present a method for estimating the confusion level of a driver using a classifier trained on multimodal sensor data. Using the driver confusion status detector, a car navigation system can proactively support the driver when he/she is confused. A corpus of data was collected during on-road driving in traffic using a navigation system and a car instrumented with a variety of sensors. The data was manually annotated with the driver's confusion status and with multiple features representing driver's behavior and the traffic conditions. We compared different types of classifiers trained from the data: logistic regression, a feed-forward neural network, a recurrent neural networks, and a long short-term memory (LSTM)-based recurrent neural network. The accuracy was evaluated using F-max as well as precision/recall. We found that the LSTM outperformed the other models.

Original language	English
Title of host publication	2016 IEEE International Conference on Multimedia and Expo, ICME 2016
Publisher	IEEE Computer Society
Volume	2016-August
ISBN (Electronic)	9781467372589
DOIs	https://doi.org/10.1109/ICME.2016.7552966
Publication status	Published - 2016 Aug 25
Externally published	Yes
Event	2016 IEEE International Conference on Multimedia and Expo, ICME 2016 - Seattle, United States Duration: 2016 Jul 11 → 2016 Jul 15

Other

Other	2016 IEEE International Conference on Multimedia and Expo, ICME 2016
Country/Territory	United States
City	Seattle
Period	16/7/11 → 16/7/15

Keywords

driver confusion status prediction
long short-term memory
multimodal processing
recurrent neural network

ASJC Scopus subject areas

Computer Networks and Communications
Computer Science Applications

Access to Document

10.1109/ICME.2016.7552966

Cite this

Driver confusion status detection using recurrent neural networks. / Hori, Chiori; Watanabe, Shinji; Hori, Takaaki; Harsham, Bret A.; Hershey, Johnr; Koji, Yusuke; Fujii, Yoichi; Furumoto, Yuki.

2016 IEEE International Conference on Multimedia and Expo, ICME 2016. Vol. 2016-August IEEE Computer Society, 2016. 7552966.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Hori, C, Watanabe, S, Hori, T, Harsham, BA, Hershey, J, Koji, Y, Fujii, Y & Furumoto, Y 2016, Driver confusion status detection using recurrent neural networks. in 2016 IEEE International Conference on Multimedia and Expo, ICME 2016. vol. 2016-August, 7552966, IEEE Computer Society, 2016 IEEE International Conference on Multimedia and Expo, ICME 2016, Seattle, United States, 16/7/11. https://doi.org/10.1109/ICME.2016.7552966

@inproceedings{bed4ffc64cac4fa59ebc0f92f1442542,

title = "Driver confusion status detection using recurrent neural networks",

abstract = "In this paper, we present a method for estimating the confusion level of a driver using a classifier trained on multimodal sensor data. Using the driver confusion status detector, a car navigation system can proactively support the driver when he/she is confused. A corpus of data was collected during on-road driving in traffic using a navigation system and a car instrumented with a variety of sensors. The data was manually annotated with the driver's confusion status and with multiple features representing driver's behavior and the traffic conditions. We compared different types of classifiers trained from the data: logistic regression, a feed-forward neural network, a recurrent neural networks, and a long short-term memory (LSTM)-based recurrent neural network. The accuracy was evaluated using F-max as well as precision/recall. We found that the LSTM outperformed the other models.",

keywords = "driver confusion status prediction, long short-term memory, multimodal processing, recurrent neural network",

author = "Chiori Hori and Shinji Watanabe and Takaaki Hori and Harsham, {Bret A.} and Johnr Hershey and Yusuke Koji and Yoichi Fujii and Yuki Furumoto",

year = "2016",

month = aug,

day = "25",

doi = "10.1109/ICME.2016.7552966",

language = "English",

volume = "2016-August",

booktitle = "2016 IEEE International Conference on Multimedia and Expo, ICME 2016",

publisher = "IEEE Computer Society",

address = "United States",

note = "2016 IEEE International Conference on Multimedia and Expo, ICME 2016 ; Conference date: 11-07-2016 Through 15-07-2016",

}

TY - GEN

T1 - Driver confusion status detection using recurrent neural networks

AU - Hori, Chiori

AU - Watanabe, Shinji

AU - Hori, Takaaki

AU - Harsham, Bret A.

AU - Hershey, Johnr

AU - Koji, Yusuke

AU - Fujii, Yoichi

AU - Furumoto, Yuki

PY - 2016/8/25

Y1 - 2016/8/25

N2 - In this paper, we present a method for estimating the confusion level of a driver using a classifier trained on multimodal sensor data. Using the driver confusion status detector, a car navigation system can proactively support the driver when he/she is confused. A corpus of data was collected during on-road driving in traffic using a navigation system and a car instrumented with a variety of sensors. The data was manually annotated with the driver's confusion status and with multiple features representing driver's behavior and the traffic conditions. We compared different types of classifiers trained from the data: logistic regression, a feed-forward neural network, a recurrent neural networks, and a long short-term memory (LSTM)-based recurrent neural network. The accuracy was evaluated using F-max as well as precision/recall. We found that the LSTM outperformed the other models.

AB - In this paper, we present a method for estimating the confusion level of a driver using a classifier trained on multimodal sensor data. Using the driver confusion status detector, a car navigation system can proactively support the driver when he/she is confused. A corpus of data was collected during on-road driving in traffic using a navigation system and a car instrumented with a variety of sensors. The data was manually annotated with the driver's confusion status and with multiple features representing driver's behavior and the traffic conditions. We compared different types of classifiers trained from the data: logistic regression, a feed-forward neural network, a recurrent neural networks, and a long short-term memory (LSTM)-based recurrent neural network. The accuracy was evaluated using F-max as well as precision/recall. We found that the LSTM outperformed the other models.

KW - driver confusion status prediction

KW - long short-term memory

KW - multimodal processing

KW - recurrent neural network

UR - http://www.scopus.com/inward/record.url?scp=84987606048&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84987606048&partnerID=8YFLogxK

U2 - 10.1109/ICME.2016.7552966

DO - 10.1109/ICME.2016.7552966

M3 - Conference contribution

AN - SCOPUS:84987606048

VL - 2016-August

BT - 2016 IEEE International Conference on Multimedia and Expo, ICME 2016

PB - IEEE Computer Society

T2 - 2016 IEEE International Conference on Multimedia and Expo, ICME 2016

Y2 - 11 July 2016 through 15 July 2016

ER -

Driver confusion status detection using recurrent neural networks

Abstract

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this