A driver situational awareness estimation system based on standard glance model for unscheduled takeover situations

Hiroaki Hayashi, Mitsuhiro Kamezaki, Udara E. Manawadu, Takahiro Kawano, Takaaki Ema, Tomoya Tomita, Lollett Catherine, Shigeki Sugano

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Highly-automated vehicles operating in level 3 issue a takeover request (TOR) to transfer the control authority from the autonomated driving (AD) system to a human driver when they encounter system limitations. In such 'unscheduled' situations, the driver is required to immediately re-engage in the driving task both physically and cognitively, and perform suitable action, e.g. lane change. Thus, evaluating driver engagement by the AD system would lead to safe takeover. Physical engagement is easily estimated but there are few studies on evaluating cognitive engagement. In this study, we thus developed a driver situational awareness estimation system based on glance information. We first defined seven standard glance areas and driver glance classification model using a convolutional neural network. We then obtained a large amount of glance data when both safe and dangerous takeover situations (lane change) by using a driving simulator, and we derived the standard glance model including the glance area and time, in order to estimate whether driver gained enough cognitive re-engagement in real-time. To evaluate the effectiveness of the proposed model, we created a situational awareness assist system to visually indicate regions with insufficient glance. As a result, we found that the assist system drastically improved driving performance and reduced the number of accidents during takeover.

Original languageEnglish
Title of host publication2019 IEEE Intelligent Vehicles Symposium, IV 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages798-803
Number of pages6
ISBN (Electronic)9781728105604
DOIs
Publication statusPublished - 2019 Jun 1
Event30th IEEE Intelligent Vehicles Symposium, IV 2019 - Paris, France
Duration: 2019 Jun 92019 Jun 12

Publication series

NameIEEE Intelligent Vehicles Symposium, Proceedings
Volume2019-June

Conference

Conference30th IEEE Intelligent Vehicles Symposium, IV 2019
CountryFrance
CityParis
Period19/6/919/6/12

Fingerprint

Situational Awareness
Driver
Standard Model
Accidents
Simulators
Driving Simulator
Neural networks
Immediately
Neural Networks
Real-time
Engagement
Evaluate
Model
Estimate

ASJC Scopus subject areas

  • Computer Science Applications
  • Automotive Engineering
  • Modelling and Simulation

Cite this

Hayashi, H., Kamezaki, M., Manawadu, U. E., Kawano, T., Ema, T., Tomita, T., ... Sugano, S. (2019). A driver situational awareness estimation system based on standard glance model for unscheduled takeover situations. In 2019 IEEE Intelligent Vehicles Symposium, IV 2019 (pp. 798-803). [8814067] (IEEE Intelligent Vehicles Symposium, Proceedings; Vol. 2019-June). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IVS.2019.8814067

A driver situational awareness estimation system based on standard glance model for unscheduled takeover situations. / Hayashi, Hiroaki; Kamezaki, Mitsuhiro; Manawadu, Udara E.; Kawano, Takahiro; Ema, Takaaki; Tomita, Tomoya; Catherine, Lollett; Sugano, Shigeki.

2019 IEEE Intelligent Vehicles Symposium, IV 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 798-803 8814067 (IEEE Intelligent Vehicles Symposium, Proceedings; Vol. 2019-June).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Hayashi, H, Kamezaki, M, Manawadu, UE, Kawano, T, Ema, T, Tomita, T, Catherine, L & Sugano, S 2019, A driver situational awareness estimation system based on standard glance model for unscheduled takeover situations. in 2019 IEEE Intelligent Vehicles Symposium, IV 2019., 8814067, IEEE Intelligent Vehicles Symposium, Proceedings, vol. 2019-June, Institute of Electrical and Electronics Engineers Inc., pp. 798-803, 30th IEEE Intelligent Vehicles Symposium, IV 2019, Paris, France, 19/6/9. https://doi.org/10.1109/IVS.2019.8814067
Hayashi H, Kamezaki M, Manawadu UE, Kawano T, Ema T, Tomita T et al. A driver situational awareness estimation system based on standard glance model for unscheduled takeover situations. In 2019 IEEE Intelligent Vehicles Symposium, IV 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 798-803. 8814067. (IEEE Intelligent Vehicles Symposium, Proceedings). https://doi.org/10.1109/IVS.2019.8814067
Hayashi, Hiroaki ; Kamezaki, Mitsuhiro ; Manawadu, Udara E. ; Kawano, Takahiro ; Ema, Takaaki ; Tomita, Tomoya ; Catherine, Lollett ; Sugano, Shigeki. / A driver situational awareness estimation system based on standard glance model for unscheduled takeover situations. 2019 IEEE Intelligent Vehicles Symposium, IV 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 798-803 (IEEE Intelligent Vehicles Symposium, Proceedings).
@inproceedings{974ae580a4104841b9ad868be1b5b454,
title = "A driver situational awareness estimation system based on standard glance model for unscheduled takeover situations",
abstract = "Highly-automated vehicles operating in level 3 issue a takeover request (TOR) to transfer the control authority from the autonomated driving (AD) system to a human driver when they encounter system limitations. In such 'unscheduled' situations, the driver is required to immediately re-engage in the driving task both physically and cognitively, and perform suitable action, e.g. lane change. Thus, evaluating driver engagement by the AD system would lead to safe takeover. Physical engagement is easily estimated but there are few studies on evaluating cognitive engagement. In this study, we thus developed a driver situational awareness estimation system based on glance information. We first defined seven standard glance areas and driver glance classification model using a convolutional neural network. We then obtained a large amount of glance data when both safe and dangerous takeover situations (lane change) by using a driving simulator, and we derived the standard glance model including the glance area and time, in order to estimate whether driver gained enough cognitive re-engagement in real-time. To evaluate the effectiveness of the proposed model, we created a situational awareness assist system to visually indicate regions with insufficient glance. As a result, we found that the assist system drastically improved driving performance and reduced the number of accidents during takeover.",
author = "Hiroaki Hayashi and Mitsuhiro Kamezaki and Manawadu, {Udara E.} and Takahiro Kawano and Takaaki Ema and Tomoya Tomita and Lollett Catherine and Shigeki Sugano",
year = "2019",
month = "6",
day = "1",
doi = "10.1109/IVS.2019.8814067",
language = "English",
series = "IEEE Intelligent Vehicles Symposium, Proceedings",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "798--803",
booktitle = "2019 IEEE Intelligent Vehicles Symposium, IV 2019",

}

TY - GEN

T1 - A driver situational awareness estimation system based on standard glance model for unscheduled takeover situations

AU - Hayashi, Hiroaki

AU - Kamezaki, Mitsuhiro

AU - Manawadu, Udara E.

AU - Kawano, Takahiro

AU - Ema, Takaaki

AU - Tomita, Tomoya

AU - Catherine, Lollett

AU - Sugano, Shigeki

PY - 2019/6/1

Y1 - 2019/6/1

N2 - Highly-automated vehicles operating in level 3 issue a takeover request (TOR) to transfer the control authority from the autonomated driving (AD) system to a human driver when they encounter system limitations. In such 'unscheduled' situations, the driver is required to immediately re-engage in the driving task both physically and cognitively, and perform suitable action, e.g. lane change. Thus, evaluating driver engagement by the AD system would lead to safe takeover. Physical engagement is easily estimated but there are few studies on evaluating cognitive engagement. In this study, we thus developed a driver situational awareness estimation system based on glance information. We first defined seven standard glance areas and driver glance classification model using a convolutional neural network. We then obtained a large amount of glance data when both safe and dangerous takeover situations (lane change) by using a driving simulator, and we derived the standard glance model including the glance area and time, in order to estimate whether driver gained enough cognitive re-engagement in real-time. To evaluate the effectiveness of the proposed model, we created a situational awareness assist system to visually indicate regions with insufficient glance. As a result, we found that the assist system drastically improved driving performance and reduced the number of accidents during takeover.

AB - Highly-automated vehicles operating in level 3 issue a takeover request (TOR) to transfer the control authority from the autonomated driving (AD) system to a human driver when they encounter system limitations. In such 'unscheduled' situations, the driver is required to immediately re-engage in the driving task both physically and cognitively, and perform suitable action, e.g. lane change. Thus, evaluating driver engagement by the AD system would lead to safe takeover. Physical engagement is easily estimated but there are few studies on evaluating cognitive engagement. In this study, we thus developed a driver situational awareness estimation system based on glance information. We first defined seven standard glance areas and driver glance classification model using a convolutional neural network. We then obtained a large amount of glance data when both safe and dangerous takeover situations (lane change) by using a driving simulator, and we derived the standard glance model including the glance area and time, in order to estimate whether driver gained enough cognitive re-engagement in real-time. To evaluate the effectiveness of the proposed model, we created a situational awareness assist system to visually indicate regions with insufficient glance. As a result, we found that the assist system drastically improved driving performance and reduced the number of accidents during takeover.

UR - http://www.scopus.com/inward/record.url?scp=85072283873&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85072283873&partnerID=8YFLogxK

U2 - 10.1109/IVS.2019.8814067

DO - 10.1109/IVS.2019.8814067

M3 - Conference contribution

AN - SCOPUS:85072283873

T3 - IEEE Intelligent Vehicles Symposium, Proceedings

SP - 798

EP - 803

BT - 2019 IEEE Intelligent Vehicles Symposium, IV 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -