Development and Evaluation of a Kinect-Based Motion Recognition System based on Kalman Filter for Upper-Limb Assistive Device

Yun Ting Liao, Hao Yang, Hee Hyol Lee, Eiichiro Tanaka

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

There is a population in the world who loses the function of upper extremity due to the accidence or disease. The upper-extremity disorders significantly reduce the people's quality of life due to losing the ability to carry out the activities of daily living, which mostly require the upper-limb function. Therefore, the needs of the upper-limb assistance devices for the upper extremity increased. In this research, we proposed a motion intention recognition system based on the Kinect® v2 sensor. The sensor directly detected the user's motion and further control the device with the corresponding angles instead of using the pre-trajectory to control the device. Since the body dimensions have the individual difference, we considered the unconstrained user-device interface by using two pressure sensor trays on each robot arm to support the user's forearm and upper arm, respectively. The unconstrained user-device system can slightly compensate not only the individual difference but the control error. Therefore, the unconstrained user-device model was established to obtain the relationship between the user and the device, and further control the device using the recorded user's motion. Additionally, the Kinect® sensor can capture the coordination of human joints and further calculate the arm length of the user, which can realize the adaptivity of different user. To realize the real-time control and assistance, the Kalman filter which has prediction function was exploited. The feasibility of assistance was confirmed by the system response. The results proved that the proposed motion recognition system and the unconstrained user-device system can successfully provide adequate assistance with a lesser time delay compared with the system without Kalman filter.

Original languageEnglish
Title of host publication2019 58th Annual Conference of the Society of Instrument and Control Engineers of Japan, SICE 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1621-1626
Number of pages6
ISBN (Electronic)9784907764678
DOIs
Publication statusPublished - 2019 Sep
Event58th Annual Conference of the Society of Instrument and Control Engineers of Japan, SICE 2019 - Hiroshima, Japan
Duration: 2019 Sep 102019 Sep 13

Publication series

Name2019 58th Annual Conference of the Society of Instrument and Control Engineers of Japan, SICE 2019

Conference

Conference58th Annual Conference of the Society of Instrument and Control Engineers of Japan, SICE 2019
CountryJapan
CityHiroshima
Period19/9/1019/9/13

Fingerprint

Kalman filters
limbs
Kalman Filter
Motion
evaluation
Evaluation
Sensors
Real time control
Pressure sensors
Individual Differences
Time delay
Sensor
sensors
Trajectories
Robots
trays
forearm
Corresponding angles
robot arms
Pressure Sensor

Keywords

  • Activities of Daily Living
  • Human Motion Recognition
  • Kinect Sensor
  • Upper-Limb Assistance
  • Wheelchair-based Assistive Device

ASJC Scopus subject areas

  • Artificial Intelligence
  • Industrial and Manufacturing Engineering
  • Safety, Risk, Reliability and Quality
  • Control and Optimization
  • Instrumentation

Cite this

Liao, Y. T., Yang, H., Lee, H. H., & Tanaka, E. (2019). Development and Evaluation of a Kinect-Based Motion Recognition System based on Kalman Filter for Upper-Limb Assistive Device. In 2019 58th Annual Conference of the Society of Instrument and Control Engineers of Japan, SICE 2019 (pp. 1621-1626). [8859744] (2019 58th Annual Conference of the Society of Instrument and Control Engineers of Japan, SICE 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.23919/SICE.2019.8859744

Development and Evaluation of a Kinect-Based Motion Recognition System based on Kalman Filter for Upper-Limb Assistive Device. / Liao, Yun Ting; Yang, Hao; Lee, Hee Hyol; Tanaka, Eiichiro.

2019 58th Annual Conference of the Society of Instrument and Control Engineers of Japan, SICE 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 1621-1626 8859744 (2019 58th Annual Conference of the Society of Instrument and Control Engineers of Japan, SICE 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Liao, YT, Yang, H, Lee, HH & Tanaka, E 2019, Development and Evaluation of a Kinect-Based Motion Recognition System based on Kalman Filter for Upper-Limb Assistive Device. in 2019 58th Annual Conference of the Society of Instrument and Control Engineers of Japan, SICE 2019., 8859744, 2019 58th Annual Conference of the Society of Instrument and Control Engineers of Japan, SICE 2019, Institute of Electrical and Electronics Engineers Inc., pp. 1621-1626, 58th Annual Conference of the Society of Instrument and Control Engineers of Japan, SICE 2019, Hiroshima, Japan, 19/9/10. https://doi.org/10.23919/SICE.2019.8859744
Liao YT, Yang H, Lee HH, Tanaka E. Development and Evaluation of a Kinect-Based Motion Recognition System based on Kalman Filter for Upper-Limb Assistive Device. In 2019 58th Annual Conference of the Society of Instrument and Control Engineers of Japan, SICE 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 1621-1626. 8859744. (2019 58th Annual Conference of the Society of Instrument and Control Engineers of Japan, SICE 2019). https://doi.org/10.23919/SICE.2019.8859744
Liao, Yun Ting ; Yang, Hao ; Lee, Hee Hyol ; Tanaka, Eiichiro. / Development and Evaluation of a Kinect-Based Motion Recognition System based on Kalman Filter for Upper-Limb Assistive Device. 2019 58th Annual Conference of the Society of Instrument and Control Engineers of Japan, SICE 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 1621-1626 (2019 58th Annual Conference of the Society of Instrument and Control Engineers of Japan, SICE 2019).
@inproceedings{985b8f7ee3e047388ac7a414e081bbc4,
title = "Development and Evaluation of a Kinect-Based Motion Recognition System based on Kalman Filter for Upper-Limb Assistive Device",
abstract = "There is a population in the world who loses the function of upper extremity due to the accidence or disease. The upper-extremity disorders significantly reduce the people's quality of life due to losing the ability to carry out the activities of daily living, which mostly require the upper-limb function. Therefore, the needs of the upper-limb assistance devices for the upper extremity increased. In this research, we proposed a motion intention recognition system based on the Kinect{\circledR} v2 sensor. The sensor directly detected the user's motion and further control the device with the corresponding angles instead of using the pre-trajectory to control the device. Since the body dimensions have the individual difference, we considered the unconstrained user-device interface by using two pressure sensor trays on each robot arm to support the user's forearm and upper arm, respectively. The unconstrained user-device system can slightly compensate not only the individual difference but the control error. Therefore, the unconstrained user-device model was established to obtain the relationship between the user and the device, and further control the device using the recorded user's motion. Additionally, the Kinect{\circledR} sensor can capture the coordination of human joints and further calculate the arm length of the user, which can realize the adaptivity of different user. To realize the real-time control and assistance, the Kalman filter which has prediction function was exploited. The feasibility of assistance was confirmed by the system response. The results proved that the proposed motion recognition system and the unconstrained user-device system can successfully provide adequate assistance with a lesser time delay compared with the system without Kalman filter.",
keywords = "Activities of Daily Living, Human Motion Recognition, Kinect Sensor, Upper-Limb Assistance, Wheelchair-based Assistive Device",
author = "Liao, {Yun Ting} and Hao Yang and Lee, {Hee Hyol} and Eiichiro Tanaka",
year = "2019",
month = "9",
doi = "10.23919/SICE.2019.8859744",
language = "English",
series = "2019 58th Annual Conference of the Society of Instrument and Control Engineers of Japan, SICE 2019",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "1621--1626",
booktitle = "2019 58th Annual Conference of the Society of Instrument and Control Engineers of Japan, SICE 2019",

}

TY - GEN

T1 - Development and Evaluation of a Kinect-Based Motion Recognition System based on Kalman Filter for Upper-Limb Assistive Device

AU - Liao, Yun Ting

AU - Yang, Hao

AU - Lee, Hee Hyol

AU - Tanaka, Eiichiro

PY - 2019/9

Y1 - 2019/9

N2 - There is a population in the world who loses the function of upper extremity due to the accidence or disease. The upper-extremity disorders significantly reduce the people's quality of life due to losing the ability to carry out the activities of daily living, which mostly require the upper-limb function. Therefore, the needs of the upper-limb assistance devices for the upper extremity increased. In this research, we proposed a motion intention recognition system based on the Kinect® v2 sensor. The sensor directly detected the user's motion and further control the device with the corresponding angles instead of using the pre-trajectory to control the device. Since the body dimensions have the individual difference, we considered the unconstrained user-device interface by using two pressure sensor trays on each robot arm to support the user's forearm and upper arm, respectively. The unconstrained user-device system can slightly compensate not only the individual difference but the control error. Therefore, the unconstrained user-device model was established to obtain the relationship between the user and the device, and further control the device using the recorded user's motion. Additionally, the Kinect® sensor can capture the coordination of human joints and further calculate the arm length of the user, which can realize the adaptivity of different user. To realize the real-time control and assistance, the Kalman filter which has prediction function was exploited. The feasibility of assistance was confirmed by the system response. The results proved that the proposed motion recognition system and the unconstrained user-device system can successfully provide adequate assistance with a lesser time delay compared with the system without Kalman filter.

AB - There is a population in the world who loses the function of upper extremity due to the accidence or disease. The upper-extremity disorders significantly reduce the people's quality of life due to losing the ability to carry out the activities of daily living, which mostly require the upper-limb function. Therefore, the needs of the upper-limb assistance devices for the upper extremity increased. In this research, we proposed a motion intention recognition system based on the Kinect® v2 sensor. The sensor directly detected the user's motion and further control the device with the corresponding angles instead of using the pre-trajectory to control the device. Since the body dimensions have the individual difference, we considered the unconstrained user-device interface by using two pressure sensor trays on each robot arm to support the user's forearm and upper arm, respectively. The unconstrained user-device system can slightly compensate not only the individual difference but the control error. Therefore, the unconstrained user-device model was established to obtain the relationship between the user and the device, and further control the device using the recorded user's motion. Additionally, the Kinect® sensor can capture the coordination of human joints and further calculate the arm length of the user, which can realize the adaptivity of different user. To realize the real-time control and assistance, the Kalman filter which has prediction function was exploited. The feasibility of assistance was confirmed by the system response. The results proved that the proposed motion recognition system and the unconstrained user-device system can successfully provide adequate assistance with a lesser time delay compared with the system without Kalman filter.

KW - Activities of Daily Living

KW - Human Motion Recognition

KW - Kinect Sensor

KW - Upper-Limb Assistance

KW - Wheelchair-based Assistive Device

UR - http://www.scopus.com/inward/record.url?scp=85073877244&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85073877244&partnerID=8YFLogxK

U2 - 10.23919/SICE.2019.8859744

DO - 10.23919/SICE.2019.8859744

M3 - Conference contribution

AN - SCOPUS:85073877244

T3 - 2019 58th Annual Conference of the Society of Instrument and Control Engineers of Japan, SICE 2019

SP - 1621

EP - 1626

BT - 2019 58th Annual Conference of the Society of Instrument and Control Engineers of Japan, SICE 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -