Detecting and tracking surgical tools for recognizing phases of the awake brain tumor removal surgery

Hiroki Fujie, Keiju Hirata, Takahiro Horigome, Hiroshi Nagahashi, Jun Ohya, Manabu Tamura, Ken Masamune, Yoshihiro Muragaki

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In order to realize automatic recognition of surgical processes in surgical brain tumor removal using microscopic camera, we propose a method of detecting and tracking surgical tools by video analysis. The proposed method consists of a detection part and tracking part. In the detection part, object detection is performed for each frame of surgery video, and the category and bounding box are acquired frame by frame. The convolution layer strengthens the robustness using data augmentation (central cropping and random erasing). The tracking part uses SORT, which predicts and updates the acquired bounding box corrected by using Kalman Filter; next, the object ID is assigned to each corrected bounding box using the Hungarian algorithm. The accuracy of our proposed method is very high as follows. As a result of experiments on spatial detection. the mean average precision is 90.58%. the mean accuracy of frame label detection is 96.58%. These results are very promising for surgical phase recognition.

Original languageEnglish
Title of host publicationICPRAM 2019 - Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods
EditorsAna Fred, Maria De Marsico, Gabriella Sanniti di Baja
PublisherSciTePress
Pages190-199
Number of pages10
ISBN (Electronic)9789897583513
Publication statusPublished - 2019 Jan 1
Event8th International Conference on Pattern Recognition Applications and Methods, ICPRAM 2019 - Prague, Czech Republic
Duration: 2019 Feb 192019 Feb 21

Publication series

NameICPRAM 2019 - Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods

Conference

Conference8th International Conference on Pattern Recognition Applications and Methods, ICPRAM 2019
CountryCzech Republic
CityPrague
Period19/2/1919/2/21

Fingerprint

Convolution
Kalman filters
Surgery
Labels
Tumors
Brain
Cameras
Experiments
Object detection

Keywords

  • Awake Brain Tumor Removal Surgery
  • Computer Vision
  • Convolutional Neural Network
  • Data Association
  • Data Augmentation
  • Detection
  • Multiple Object Tracking

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition

Cite this

Fujie, H., Hirata, K., Horigome, T., Nagahashi, H., Ohya, J., Tamura, M., ... Muragaki, Y. (2019). Detecting and tracking surgical tools for recognizing phases of the awake brain tumor removal surgery. In A. Fred, M. De Marsico, & G. S. di Baja (Eds.), ICPRAM 2019 - Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods (pp. 190-199). (ICPRAM 2019 - Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods). SciTePress.

Detecting and tracking surgical tools for recognizing phases of the awake brain tumor removal surgery. / Fujie, Hiroki; Hirata, Keiju; Horigome, Takahiro; Nagahashi, Hiroshi; Ohya, Jun; Tamura, Manabu; Masamune, Ken; Muragaki, Yoshihiro.

ICPRAM 2019 - Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods. ed. / Ana Fred; Maria De Marsico; Gabriella Sanniti di Baja. SciTePress, 2019. p. 190-199 (ICPRAM 2019 - Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Fujie, H, Hirata, K, Horigome, T, Nagahashi, H, Ohya, J, Tamura, M, Masamune, K & Muragaki, Y 2019, Detecting and tracking surgical tools for recognizing phases of the awake brain tumor removal surgery. in A Fred, M De Marsico & GS di Baja (eds), ICPRAM 2019 - Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods. ICPRAM 2019 - Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods, SciTePress, pp. 190-199, 8th International Conference on Pattern Recognition Applications and Methods, ICPRAM 2019, Prague, Czech Republic, 19/2/19.
Fujie H, Hirata K, Horigome T, Nagahashi H, Ohya J, Tamura M et al. Detecting and tracking surgical tools for recognizing phases of the awake brain tumor removal surgery. In Fred A, De Marsico M, di Baja GS, editors, ICPRAM 2019 - Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods. SciTePress. 2019. p. 190-199. (ICPRAM 2019 - Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods).
Fujie, Hiroki ; Hirata, Keiju ; Horigome, Takahiro ; Nagahashi, Hiroshi ; Ohya, Jun ; Tamura, Manabu ; Masamune, Ken ; Muragaki, Yoshihiro. / Detecting and tracking surgical tools for recognizing phases of the awake brain tumor removal surgery. ICPRAM 2019 - Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods. editor / Ana Fred ; Maria De Marsico ; Gabriella Sanniti di Baja. SciTePress, 2019. pp. 190-199 (ICPRAM 2019 - Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods).
@inproceedings{5e6bfc430a7b45d2ab0ec23bf075524b,
title = "Detecting and tracking surgical tools for recognizing phases of the awake brain tumor removal surgery",
abstract = "In order to realize automatic recognition of surgical processes in surgical brain tumor removal using microscopic camera, we propose a method of detecting and tracking surgical tools by video analysis. The proposed method consists of a detection part and tracking part. In the detection part, object detection is performed for each frame of surgery video, and the category and bounding box are acquired frame by frame. The convolution layer strengthens the robustness using data augmentation (central cropping and random erasing). The tracking part uses SORT, which predicts and updates the acquired bounding box corrected by using Kalman Filter; next, the object ID is assigned to each corrected bounding box using the Hungarian algorithm. The accuracy of our proposed method is very high as follows. As a result of experiments on spatial detection. the mean average precision is 90.58{\%}. the mean accuracy of frame label detection is 96.58{\%}. These results are very promising for surgical phase recognition.",
keywords = "Awake Brain Tumor Removal Surgery, Computer Vision, Convolutional Neural Network, Data Association, Data Augmentation, Detection, Multiple Object Tracking",
author = "Hiroki Fujie and Keiju Hirata and Takahiro Horigome and Hiroshi Nagahashi and Jun Ohya and Manabu Tamura and Ken Masamune and Yoshihiro Muragaki",
year = "2019",
month = "1",
day = "1",
language = "English",
series = "ICPRAM 2019 - Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods",
publisher = "SciTePress",
pages = "190--199",
editor = "Ana Fred and {De Marsico}, Maria and {di Baja}, {Gabriella Sanniti}",
booktitle = "ICPRAM 2019 - Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods",

}

TY - GEN

T1 - Detecting and tracking surgical tools for recognizing phases of the awake brain tumor removal surgery

AU - Fujie, Hiroki

AU - Hirata, Keiju

AU - Horigome, Takahiro

AU - Nagahashi, Hiroshi

AU - Ohya, Jun

AU - Tamura, Manabu

AU - Masamune, Ken

AU - Muragaki, Yoshihiro

PY - 2019/1/1

Y1 - 2019/1/1

N2 - In order to realize automatic recognition of surgical processes in surgical brain tumor removal using microscopic camera, we propose a method of detecting and tracking surgical tools by video analysis. The proposed method consists of a detection part and tracking part. In the detection part, object detection is performed for each frame of surgery video, and the category and bounding box are acquired frame by frame. The convolution layer strengthens the robustness using data augmentation (central cropping and random erasing). The tracking part uses SORT, which predicts and updates the acquired bounding box corrected by using Kalman Filter; next, the object ID is assigned to each corrected bounding box using the Hungarian algorithm. The accuracy of our proposed method is very high as follows. As a result of experiments on spatial detection. the mean average precision is 90.58%. the mean accuracy of frame label detection is 96.58%. These results are very promising for surgical phase recognition.

AB - In order to realize automatic recognition of surgical processes in surgical brain tumor removal using microscopic camera, we propose a method of detecting and tracking surgical tools by video analysis. The proposed method consists of a detection part and tracking part. In the detection part, object detection is performed for each frame of surgery video, and the category and bounding box are acquired frame by frame. The convolution layer strengthens the robustness using data augmentation (central cropping and random erasing). The tracking part uses SORT, which predicts and updates the acquired bounding box corrected by using Kalman Filter; next, the object ID is assigned to each corrected bounding box using the Hungarian algorithm. The accuracy of our proposed method is very high as follows. As a result of experiments on spatial detection. the mean average precision is 90.58%. the mean accuracy of frame label detection is 96.58%. These results are very promising for surgical phase recognition.

KW - Awake Brain Tumor Removal Surgery

KW - Computer Vision

KW - Convolutional Neural Network

KW - Data Association

KW - Data Augmentation

KW - Detection

KW - Multiple Object Tracking

UR - http://www.scopus.com/inward/record.url?scp=85064675027&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85064675027&partnerID=8YFLogxK

M3 - Conference contribution

T3 - ICPRAM 2019 - Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods

SP - 190

EP - 199

BT - ICPRAM 2019 - Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods

A2 - Fred, Ana

A2 - De Marsico, Maria

A2 - di Baja, Gabriella Sanniti

PB - SciTePress

ER -