Manga character clustering with DBSCAN using fine-tuned CNN model

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Manga (Japanese comic) is popular content worldwide. In Japan, e-comic accounts for about 80% of e-book market. In recent years, metadata extraction from manga image has been studied for providing e-comic service. Manga character is one of the important contents for story understanding. In conventional research, some character identification methods are proposed those classify characters' face images using k-means clustering. However, there are two problems. First, kmeans method needs to specify the number of clusters, however the number of characters in target manga images is commonly unknown. Second, manga includes characters with few appearing, so it is difficult to classify characters with high purity. To solve these problems, we propose clustering method using DBSCAN which decides number of clusters automatically and is robust to noise data. In our prior research, it is experimented that character face clustering using DBSCAN and general CNN features. However, general CNN model is difficult to capture detailed features of manga characters. In this paper, we apply DBSCAN to fine-tuned CNN with manga character faces to improve the clustering accuracy. We also compare the optimal parameter determination method of DBSCAN. Experimental results showed that the dimensional reduction using Kernel PCA and UMAP is effective. In addition, we confirmed the validity of proposed method that determining the parameters of DBSCAN based on the slope changing of k-distance graph.

Original languageEnglish
Title of host publicationInternational Workshop on Advanced Image Technology, IWAIT 2019
EditorsQian Kemao, Yung-Lyul Lee, Kazuya Hayase, Phooi Yee Lau, Wen-Nung Lie, Lu Yu, Sanun Srisuk
PublisherSPIE
ISBN (Electronic)9781510627734
DOIs
Publication statusPublished - 2019 Jan 1
EventInternational Workshop on Advanced Image Technology 2019, IWAIT 2019 - Singapore, Singapore
Duration: 2019 Jan 62019 Jan 9

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
Volume11049
ISSN (Print)0277-786X
ISSN (Electronic)1996-756X

Conference

ConferenceInternational Workshop on Advanced Image Technology 2019, IWAIT 2019
CountrySingapore
CitySingapore
Period19/1/619/1/9

Fingerprint

Clustering
Metadata
metadata
Japan
purity
Model
Number of Clusters
slopes
Classify
Kernel PCA
Face
Image Clustering
Distance Graph
Character
Dimensional Reduction
K-means Clustering
K-means
Optimal Parameter
Clustering Methods
Slope

Keywords

  • clustering
  • CNN
  • DBSCAN
  • manga

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering

Cite this

Yanagisawa, H., Yamashita, T., & Watanabe, H. (2019). Manga character clustering with DBSCAN using fine-tuned CNN model. In Q. Kemao, Y-L. Lee, K. Hayase, P. Y. Lau, W-N. Lie, L. Yu, & S. Srisuk (Eds.), International Workshop on Advanced Image Technology, IWAIT 2019 [110491M] (Proceedings of SPIE - The International Society for Optical Engineering; Vol. 11049). SPIE. https://doi.org/10.1117/12.2521116

Manga character clustering with DBSCAN using fine-tuned CNN model. / Yanagisawa, Hideaki; Yamashita, Takuro; Watanabe, Hiroshi.

International Workshop on Advanced Image Technology, IWAIT 2019. ed. / Qian Kemao; Yung-Lyul Lee; Kazuya Hayase; Phooi Yee Lau; Wen-Nung Lie; Lu Yu; Sanun Srisuk. SPIE, 2019. 110491M (Proceedings of SPIE - The International Society for Optical Engineering; Vol. 11049).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yanagisawa, H, Yamashita, T & Watanabe, H 2019, Manga character clustering with DBSCAN using fine-tuned CNN model. in Q Kemao, Y-L Lee, K Hayase, PY Lau, W-N Lie, L Yu & S Srisuk (eds), International Workshop on Advanced Image Technology, IWAIT 2019., 110491M, Proceedings of SPIE - The International Society for Optical Engineering, vol. 11049, SPIE, International Workshop on Advanced Image Technology 2019, IWAIT 2019, Singapore, Singapore, 19/1/6. https://doi.org/10.1117/12.2521116
Yanagisawa H, Yamashita T, Watanabe H. Manga character clustering with DBSCAN using fine-tuned CNN model. In Kemao Q, Lee Y-L, Hayase K, Lau PY, Lie W-N, Yu L, Srisuk S, editors, International Workshop on Advanced Image Technology, IWAIT 2019. SPIE. 2019. 110491M. (Proceedings of SPIE - The International Society for Optical Engineering). https://doi.org/10.1117/12.2521116
Yanagisawa, Hideaki ; Yamashita, Takuro ; Watanabe, Hiroshi. / Manga character clustering with DBSCAN using fine-tuned CNN model. International Workshop on Advanced Image Technology, IWAIT 2019. editor / Qian Kemao ; Yung-Lyul Lee ; Kazuya Hayase ; Phooi Yee Lau ; Wen-Nung Lie ; Lu Yu ; Sanun Srisuk. SPIE, 2019. (Proceedings of SPIE - The International Society for Optical Engineering).
@inproceedings{ffbc41ce15984050bd458e435b7d9cc2,
title = "Manga character clustering with DBSCAN using fine-tuned CNN model",
abstract = "Manga (Japanese comic) is popular content worldwide. In Japan, e-comic accounts for about 80{\%} of e-book market. In recent years, metadata extraction from manga image has been studied for providing e-comic service. Manga character is one of the important contents for story understanding. In conventional research, some character identification methods are proposed those classify characters' face images using k-means clustering. However, there are two problems. First, kmeans method needs to specify the number of clusters, however the number of characters in target manga images is commonly unknown. Second, manga includes characters with few appearing, so it is difficult to classify characters with high purity. To solve these problems, we propose clustering method using DBSCAN which decides number of clusters automatically and is robust to noise data. In our prior research, it is experimented that character face clustering using DBSCAN and general CNN features. However, general CNN model is difficult to capture detailed features of manga characters. In this paper, we apply DBSCAN to fine-tuned CNN with manga character faces to improve the clustering accuracy. We also compare the optimal parameter determination method of DBSCAN. Experimental results showed that the dimensional reduction using Kernel PCA and UMAP is effective. In addition, we confirmed the validity of proposed method that determining the parameters of DBSCAN based on the slope changing of k-distance graph.",
keywords = "clustering, CNN, DBSCAN, manga",
author = "Hideaki Yanagisawa and Takuro Yamashita and Hiroshi Watanabe",
year = "2019",
month = "1",
day = "1",
doi = "10.1117/12.2521116",
language = "English",
series = "Proceedings of SPIE - The International Society for Optical Engineering",
publisher = "SPIE",
editor = "Qian Kemao and Yung-Lyul Lee and Kazuya Hayase and Lau, {Phooi Yee} and Wen-Nung Lie and Lu Yu and Sanun Srisuk",
booktitle = "International Workshop on Advanced Image Technology, IWAIT 2019",

}

TY - GEN

T1 - Manga character clustering with DBSCAN using fine-tuned CNN model

AU - Yanagisawa, Hideaki

AU - Yamashita, Takuro

AU - Watanabe, Hiroshi

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Manga (Japanese comic) is popular content worldwide. In Japan, e-comic accounts for about 80% of e-book market. In recent years, metadata extraction from manga image has been studied for providing e-comic service. Manga character is one of the important contents for story understanding. In conventional research, some character identification methods are proposed those classify characters' face images using k-means clustering. However, there are two problems. First, kmeans method needs to specify the number of clusters, however the number of characters in target manga images is commonly unknown. Second, manga includes characters with few appearing, so it is difficult to classify characters with high purity. To solve these problems, we propose clustering method using DBSCAN which decides number of clusters automatically and is robust to noise data. In our prior research, it is experimented that character face clustering using DBSCAN and general CNN features. However, general CNN model is difficult to capture detailed features of manga characters. In this paper, we apply DBSCAN to fine-tuned CNN with manga character faces to improve the clustering accuracy. We also compare the optimal parameter determination method of DBSCAN. Experimental results showed that the dimensional reduction using Kernel PCA and UMAP is effective. In addition, we confirmed the validity of proposed method that determining the parameters of DBSCAN based on the slope changing of k-distance graph.

AB - Manga (Japanese comic) is popular content worldwide. In Japan, e-comic accounts for about 80% of e-book market. In recent years, metadata extraction from manga image has been studied for providing e-comic service. Manga character is one of the important contents for story understanding. In conventional research, some character identification methods are proposed those classify characters' face images using k-means clustering. However, there are two problems. First, kmeans method needs to specify the number of clusters, however the number of characters in target manga images is commonly unknown. Second, manga includes characters with few appearing, so it is difficult to classify characters with high purity. To solve these problems, we propose clustering method using DBSCAN which decides number of clusters automatically and is robust to noise data. In our prior research, it is experimented that character face clustering using DBSCAN and general CNN features. However, general CNN model is difficult to capture detailed features of manga characters. In this paper, we apply DBSCAN to fine-tuned CNN with manga character faces to improve the clustering accuracy. We also compare the optimal parameter determination method of DBSCAN. Experimental results showed that the dimensional reduction using Kernel PCA and UMAP is effective. In addition, we confirmed the validity of proposed method that determining the parameters of DBSCAN based on the slope changing of k-distance graph.

KW - clustering

KW - CNN

KW - DBSCAN

KW - manga

UR - http://www.scopus.com/inward/record.url?scp=85063877509&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85063877509&partnerID=8YFLogxK

U2 - 10.1117/12.2521116

DO - 10.1117/12.2521116

M3 - Conference contribution

AN - SCOPUS:85063877509

T3 - Proceedings of SPIE - The International Society for Optical Engineering

BT - International Workshop on Advanced Image Technology, IWAIT 2019

A2 - Kemao, Qian

A2 - Lee, Yung-Lyul

A2 - Hayase, Kazuya

A2 - Lau, Phooi Yee

A2 - Lie, Wen-Nung

A2 - Yu, Lu

A2 - Srisuk, Sanun

PB - SPIE

ER -