Learning to restore deteriorated line drawing

Kazuma Sasaki, Satoshi Iizuka, Edgar Simo Serra, Hiroshi Ishikawa

Research output: Contribution to journalArticle

Abstract

We propose a fully automatic approach to restore aged old line drawings. We decompose the task into two subtasks: the line extraction subtask, which aims to extract line fragments and remove the paper texture background, and the restoration subtask, which fills in possible gaps and deterioration of the lines to produce a clean line drawing. Our approach is based on a convolutional neural network that consists of two sub-networks corresponding to the two subtasks. They are trained as part of a single framework in an end-to-end fashion. We also introduce a new dataset consisting of manually annotated sketches by Leonardo da Vinci which, in combination with a synthetic data generation approach, allows training the network to restore deteriorated line drawings. We evaluate our method on challenging 500-year-old sketches and compare with existing approaches with a user study, in which it is found that our approach is preferred 72.7% of the time.

Original languageEnglish
Pages (from-to)1-9
Number of pages9
JournalVisual Computer
DOIs
Publication statusAccepted/In press - 2018 May 3

Fingerprint

Restoration
Deterioration
Textures
Neural networks

Keywords

  • Convolutional neural network
  • Image manipulation
  • Image restoration
  • Line drawings

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Computer Graphics and Computer-Aided Design

Cite this

Learning to restore deteriorated line drawing. / Sasaki, Kazuma; Iizuka, Satoshi; Simo Serra, Edgar; Ishikawa, Hiroshi.

In: Visual Computer, 03.05.2018, p. 1-9.

Research output: Contribution to journalArticle

Sasaki, Kazuma ; Iizuka, Satoshi ; Simo Serra, Edgar ; Ishikawa, Hiroshi. / Learning to restore deteriorated line drawing. In: Visual Computer. 2018 ; pp. 1-9.
@article{fda873f9cb704393b5278c4052f61b46,
title = "Learning to restore deteriorated line drawing",
abstract = "We propose a fully automatic approach to restore aged old line drawings. We decompose the task into two subtasks: the line extraction subtask, which aims to extract line fragments and remove the paper texture background, and the restoration subtask, which fills in possible gaps and deterioration of the lines to produce a clean line drawing. Our approach is based on a convolutional neural network that consists of two sub-networks corresponding to the two subtasks. They are trained as part of a single framework in an end-to-end fashion. We also introduce a new dataset consisting of manually annotated sketches by Leonardo da Vinci which, in combination with a synthetic data generation approach, allows training the network to restore deteriorated line drawings. We evaluate our method on challenging 500-year-old sketches and compare with existing approaches with a user study, in which it is found that our approach is preferred 72.7{\%} of the time.",
keywords = "Convolutional neural network, Image manipulation, Image restoration, Line drawings",
author = "Kazuma Sasaki and Satoshi Iizuka and {Simo Serra}, Edgar and Hiroshi Ishikawa",
year = "2018",
month = "5",
day = "3",
doi = "10.1007/s00371-018-1528-4",
language = "English",
pages = "1--9",
journal = "Visual Computer",
issn = "0178-2789",
publisher = "Springer Verlag",

}

TY - JOUR

T1 - Learning to restore deteriorated line drawing

AU - Sasaki, Kazuma

AU - Iizuka, Satoshi

AU - Simo Serra, Edgar

AU - Ishikawa, Hiroshi

PY - 2018/5/3

Y1 - 2018/5/3

N2 - We propose a fully automatic approach to restore aged old line drawings. We decompose the task into two subtasks: the line extraction subtask, which aims to extract line fragments and remove the paper texture background, and the restoration subtask, which fills in possible gaps and deterioration of the lines to produce a clean line drawing. Our approach is based on a convolutional neural network that consists of two sub-networks corresponding to the two subtasks. They are trained as part of a single framework in an end-to-end fashion. We also introduce a new dataset consisting of manually annotated sketches by Leonardo da Vinci which, in combination with a synthetic data generation approach, allows training the network to restore deteriorated line drawings. We evaluate our method on challenging 500-year-old sketches and compare with existing approaches with a user study, in which it is found that our approach is preferred 72.7% of the time.

AB - We propose a fully automatic approach to restore aged old line drawings. We decompose the task into two subtasks: the line extraction subtask, which aims to extract line fragments and remove the paper texture background, and the restoration subtask, which fills in possible gaps and deterioration of the lines to produce a clean line drawing. Our approach is based on a convolutional neural network that consists of two sub-networks corresponding to the two subtasks. They are trained as part of a single framework in an end-to-end fashion. We also introduce a new dataset consisting of manually annotated sketches by Leonardo da Vinci which, in combination with a synthetic data generation approach, allows training the network to restore deteriorated line drawings. We evaluate our method on challenging 500-year-old sketches and compare with existing approaches with a user study, in which it is found that our approach is preferred 72.7% of the time.

KW - Convolutional neural network

KW - Image manipulation

KW - Image restoration

KW - Line drawings

UR - http://www.scopus.com/inward/record.url?scp=85046467877&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85046467877&partnerID=8YFLogxK

U2 - 10.1007/s00371-018-1528-4

DO - 10.1007/s00371-018-1528-4

M3 - Article

AN - SCOPUS:85046467877

SP - 1

EP - 9

JO - Visual Computer

JF - Visual Computer

SN - 0178-2789

ER -