Learning to simplify

Fully convolutional networks for rough sketch cleanup

Edgar Simo Serra, Satoshi Iizuka, Kazuma Sasaki, Hiroshi Ishikawa

Research output: Contribution to journalArticle

53 Citations (Scopus)

Abstract

In this paper, we present a novel technique to simplify sketch drawings based on learning a series of convolution operators. In contrast to existing approaches that require vector images as input, we allow the more general and challenging input of rough raster sketches such as those obtained from scanning pencil sketches. We convert the rough sketch into a simplified version which is then amendable for vectorization. This is all done in a fully automatic way without user intervention. Our model consists of a fully convolutional neural network which, unlike most existing convolutional neural networks, is able to process images of any dimensions and aspect ratio as input, and outputs a simplified sketch which has the same dimensions as the input image. In order to teach our model to simplify, we present a new dataset of pairs of rough and simplified sketch drawings. By leveraging convolution operators in combination with efficient use of our proposed dataset, we are able to train our sketch simplification model. Our approach naturally overcomes the limitations of existing methods, e.g., vector images as input and long computation time; and we show that meaningful simplifications can be obtained for many different test cases. Finally, we validate our results with a user study in which we greatly outperform similar approaches and establish the state of the art in sketch simplification of raster images.

Original languageEnglish
Article numbera121
JournalACM Transactions on Graphics
Volume35
Issue number4
DOIs
Publication statusPublished - 2016 Jul 11

Fingerprint

Convolution
Mathematical operators
Neural networks
Aspect ratio
Scanning

Keywords

  • Convolutional neural network
  • Sketch simplification

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design

Cite this

Learning to simplify : Fully convolutional networks for rough sketch cleanup. / Simo Serra, Edgar; Iizuka, Satoshi; Sasaki, Kazuma; Ishikawa, Hiroshi.

In: ACM Transactions on Graphics, Vol. 35, No. 4, a121, 11.07.2016.

Research output: Contribution to journalArticle

@article{8faa1268eea64edb8be4d5086a63d679,
title = "Learning to simplify: Fully convolutional networks for rough sketch cleanup",
abstract = "In this paper, we present a novel technique to simplify sketch drawings based on learning a series of convolution operators. In contrast to existing approaches that require vector images as input, we allow the more general and challenging input of rough raster sketches such as those obtained from scanning pencil sketches. We convert the rough sketch into a simplified version which is then amendable for vectorization. This is all done in a fully automatic way without user intervention. Our model consists of a fully convolutional neural network which, unlike most existing convolutional neural networks, is able to process images of any dimensions and aspect ratio as input, and outputs a simplified sketch which has the same dimensions as the input image. In order to teach our model to simplify, we present a new dataset of pairs of rough and simplified sketch drawings. By leveraging convolution operators in combination with efficient use of our proposed dataset, we are able to train our sketch simplification model. Our approach naturally overcomes the limitations of existing methods, e.g., vector images as input and long computation time; and we show that meaningful simplifications can be obtained for many different test cases. Finally, we validate our results with a user study in which we greatly outperform similar approaches and establish the state of the art in sketch simplification of raster images.",
keywords = "Convolutional neural network, Sketch simplification",
author = "{Simo Serra}, Edgar and Satoshi Iizuka and Kazuma Sasaki and Hiroshi Ishikawa",
year = "2016",
month = "7",
day = "11",
doi = "10.1145/2897824.2925972",
language = "English",
volume = "35",
journal = "ACM Transactions on Graphics",
issn = "0730-0301",
publisher = "Association for Computing Machinery (ACM)",
number = "4",

}

TY - JOUR

T1 - Learning to simplify

T2 - Fully convolutional networks for rough sketch cleanup

AU - Simo Serra, Edgar

AU - Iizuka, Satoshi

AU - Sasaki, Kazuma

AU - Ishikawa, Hiroshi

PY - 2016/7/11

Y1 - 2016/7/11

N2 - In this paper, we present a novel technique to simplify sketch drawings based on learning a series of convolution operators. In contrast to existing approaches that require vector images as input, we allow the more general and challenging input of rough raster sketches such as those obtained from scanning pencil sketches. We convert the rough sketch into a simplified version which is then amendable for vectorization. This is all done in a fully automatic way without user intervention. Our model consists of a fully convolutional neural network which, unlike most existing convolutional neural networks, is able to process images of any dimensions and aspect ratio as input, and outputs a simplified sketch which has the same dimensions as the input image. In order to teach our model to simplify, we present a new dataset of pairs of rough and simplified sketch drawings. By leveraging convolution operators in combination with efficient use of our proposed dataset, we are able to train our sketch simplification model. Our approach naturally overcomes the limitations of existing methods, e.g., vector images as input and long computation time; and we show that meaningful simplifications can be obtained for many different test cases. Finally, we validate our results with a user study in which we greatly outperform similar approaches and establish the state of the art in sketch simplification of raster images.

AB - In this paper, we present a novel technique to simplify sketch drawings based on learning a series of convolution operators. In contrast to existing approaches that require vector images as input, we allow the more general and challenging input of rough raster sketches such as those obtained from scanning pencil sketches. We convert the rough sketch into a simplified version which is then amendable for vectorization. This is all done in a fully automatic way without user intervention. Our model consists of a fully convolutional neural network which, unlike most existing convolutional neural networks, is able to process images of any dimensions and aspect ratio as input, and outputs a simplified sketch which has the same dimensions as the input image. In order to teach our model to simplify, we present a new dataset of pairs of rough and simplified sketch drawings. By leveraging convolution operators in combination with efficient use of our proposed dataset, we are able to train our sketch simplification model. Our approach naturally overcomes the limitations of existing methods, e.g., vector images as input and long computation time; and we show that meaningful simplifications can be obtained for many different test cases. Finally, we validate our results with a user study in which we greatly outperform similar approaches and establish the state of the art in sketch simplification of raster images.

KW - Convolutional neural network

KW - Sketch simplification

UR - http://www.scopus.com/inward/record.url?scp=84979964959&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84979964959&partnerID=8YFLogxK

U2 - 10.1145/2897824.2925972

DO - 10.1145/2897824.2925972

M3 - Article

VL - 35

JO - ACM Transactions on Graphics

JF - ACM Transactions on Graphics

SN - 0730-0301

IS - 4

M1 - a121

ER -