Real-time data-driven interactive rough sketch inking

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

We present an interactive approach for inking, which is the process of turning a pencil rough sketch into a clean line drawing. The approach, which we call the Smart Inker, consists of several "smart" tools that intuitively react to user input, while guided by the input rough sketch, to efficiently and naturally connect lines, erase shading, and fine-tune the line drawing output. Our approach is data-driven: The tools are based on fully convolutional networks, which we train to exploit both the user edits and inaccurate rough sketch to produce accurate line drawings, allowing high-performance interactive editing in real-time on a variety of challenging rough sketch images. For the training of the tools, we developed two key techniques: One is the creation of training data by simulation of vague and quick user edits; the other is a line normalization based on learning from vector data. These techniques, in combination with our sketch-specific data augmentation, allow us to train the tools on heterogeneous data without actual user interaction. We validate our approach with an in-depth user study, comparing it with professional illustration software, and show that our approach is able to reduce inking time by a factor of 1.8×, while improving the results of amateur users.

Original languageEnglish
Article numberA59
JournalACM Transactions on Graphics
Volume37
Issue number4
DOIs
Publication statusPublished - 2018 Jan 1

Keywords

  • Iine drawing
  • Inking
  • Interaction
  • Sketching

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design

Cite this

Real-time data-driven interactive rough sketch inking. / Simo Serra, Edgar; Iizuka, Satoshi; Ishikawa, Hiroshi.

In: ACM Transactions on Graphics, Vol. 37, No. 4, A59, 01.01.2018.

Research output: Contribution to journalArticle

@article{73b48933222b43b7b77d11d4d83d2b2a,
title = "Real-time data-driven interactive rough sketch inking",
abstract = "We present an interactive approach for inking, which is the process of turning a pencil rough sketch into a clean line drawing. The approach, which we call the Smart Inker, consists of several {"}smart{"} tools that intuitively react to user input, while guided by the input rough sketch, to efficiently and naturally connect lines, erase shading, and fine-tune the line drawing output. Our approach is data-driven: The tools are based on fully convolutional networks, which we train to exploit both the user edits and inaccurate rough sketch to produce accurate line drawings, allowing high-performance interactive editing in real-time on a variety of challenging rough sketch images. For the training of the tools, we developed two key techniques: One is the creation of training data by simulation of vague and quick user edits; the other is a line normalization based on learning from vector data. These techniques, in combination with our sketch-specific data augmentation, allow us to train the tools on heterogeneous data without actual user interaction. We validate our approach with an in-depth user study, comparing it with professional illustration software, and show that our approach is able to reduce inking time by a factor of 1.8×, while improving the results of amateur users.",
keywords = "Iine drawing, Inking, Interaction, Sketching",
author = "{Simo Serra}, Edgar and Satoshi Iizuka and Hiroshi Ishikawa",
year = "2018",
month = "1",
day = "1",
doi = "10.1145/3197517.3201370",
language = "English",
volume = "37",
journal = "ACM Transactions on Graphics",
issn = "0730-0301",
publisher = "Association for Computing Machinery (ACM)",
number = "4",

}

TY - JOUR

T1 - Real-time data-driven interactive rough sketch inking

AU - Simo Serra, Edgar

AU - Iizuka, Satoshi

AU - Ishikawa, Hiroshi

PY - 2018/1/1

Y1 - 2018/1/1

N2 - We present an interactive approach for inking, which is the process of turning a pencil rough sketch into a clean line drawing. The approach, which we call the Smart Inker, consists of several "smart" tools that intuitively react to user input, while guided by the input rough sketch, to efficiently and naturally connect lines, erase shading, and fine-tune the line drawing output. Our approach is data-driven: The tools are based on fully convolutional networks, which we train to exploit both the user edits and inaccurate rough sketch to produce accurate line drawings, allowing high-performance interactive editing in real-time on a variety of challenging rough sketch images. For the training of the tools, we developed two key techniques: One is the creation of training data by simulation of vague and quick user edits; the other is a line normalization based on learning from vector data. These techniques, in combination with our sketch-specific data augmentation, allow us to train the tools on heterogeneous data without actual user interaction. We validate our approach with an in-depth user study, comparing it with professional illustration software, and show that our approach is able to reduce inking time by a factor of 1.8×, while improving the results of amateur users.

AB - We present an interactive approach for inking, which is the process of turning a pencil rough sketch into a clean line drawing. The approach, which we call the Smart Inker, consists of several "smart" tools that intuitively react to user input, while guided by the input rough sketch, to efficiently and naturally connect lines, erase shading, and fine-tune the line drawing output. Our approach is data-driven: The tools are based on fully convolutional networks, which we train to exploit both the user edits and inaccurate rough sketch to produce accurate line drawings, allowing high-performance interactive editing in real-time on a variety of challenging rough sketch images. For the training of the tools, we developed two key techniques: One is the creation of training data by simulation of vague and quick user edits; the other is a line normalization based on learning from vector data. These techniques, in combination with our sketch-specific data augmentation, allow us to train the tools on heterogeneous data without actual user interaction. We validate our approach with an in-depth user study, comparing it with professional illustration software, and show that our approach is able to reduce inking time by a factor of 1.8×, while improving the results of amateur users.

KW - Iine drawing

KW - Inking

KW - Interaction

KW - Sketching

UR - http://www.scopus.com/inward/record.url?scp=85056735494&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85056735494&partnerID=8YFLogxK

U2 - 10.1145/3197517.3201370

DO - 10.1145/3197517.3201370

M3 - Article

VL - 37

JO - ACM Transactions on Graphics

JF - ACM Transactions on Graphics

SN - 0730-0301

IS - 4

M1 - A59

ER -