Globally and locally consistent image completion

Research output: Contribution to journalConference articlepeer-review

492 Citations (Scopus)

Abstract

We present a novel approach for image completion that results in images that are both locally and globally consistent. With a fully-convolutional neural network, we can complete images of arbitrary resolutions by filling-in missing regions of any shape. To train this image completion network to be consistent, we use global and local context discriminators that are trained to distinguish real images from completed ones. The global discriminator looks at the entire image to assess if it is coherent as a whole, while the local discriminator looks only at a small area centered at the completed region to ensure the local consistency of the generated patches. The image completion network is then trained to fool the both context discriminator networks, which requires it to generate images that are indistinguishable from real ones with regard to overall consistency as well as in details. We show that our approach can be used to complete a wide variety of scenes. Furthermore, in contrast with the patch-based approaches such as PatchMatch, our approach can generate fragments that do not appear elsewhere in the image, which allows us to naturally complete the images of objects with familiar and highly specific structures, such as faces.

Original languageEnglish
Article number107
JournalACM Transactions on Graphics
Volume36
Issue number4
DOIs
Publication statusPublished - 2017 Jan 1
EventACM SIGGRAPH 2017 - Los Angeles, United States
Duration: 2017 Jul 302017 Aug 3

Keywords

  • Convolutional neural network
  • Image completion

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design

Fingerprint Dive into the research topics of 'Globally and locally consistent image completion'. Together they form a unique fingerprint.

Cite this