Spectral normalization and relativistic adversarial training for conditional pose generation with self-attention

Yusuke Horiuchi, Satoshi Iizuka, Edgar Simo Serra, Hiroshi Ishikawa

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We address the problem of conditional image generation of synthesizing a new image of an individual given a reference image and target pose. We base our approach on generative adversarial networks and leverage deformable skip connections to deal with pixel-to-pixel misalignments, self-attention to leverage complementary features in separate portions of the image, e.g., arms or legs, and spectral normalization to improve the quality of the synthesized images. We train the synthesis model with a nearest-neighbour loss in combination with a relativistic average hinge adversarial loss. We evaluate on the Market-1501 dataset and show how our proposed approach can surpass existing approaches in conditional image synthesis performance.

Original languageEnglish
Title of host publicationProceedings of the 16th International Conference on Machine Vision Applications, MVA 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9784901122184
DOIs
Publication statusPublished - 2019 May 1
Event16th International Conference on Machine Vision Applications, MVA 2019 - Tokyo, Japan
Duration: 2019 May 272019 May 31

Publication series

NameProceedings of the 16th International Conference on Machine Vision Applications, MVA 2019

Conference

Conference16th International Conference on Machine Vision Applications, MVA 2019
CountryJapan
CityTokyo
Period19/5/2719/5/31

Fingerprint

Pixels
Hinges

ASJC Scopus subject areas

  • Computer Science Applications
  • Signal Processing
  • Computer Vision and Pattern Recognition

Cite this

Horiuchi, Y., Iizuka, S., Simo Serra, E., & Ishikawa, H. (2019). Spectral normalization and relativistic adversarial training for conditional pose generation with self-attention. In Proceedings of the 16th International Conference on Machine Vision Applications, MVA 2019 [8758013] (Proceedings of the 16th International Conference on Machine Vision Applications, MVA 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.23919/MVA.2019.8758013

Spectral normalization and relativistic adversarial training for conditional pose generation with self-attention. / Horiuchi, Yusuke; Iizuka, Satoshi; Simo Serra, Edgar; Ishikawa, Hiroshi.

Proceedings of the 16th International Conference on Machine Vision Applications, MVA 2019. Institute of Electrical and Electronics Engineers Inc., 2019. 8758013 (Proceedings of the 16th International Conference on Machine Vision Applications, MVA 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Horiuchi, Y, Iizuka, S, Simo Serra, E & Ishikawa, H 2019, Spectral normalization and relativistic adversarial training for conditional pose generation with self-attention. in Proceedings of the 16th International Conference on Machine Vision Applications, MVA 2019., 8758013, Proceedings of the 16th International Conference on Machine Vision Applications, MVA 2019, Institute of Electrical and Electronics Engineers Inc., 16th International Conference on Machine Vision Applications, MVA 2019, Tokyo, Japan, 19/5/27. https://doi.org/10.23919/MVA.2019.8758013
Horiuchi Y, Iizuka S, Simo Serra E, Ishikawa H. Spectral normalization and relativistic adversarial training for conditional pose generation with self-attention. In Proceedings of the 16th International Conference on Machine Vision Applications, MVA 2019. Institute of Electrical and Electronics Engineers Inc. 2019. 8758013. (Proceedings of the 16th International Conference on Machine Vision Applications, MVA 2019). https://doi.org/10.23919/MVA.2019.8758013
Horiuchi, Yusuke ; Iizuka, Satoshi ; Simo Serra, Edgar ; Ishikawa, Hiroshi. / Spectral normalization and relativistic adversarial training for conditional pose generation with self-attention. Proceedings of the 16th International Conference on Machine Vision Applications, MVA 2019. Institute of Electrical and Electronics Engineers Inc., 2019. (Proceedings of the 16th International Conference on Machine Vision Applications, MVA 2019).
@inproceedings{56a062e751474f508771ebb29278df10,
title = "Spectral normalization and relativistic adversarial training for conditional pose generation with self-attention",
abstract = "We address the problem of conditional image generation of synthesizing a new image of an individual given a reference image and target pose. We base our approach on generative adversarial networks and leverage deformable skip connections to deal with pixel-to-pixel misalignments, self-attention to leverage complementary features in separate portions of the image, e.g., arms or legs, and spectral normalization to improve the quality of the synthesized images. We train the synthesis model with a nearest-neighbour loss in combination with a relativistic average hinge adversarial loss. We evaluate on the Market-1501 dataset and show how our proposed approach can surpass existing approaches in conditional image synthesis performance.",
author = "Yusuke Horiuchi and Satoshi Iizuka and {Simo Serra}, Edgar and Hiroshi Ishikawa",
year = "2019",
month = "5",
day = "1",
doi = "10.23919/MVA.2019.8758013",
language = "English",
series = "Proceedings of the 16th International Conference on Machine Vision Applications, MVA 2019",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
booktitle = "Proceedings of the 16th International Conference on Machine Vision Applications, MVA 2019",

}

TY - GEN

T1 - Spectral normalization and relativistic adversarial training for conditional pose generation with self-attention

AU - Horiuchi, Yusuke

AU - Iizuka, Satoshi

AU - Simo Serra, Edgar

AU - Ishikawa, Hiroshi

PY - 2019/5/1

Y1 - 2019/5/1

N2 - We address the problem of conditional image generation of synthesizing a new image of an individual given a reference image and target pose. We base our approach on generative adversarial networks and leverage deformable skip connections to deal with pixel-to-pixel misalignments, self-attention to leverage complementary features in separate portions of the image, e.g., arms or legs, and spectral normalization to improve the quality of the synthesized images. We train the synthesis model with a nearest-neighbour loss in combination with a relativistic average hinge adversarial loss. We evaluate on the Market-1501 dataset and show how our proposed approach can surpass existing approaches in conditional image synthesis performance.

AB - We address the problem of conditional image generation of synthesizing a new image of an individual given a reference image and target pose. We base our approach on generative adversarial networks and leverage deformable skip connections to deal with pixel-to-pixel misalignments, self-attention to leverage complementary features in separate portions of the image, e.g., arms or legs, and spectral normalization to improve the quality of the synthesized images. We train the synthesis model with a nearest-neighbour loss in combination with a relativistic average hinge adversarial loss. We evaluate on the Market-1501 dataset and show how our proposed approach can surpass existing approaches in conditional image synthesis performance.

UR - http://www.scopus.com/inward/record.url?scp=85070455014&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85070455014&partnerID=8YFLogxK

U2 - 10.23919/MVA.2019.8758013

DO - 10.23919/MVA.2019.8758013

M3 - Conference contribution

AN - SCOPUS:85070455014

T3 - Proceedings of the 16th International Conference on Machine Vision Applications, MVA 2019

BT - Proceedings of the 16th International Conference on Machine Vision Applications, MVA 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -