Adaptive energy selection for content-Aware image resizing

Kazuma Sasaki, Yuya Nagahama, Zheng Ze, Satoshi Iizuka, Edgar Simo Serra, Yoshihiko Mochizuki, Hiroshi Ishikawa

研究成果: Conference contribution

抄録

Content-Aware image resizing aims to reduce the size of an image without touching important objects and regions. In seam carving, this is done by assessing the importance of each pixel by an energy function and repeatedly removing a string of pixels avoiding pixels with high energy. However, there is no single energy function that is best for all images: The optimal energy function is itself a function of the image. In this paper, we present a method for predicting the quality of the results of resizing an image with different energy functions, so as to select the energy best suited for that particular image. We formulate the selection as a classification problem; i.e., we 'classify' the input into the class of images for which one of the energies works best. The standard approach would be to use a CNN for the classification. However, the existence of a fully connected layer forces us to resize the input to a fixed size, which obliterates useful information, especially lower-level features that more closely relate to the energies used for seam carving. Instead, we extract a feature from internal convolutional layers, which results in a fixed-length vector regardless of the input size, making it amenable to classification with a Support Vector Machine. This formulation of the algorithm selection as a classification problem can be used whenever there are multiple approaches for a specific image processing task. We validate our approach with a user study, where our method outperforms recent seam carving approaches.

元の言語English
ホスト出版物のタイトルProceedings - 4th Asian Conference on Pattern Recognition, ACPR 2017
出版者Institute of Electrical and Electronics Engineers Inc.
ページ858-863
ページ数6
ISBN(電子版)9781538633540
DOI
出版物ステータスPublished - 2018 12 13
イベント4th Asian Conference on Pattern Recognition, ACPR 2017 - Nanjing, China
継続期間: 2017 11 262017 11 29

Other

Other4th Asian Conference on Pattern Recognition, ACPR 2017
China
Nanjing
期間17/11/2617/11/29

Fingerprint

Pixels
Support vector machines
Image processing

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Vision and Pattern Recognition
  • Signal Processing

これを引用

Sasaki, K., Nagahama, Y., Ze, Z., Iizuka, S., Simo Serra, E., Mochizuki, Y., & Ishikawa, H. (2018). Adaptive energy selection for content-Aware image resizing. : Proceedings - 4th Asian Conference on Pattern Recognition, ACPR 2017 (pp. 858-863). [8575934] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ACPR.2017.105

Adaptive energy selection for content-Aware image resizing. / Sasaki, Kazuma; Nagahama, Yuya; Ze, Zheng; Iizuka, Satoshi; Simo Serra, Edgar; Mochizuki, Yoshihiko; Ishikawa, Hiroshi.

Proceedings - 4th Asian Conference on Pattern Recognition, ACPR 2017. Institute of Electrical and Electronics Engineers Inc., 2018. p. 858-863 8575934.

研究成果: Conference contribution

Sasaki, K, Nagahama, Y, Ze, Z, Iizuka, S, Simo Serra, E, Mochizuki, Y & Ishikawa, H 2018, Adaptive energy selection for content-Aware image resizing. : Proceedings - 4th Asian Conference on Pattern Recognition, ACPR 2017., 8575934, Institute of Electrical and Electronics Engineers Inc., pp. 858-863, 4th Asian Conference on Pattern Recognition, ACPR 2017, Nanjing, China, 17/11/26. https://doi.org/10.1109/ACPR.2017.105
Sasaki K, Nagahama Y, Ze Z, Iizuka S, Simo Serra E, Mochizuki Y その他. Adaptive energy selection for content-Aware image resizing. : Proceedings - 4th Asian Conference on Pattern Recognition, ACPR 2017. Institute of Electrical and Electronics Engineers Inc. 2018. p. 858-863. 8575934 https://doi.org/10.1109/ACPR.2017.105
Sasaki, Kazuma ; Nagahama, Yuya ; Ze, Zheng ; Iizuka, Satoshi ; Simo Serra, Edgar ; Mochizuki, Yoshihiko ; Ishikawa, Hiroshi. / Adaptive energy selection for content-Aware image resizing. Proceedings - 4th Asian Conference on Pattern Recognition, ACPR 2017. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 858-863
@inproceedings{84947150733d403bbf9b47b50d7ea632,
title = "Adaptive energy selection for content-Aware image resizing",
abstract = "Content-Aware image resizing aims to reduce the size of an image without touching important objects and regions. In seam carving, this is done by assessing the importance of each pixel by an energy function and repeatedly removing a string of pixels avoiding pixels with high energy. However, there is no single energy function that is best for all images: The optimal energy function is itself a function of the image. In this paper, we present a method for predicting the quality of the results of resizing an image with different energy functions, so as to select the energy best suited for that particular image. We formulate the selection as a classification problem; i.e., we 'classify' the input into the class of images for which one of the energies works best. The standard approach would be to use a CNN for the classification. However, the existence of a fully connected layer forces us to resize the input to a fixed size, which obliterates useful information, especially lower-level features that more closely relate to the energies used for seam carving. Instead, we extract a feature from internal convolutional layers, which results in a fixed-length vector regardless of the input size, making it amenable to classification with a Support Vector Machine. This formulation of the algorithm selection as a classification problem can be used whenever there are multiple approaches for a specific image processing task. We validate our approach with a user study, where our method outperforms recent seam carving approaches.",
keywords = "Content-Aware image resizing Seam carving Convolutional Neural Network",
author = "Kazuma Sasaki and Yuya Nagahama and Zheng Ze and Satoshi Iizuka and {Simo Serra}, Edgar and Yoshihiko Mochizuki and Hiroshi Ishikawa",
year = "2018",
month = "12",
day = "13",
doi = "10.1109/ACPR.2017.105",
language = "English",
pages = "858--863",
booktitle = "Proceedings - 4th Asian Conference on Pattern Recognition, ACPR 2017",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Adaptive energy selection for content-Aware image resizing

AU - Sasaki, Kazuma

AU - Nagahama, Yuya

AU - Ze, Zheng

AU - Iizuka, Satoshi

AU - Simo Serra, Edgar

AU - Mochizuki, Yoshihiko

AU - Ishikawa, Hiroshi

PY - 2018/12/13

Y1 - 2018/12/13

N2 - Content-Aware image resizing aims to reduce the size of an image without touching important objects and regions. In seam carving, this is done by assessing the importance of each pixel by an energy function and repeatedly removing a string of pixels avoiding pixels with high energy. However, there is no single energy function that is best for all images: The optimal energy function is itself a function of the image. In this paper, we present a method for predicting the quality of the results of resizing an image with different energy functions, so as to select the energy best suited for that particular image. We formulate the selection as a classification problem; i.e., we 'classify' the input into the class of images for which one of the energies works best. The standard approach would be to use a CNN for the classification. However, the existence of a fully connected layer forces us to resize the input to a fixed size, which obliterates useful information, especially lower-level features that more closely relate to the energies used for seam carving. Instead, we extract a feature from internal convolutional layers, which results in a fixed-length vector regardless of the input size, making it amenable to classification with a Support Vector Machine. This formulation of the algorithm selection as a classification problem can be used whenever there are multiple approaches for a specific image processing task. We validate our approach with a user study, where our method outperforms recent seam carving approaches.

AB - Content-Aware image resizing aims to reduce the size of an image without touching important objects and regions. In seam carving, this is done by assessing the importance of each pixel by an energy function and repeatedly removing a string of pixels avoiding pixels with high energy. However, there is no single energy function that is best for all images: The optimal energy function is itself a function of the image. In this paper, we present a method for predicting the quality of the results of resizing an image with different energy functions, so as to select the energy best suited for that particular image. We formulate the selection as a classification problem; i.e., we 'classify' the input into the class of images for which one of the energies works best. The standard approach would be to use a CNN for the classification. However, the existence of a fully connected layer forces us to resize the input to a fixed size, which obliterates useful information, especially lower-level features that more closely relate to the energies used for seam carving. Instead, we extract a feature from internal convolutional layers, which results in a fixed-length vector regardless of the input size, making it amenable to classification with a Support Vector Machine. This formulation of the algorithm selection as a classification problem can be used whenever there are multiple approaches for a specific image processing task. We validate our approach with a user study, where our method outperforms recent seam carving approaches.

KW - Content-Aware image resizing Seam carving Convolutional Neural Network

UR - http://www.scopus.com/inward/record.url?scp=85060539192&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85060539192&partnerID=8YFLogxK

U2 - 10.1109/ACPR.2017.105

DO - 10.1109/ACPR.2017.105

M3 - Conference contribution

AN - SCOPUS:85060539192

SP - 858

EP - 863

BT - Proceedings - 4th Asian Conference on Pattern Recognition, ACPR 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -