Discriminative learning of deep convolutional feature point descriptors

Edgar Simo Serra, Eduard Trulls, Luis Ferraz, Iasonas Kokkinos, Pascal Fua, Francesc Moreno-Noguer

Research output: Chapter in Book/Report/Conference proceedingConference contribution

220 Citations (Scopus)

Abstract

Deep learning has revolutionalized image-level tasks such as classification, but patch-level tasks, such as correspondence, still rely on hand-crafted features, e.g. SIFT. In this paper we use Convolutional Neural Networks (CNNs) to learn discriminant patch representations and in particular train a Siamese network with pairs of (non-)corresponding patches. We deal with the large number of potential pairs with the combination of a stochastic sampling of the training set and an aggressive mining strategy biased towards patches that are hard to classify. By using the L2 distance during both training and testing we develop 128-D descriptors whose euclidean distances reflect patch similarity, and which can be used as a drop-in replacement for any task involving SIFT. We demonstrate consistent performance gains over the state of the art, and generalize well against scaling and rotation, perspective transformation, non-rigid deformation, and illumination changes. Our descriptors are efficient to compute and amenable to modern GPUs, and are publicly available.

Original languageEnglish
Title of host publicationProceedings - 2015 IEEE International Conference on Computer Vision, ICCV 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages118-126
Number of pages9
Volume11-18-December-2015
ISBN (Electronic)9781467383912
DOIs
Publication statusPublished - 2016 Feb 17
Event15th IEEE International Conference on Computer Vision, ICCV 2015 - Santiago, Chile
Duration: 2015 Dec 112015 Dec 18

Other

Other15th IEEE International Conference on Computer Vision, ICCV 2015
CountryChile
CitySantiago
Period15/12/1115/12/18

Fingerprint

Lighting
Sampling
Neural networks
Testing
Graphics processing unit
Deep learning

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition

Cite this

Simo Serra, E., Trulls, E., Ferraz, L., Kokkinos, I., Fua, P., & Moreno-Noguer, F. (2016). Discriminative learning of deep convolutional feature point descriptors. In Proceedings - 2015 IEEE International Conference on Computer Vision, ICCV 2015 (Vol. 11-18-December-2015, pp. 118-126). [7410379] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCV.2015.22

Discriminative learning of deep convolutional feature point descriptors. / Simo Serra, Edgar; Trulls, Eduard; Ferraz, Luis; Kokkinos, Iasonas; Fua, Pascal; Moreno-Noguer, Francesc.

Proceedings - 2015 IEEE International Conference on Computer Vision, ICCV 2015. Vol. 11-18-December-2015 Institute of Electrical and Electronics Engineers Inc., 2016. p. 118-126 7410379.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Simo Serra, E, Trulls, E, Ferraz, L, Kokkinos, I, Fua, P & Moreno-Noguer, F 2016, Discriminative learning of deep convolutional feature point descriptors. in Proceedings - 2015 IEEE International Conference on Computer Vision, ICCV 2015. vol. 11-18-December-2015, 7410379, Institute of Electrical and Electronics Engineers Inc., pp. 118-126, 15th IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile, 15/12/11. https://doi.org/10.1109/ICCV.2015.22
Simo Serra E, Trulls E, Ferraz L, Kokkinos I, Fua P, Moreno-Noguer F. Discriminative learning of deep convolutional feature point descriptors. In Proceedings - 2015 IEEE International Conference on Computer Vision, ICCV 2015. Vol. 11-18-December-2015. Institute of Electrical and Electronics Engineers Inc. 2016. p. 118-126. 7410379 https://doi.org/10.1109/ICCV.2015.22
Simo Serra, Edgar ; Trulls, Eduard ; Ferraz, Luis ; Kokkinos, Iasonas ; Fua, Pascal ; Moreno-Noguer, Francesc. / Discriminative learning of deep convolutional feature point descriptors. Proceedings - 2015 IEEE International Conference on Computer Vision, ICCV 2015. Vol. 11-18-December-2015 Institute of Electrical and Electronics Engineers Inc., 2016. pp. 118-126
@inproceedings{3daa1e73662e4849a47888fb13e7b5ab,
title = "Discriminative learning of deep convolutional feature point descriptors",
abstract = "Deep learning has revolutionalized image-level tasks such as classification, but patch-level tasks, such as correspondence, still rely on hand-crafted features, e.g. SIFT. In this paper we use Convolutional Neural Networks (CNNs) to learn discriminant patch representations and in particular train a Siamese network with pairs of (non-)corresponding patches. We deal with the large number of potential pairs with the combination of a stochastic sampling of the training set and an aggressive mining strategy biased towards patches that are hard to classify. By using the L2 distance during both training and testing we develop 128-D descriptors whose euclidean distances reflect patch similarity, and which can be used as a drop-in replacement for any task involving SIFT. We demonstrate consistent performance gains over the state of the art, and generalize well against scaling and rotation, perspective transformation, non-rigid deformation, and illumination changes. Our descriptors are efficient to compute and amenable to modern GPUs, and are publicly available.",
author = "{Simo Serra}, Edgar and Eduard Trulls and Luis Ferraz and Iasonas Kokkinos and Pascal Fua and Francesc Moreno-Noguer",
year = "2016",
month = "2",
day = "17",
doi = "10.1109/ICCV.2015.22",
language = "English",
volume = "11-18-December-2015",
pages = "118--126",
booktitle = "Proceedings - 2015 IEEE International Conference on Computer Vision, ICCV 2015",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
address = "United States",

}

TY - GEN

T1 - Discriminative learning of deep convolutional feature point descriptors

AU - Simo Serra, Edgar

AU - Trulls, Eduard

AU - Ferraz, Luis

AU - Kokkinos, Iasonas

AU - Fua, Pascal

AU - Moreno-Noguer, Francesc

PY - 2016/2/17

Y1 - 2016/2/17

N2 - Deep learning has revolutionalized image-level tasks such as classification, but patch-level tasks, such as correspondence, still rely on hand-crafted features, e.g. SIFT. In this paper we use Convolutional Neural Networks (CNNs) to learn discriminant patch representations and in particular train a Siamese network with pairs of (non-)corresponding patches. We deal with the large number of potential pairs with the combination of a stochastic sampling of the training set and an aggressive mining strategy biased towards patches that are hard to classify. By using the L2 distance during both training and testing we develop 128-D descriptors whose euclidean distances reflect patch similarity, and which can be used as a drop-in replacement for any task involving SIFT. We demonstrate consistent performance gains over the state of the art, and generalize well against scaling and rotation, perspective transformation, non-rigid deformation, and illumination changes. Our descriptors are efficient to compute and amenable to modern GPUs, and are publicly available.

AB - Deep learning has revolutionalized image-level tasks such as classification, but patch-level tasks, such as correspondence, still rely on hand-crafted features, e.g. SIFT. In this paper we use Convolutional Neural Networks (CNNs) to learn discriminant patch representations and in particular train a Siamese network with pairs of (non-)corresponding patches. We deal with the large number of potential pairs with the combination of a stochastic sampling of the training set and an aggressive mining strategy biased towards patches that are hard to classify. By using the L2 distance during both training and testing we develop 128-D descriptors whose euclidean distances reflect patch similarity, and which can be used as a drop-in replacement for any task involving SIFT. We demonstrate consistent performance gains over the state of the art, and generalize well against scaling and rotation, perspective transformation, non-rigid deformation, and illumination changes. Our descriptors are efficient to compute and amenable to modern GPUs, and are publicly available.

UR - http://www.scopus.com/inward/record.url?scp=84973915418&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84973915418&partnerID=8YFLogxK

U2 - 10.1109/ICCV.2015.22

DO - 10.1109/ICCV.2015.22

M3 - Conference contribution

VL - 11-18-December-2015

SP - 118

EP - 126

BT - Proceedings - 2015 IEEE International Conference on Computer Vision, ICCV 2015

PB - Institute of Electrical and Electronics Engineers Inc.

ER -