Low-dimensional superpixel descriptor and its application in visual correspondence estimation

Songlin Du, Takeshi Ikenaga

Research output: Contribution to journalArticle

Abstract

Establishing local visual correspondence between video frames is an important and challenging problem in many vision based applications. Local keypoint detection and description based pixel-level matching is a typical way for visual correspondence estimation. Unlike traditional local keypoint descriptor based methods, this paper proposes a comprehensive yet low-dimensional local feature descriptor based on superpixels generated by over segmentation. The proposed local feature descriptor extracts shape feature, texture feature, and color feature from superpixels by orientated center-boundary distance (OCBD), gray-level co-occurrence matrix (GLCM), and saturation histogram (SHIST), respectively. The types of features are more comprehensive than existing descriptors which extract only one specific kind of feature. Experimental results on the widely used Middlebury optical flow dataset prove that the proposed superpixel descriptor achieves triple accuracy compared with the state-of-the-art ORB descriptor which has the same dimension of features with the proposed one. In addition, since the dimension of the proposed superpixel descriptor is low, it is convenient for matching and memory-efficient for hardware implementation.

Original languageEnglish
JournalMultimedia Tools and Applications
DOIs
Publication statusPublished - 2019 Jan 1

Fingerprint

Optical flows
Computer hardware
Textures
Pixels
Color
Data storage equipment

Keywords

  • Low-dimensional feature
  • Superpixel descriptor
  • Visual correspondence estimation

ASJC Scopus subject areas

  • Software
  • Media Technology
  • Hardware and Architecture
  • Computer Networks and Communications

Cite this

@article{e4558f91dc2b4c90886ea5b821385281,
title = "Low-dimensional superpixel descriptor and its application in visual correspondence estimation",
abstract = "Establishing local visual correspondence between video frames is an important and challenging problem in many vision based applications. Local keypoint detection and description based pixel-level matching is a typical way for visual correspondence estimation. Unlike traditional local keypoint descriptor based methods, this paper proposes a comprehensive yet low-dimensional local feature descriptor based on superpixels generated by over segmentation. The proposed local feature descriptor extracts shape feature, texture feature, and color feature from superpixels by orientated center-boundary distance (OCBD), gray-level co-occurrence matrix (GLCM), and saturation histogram (SHIST), respectively. The types of features are more comprehensive than existing descriptors which extract only one specific kind of feature. Experimental results on the widely used Middlebury optical flow dataset prove that the proposed superpixel descriptor achieves triple accuracy compared with the state-of-the-art ORB descriptor which has the same dimension of features with the proposed one. In addition, since the dimension of the proposed superpixel descriptor is low, it is convenient for matching and memory-efficient for hardware implementation.",
keywords = "Low-dimensional feature, Superpixel descriptor, Visual correspondence estimation",
author = "Songlin Du and Takeshi Ikenaga",
year = "2019",
month = "1",
day = "1",
doi = "10.1007/s11042-019-7248-6",
language = "English",
journal = "Multimedia Tools and Applications",
issn = "1380-7501",
publisher = "Springer Netherlands",

}

TY - JOUR

T1 - Low-dimensional superpixel descriptor and its application in visual correspondence estimation

AU - Du, Songlin

AU - Ikenaga, Takeshi

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Establishing local visual correspondence between video frames is an important and challenging problem in many vision based applications. Local keypoint detection and description based pixel-level matching is a typical way for visual correspondence estimation. Unlike traditional local keypoint descriptor based methods, this paper proposes a comprehensive yet low-dimensional local feature descriptor based on superpixels generated by over segmentation. The proposed local feature descriptor extracts shape feature, texture feature, and color feature from superpixels by orientated center-boundary distance (OCBD), gray-level co-occurrence matrix (GLCM), and saturation histogram (SHIST), respectively. The types of features are more comprehensive than existing descriptors which extract only one specific kind of feature. Experimental results on the widely used Middlebury optical flow dataset prove that the proposed superpixel descriptor achieves triple accuracy compared with the state-of-the-art ORB descriptor which has the same dimension of features with the proposed one. In addition, since the dimension of the proposed superpixel descriptor is low, it is convenient for matching and memory-efficient for hardware implementation.

AB - Establishing local visual correspondence between video frames is an important and challenging problem in many vision based applications. Local keypoint detection and description based pixel-level matching is a typical way for visual correspondence estimation. Unlike traditional local keypoint descriptor based methods, this paper proposes a comprehensive yet low-dimensional local feature descriptor based on superpixels generated by over segmentation. The proposed local feature descriptor extracts shape feature, texture feature, and color feature from superpixels by orientated center-boundary distance (OCBD), gray-level co-occurrence matrix (GLCM), and saturation histogram (SHIST), respectively. The types of features are more comprehensive than existing descriptors which extract only one specific kind of feature. Experimental results on the widely used Middlebury optical flow dataset prove that the proposed superpixel descriptor achieves triple accuracy compared with the state-of-the-art ORB descriptor which has the same dimension of features with the proposed one. In addition, since the dimension of the proposed superpixel descriptor is low, it is convenient for matching and memory-efficient for hardware implementation.

KW - Low-dimensional feature

KW - Superpixel descriptor

KW - Visual correspondence estimation

UR - http://www.scopus.com/inward/record.url?scp=85061656862&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85061656862&partnerID=8YFLogxK

U2 - 10.1007/s11042-019-7248-6

DO - 10.1007/s11042-019-7248-6

M3 - Article

AN - SCOPUS:85061656862

JO - Multimedia Tools and Applications

JF - Multimedia Tools and Applications

SN - 1380-7501

ER -