Interactive face retrieval framework for clarifying user’s visual memory

Yugo Sato, Tsukasa Fukusato, Shigeo Morishima

Research output: Contribution to journalArticle

Abstract

This paper presents an interactive face retrieval framework for clarifying an image representation envisioned by a user. Our system is designed for a situation in which the user wishes to find a person but has only visual memory of the person. We address a critical challenge of image retrieval across the user’s inputs. Instead of target-specific information, the user can select several images that are similar to an impression of the target person the user wishes to search for. Based on the user’s selection, our proposed system automatically updates a deep convolutional neural network. By interactively repeating these process, the system can reduce the gap between human-based similarities and computer-based similarities and estimate the target image representation. We ran user studies with 10 participants on a public database and confirmed that the proposed framework is effective for clarifying the image representation envisioned by the user easily and quickly.

Original languageEnglish
Pages (from-to)68-79
Number of pages12
JournalITE Transactions on Media Technology and Applications
Volume7
Issue number2
DOIs
Publication statusPublished - 2019 Jan 1

Fingerprint

Image retrieval
Neural networks
Data storage equipment

Keywords

  • Active learning
  • Deep convolutional neural network
  • Face retrieval
  • Relevance feedback
  • User interaction

ASJC Scopus subject areas

  • Signal Processing
  • Media Technology
  • Computer Graphics and Computer-Aided Design

Cite this

Interactive face retrieval framework for clarifying user’s visual memory. / Sato, Yugo; Fukusato, Tsukasa; Morishima, Shigeo.

In: ITE Transactions on Media Technology and Applications, Vol. 7, No. 2, 01.01.2019, p. 68-79.

Research output: Contribution to journalArticle

@article{404cda2a0abd48ac8a1107be0ac00d6a,
title = "Interactive face retrieval framework for clarifying user’s visual memory",
abstract = "This paper presents an interactive face retrieval framework for clarifying an image representation envisioned by a user. Our system is designed for a situation in which the user wishes to find a person but has only visual memory of the person. We address a critical challenge of image retrieval across the user’s inputs. Instead of target-specific information, the user can select several images that are similar to an impression of the target person the user wishes to search for. Based on the user’s selection, our proposed system automatically updates a deep convolutional neural network. By interactively repeating these process, the system can reduce the gap between human-based similarities and computer-based similarities and estimate the target image representation. We ran user studies with 10 participants on a public database and confirmed that the proposed framework is effective for clarifying the image representation envisioned by the user easily and quickly.",
keywords = "Active learning, Deep convolutional neural network, Face retrieval, Relevance feedback, User interaction",
author = "Yugo Sato and Tsukasa Fukusato and Shigeo Morishima",
year = "2019",
month = "1",
day = "1",
doi = "10.3169/mta.7.68",
language = "English",
volume = "7",
pages = "68--79",
journal = "ITE Transactions on Media Technology and Applications",
issn = "2186-7364",
publisher = "Institute of Image Information and Television Engineers",
number = "2",

}

TY - JOUR

T1 - Interactive face retrieval framework for clarifying user’s visual memory

AU - Sato, Yugo

AU - Fukusato, Tsukasa

AU - Morishima, Shigeo

PY - 2019/1/1

Y1 - 2019/1/1

N2 - This paper presents an interactive face retrieval framework for clarifying an image representation envisioned by a user. Our system is designed for a situation in which the user wishes to find a person but has only visual memory of the person. We address a critical challenge of image retrieval across the user’s inputs. Instead of target-specific information, the user can select several images that are similar to an impression of the target person the user wishes to search for. Based on the user’s selection, our proposed system automatically updates a deep convolutional neural network. By interactively repeating these process, the system can reduce the gap between human-based similarities and computer-based similarities and estimate the target image representation. We ran user studies with 10 participants on a public database and confirmed that the proposed framework is effective for clarifying the image representation envisioned by the user easily and quickly.

AB - This paper presents an interactive face retrieval framework for clarifying an image representation envisioned by a user. Our system is designed for a situation in which the user wishes to find a person but has only visual memory of the person. We address a critical challenge of image retrieval across the user’s inputs. Instead of target-specific information, the user can select several images that are similar to an impression of the target person the user wishes to search for. Based on the user’s selection, our proposed system automatically updates a deep convolutional neural network. By interactively repeating these process, the system can reduce the gap between human-based similarities and computer-based similarities and estimate the target image representation. We ran user studies with 10 participants on a public database and confirmed that the proposed framework is effective for clarifying the image representation envisioned by the user easily and quickly.

KW - Active learning

KW - Deep convolutional neural network

KW - Face retrieval

KW - Relevance feedback

KW - User interaction

UR - http://www.scopus.com/inward/record.url?scp=85065100927&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85065100927&partnerID=8YFLogxK

U2 - 10.3169/mta.7.68

DO - 10.3169/mta.7.68

M3 - Article

AN - SCOPUS:85065100927

VL - 7

SP - 68

EP - 79

JO - ITE Transactions on Media Technology and Applications

JF - ITE Transactions on Media Technology and Applications

SN - 2186-7364

IS - 2

ER -