Face retrieval framework relying on user's visual memory

Yugo Sato, Tsukasa Fukusato, Shigeo Morishima

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

This paper presents an interactive face retrieval framework for clarifying an image representation envisioned by a user. Our system is designed for a situation in which the user wishes to find a person but has only visual memory of the person. We address a critical challenge of image retrieval across the user's inputs. Instead of target-specific information, the user can select several images (or a single image) that are similar to an impression of the target person the user wishes to search for. Based on the user's selection, our proposed system automatically updates a deep convolutional neural network. By interactively repeating these process (human-in-the-loop optimization), the system can reduce the gap between human-based similarities and computer-based similarities and estimate the target image representation. We ran user studies with 10 subjects on a public database and confirmed that the proposed framework is effective for clarifying the image representation envisioned by the user easily and quickly.

Original languageEnglish
Title of host publicationICMR 2018 - Proceedings of the 2018 ACM International Conference on Multimedia Retrieval
PublisherAssociation for Computing Machinery, Inc
Pages274-282
Number of pages9
ISBN (Print)9781450350464
DOIs
Publication statusPublished - 2018 Jun 5
Event8th ACM International Conference on Multimedia Retrieval, ICMR 2018 - Yokohama, Japan
Duration: 2018 Jun 112018 Jun 14

Other

Other8th ACM International Conference on Multimedia Retrieval, ICMR 2018
CountryJapan
CityYokohama
Period18/6/1118/6/14

Fingerprint

Image retrieval
Neural networks
Data storage equipment

Keywords

  • Active learning
  • Deep convolutional neural network
  • Relevance feedback
  • User interaction

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Computer Networks and Communications
  • Computer Science Applications

Cite this

Sato, Y., Fukusato, T., & Morishima, S. (2018). Face retrieval framework relying on user's visual memory. In ICMR 2018 - Proceedings of the 2018 ACM International Conference on Multimedia Retrieval (pp. 274-282). Association for Computing Machinery, Inc. https://doi.org/10.1145/3206025.3206038

Face retrieval framework relying on user's visual memory. / Sato, Yugo; Fukusato, Tsukasa; Morishima, Shigeo.

ICMR 2018 - Proceedings of the 2018 ACM International Conference on Multimedia Retrieval. Association for Computing Machinery, Inc, 2018. p. 274-282.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Sato, Y, Fukusato, T & Morishima, S 2018, Face retrieval framework relying on user's visual memory. in ICMR 2018 - Proceedings of the 2018 ACM International Conference on Multimedia Retrieval. Association for Computing Machinery, Inc, pp. 274-282, 8th ACM International Conference on Multimedia Retrieval, ICMR 2018, Yokohama, Japan, 18/6/11. https://doi.org/10.1145/3206025.3206038
Sato Y, Fukusato T, Morishima S. Face retrieval framework relying on user's visual memory. In ICMR 2018 - Proceedings of the 2018 ACM International Conference on Multimedia Retrieval. Association for Computing Machinery, Inc. 2018. p. 274-282 https://doi.org/10.1145/3206025.3206038
Sato, Yugo ; Fukusato, Tsukasa ; Morishima, Shigeo. / Face retrieval framework relying on user's visual memory. ICMR 2018 - Proceedings of the 2018 ACM International Conference on Multimedia Retrieval. Association for Computing Machinery, Inc, 2018. pp. 274-282
@inproceedings{a7d23ab7324e45b7bb7480d4169ad7dd,
title = "Face retrieval framework relying on user's visual memory",
abstract = "This paper presents an interactive face retrieval framework for clarifying an image representation envisioned by a user. Our system is designed for a situation in which the user wishes to find a person but has only visual memory of the person. We address a critical challenge of image retrieval across the user's inputs. Instead of target-specific information, the user can select several images (or a single image) that are similar to an impression of the target person the user wishes to search for. Based on the user's selection, our proposed system automatically updates a deep convolutional neural network. By interactively repeating these process (human-in-the-loop optimization), the system can reduce the gap between human-based similarities and computer-based similarities and estimate the target image representation. We ran user studies with 10 subjects on a public database and confirmed that the proposed framework is effective for clarifying the image representation envisioned by the user easily and quickly.",
keywords = "Active learning, Deep convolutional neural network, Relevance feedback, User interaction",
author = "Yugo Sato and Tsukasa Fukusato and Shigeo Morishima",
year = "2018",
month = "6",
day = "5",
doi = "10.1145/3206025.3206038",
language = "English",
isbn = "9781450350464",
pages = "274--282",
booktitle = "ICMR 2018 - Proceedings of the 2018 ACM International Conference on Multimedia Retrieval",
publisher = "Association for Computing Machinery, Inc",

}

TY - GEN

T1 - Face retrieval framework relying on user's visual memory

AU - Sato, Yugo

AU - Fukusato, Tsukasa

AU - Morishima, Shigeo

PY - 2018/6/5

Y1 - 2018/6/5

N2 - This paper presents an interactive face retrieval framework for clarifying an image representation envisioned by a user. Our system is designed for a situation in which the user wishes to find a person but has only visual memory of the person. We address a critical challenge of image retrieval across the user's inputs. Instead of target-specific information, the user can select several images (or a single image) that are similar to an impression of the target person the user wishes to search for. Based on the user's selection, our proposed system automatically updates a deep convolutional neural network. By interactively repeating these process (human-in-the-loop optimization), the system can reduce the gap between human-based similarities and computer-based similarities and estimate the target image representation. We ran user studies with 10 subjects on a public database and confirmed that the proposed framework is effective for clarifying the image representation envisioned by the user easily and quickly.

AB - This paper presents an interactive face retrieval framework for clarifying an image representation envisioned by a user. Our system is designed for a situation in which the user wishes to find a person but has only visual memory of the person. We address a critical challenge of image retrieval across the user's inputs. Instead of target-specific information, the user can select several images (or a single image) that are similar to an impression of the target person the user wishes to search for. Based on the user's selection, our proposed system automatically updates a deep convolutional neural network. By interactively repeating these process (human-in-the-loop optimization), the system can reduce the gap between human-based similarities and computer-based similarities and estimate the target image representation. We ran user studies with 10 subjects on a public database and confirmed that the proposed framework is effective for clarifying the image representation envisioned by the user easily and quickly.

KW - Active learning

KW - Deep convolutional neural network

KW - Relevance feedback

KW - User interaction

UR - http://www.scopus.com/inward/record.url?scp=85053869416&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85053869416&partnerID=8YFLogxK

U2 - 10.1145/3206025.3206038

DO - 10.1145/3206025.3206038

M3 - Conference contribution

AN - SCOPUS:85053869416

SN - 9781450350464

SP - 274

EP - 282

BT - ICMR 2018 - Proceedings of the 2018 ACM International Conference on Multimedia Retrieval

PB - Association for Computing Machinery, Inc

ER -