Show Something: Intelligent Shopping Assistant Supporting Quick Scene Understanding and Immersive Preview

Hao Dou, Zhinan Li, Minghao Cai, Kelvin Cheng, Soh Masuko, Jiro Tanaka

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper, we introduce an intelligent shopping assistant system supported by quick scene understanding and augmented-reality 3D preview. By understanding the scene that users are looking at, and using the detected scene information, our system recommends related products that are not in the current scene and which could potentially interest users. With the help of existing speech recognition techniques, our system extracts users’ voice command and keywords, and provides responses in real-time, which allows users to search and filter specific products just by using voice. After finding the potential target products, our system provides users with an augmented-reality preview experience. It automatically brings products to the suitable space in front of the users by using life-size three-dimensional virtual products and spatial understanding. Users can also use two-hand gestural manipulation to operate the virtual products. Through our system, users can obtain products that are strongly related to the current environment, and intuitively preview the products in the current scene by automatic placement and two-hand manipulation to make shopping decisions.

Original languageEnglish
Title of host publicationHuman Interface and the Management of Information. Information in Intelligent Systems - Thematic Area, HIMI 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Proceedings
EditorsHirohiko Mori, Sakae Yamamoto
PublisherSpringer-Verlag
Pages205-218
Number of pages14
ISBN (Print)9783030226480
DOIs
Publication statusPublished - 2019 Jan 1
EventThematic Area on Human Interface and the Management of Information, HIMI 2019, held as part of the 21st International Conference on Human-Computer Interaction, HCI International 2019 - Orlando, United States
Duration: 2019 Jul 262019 Jul 31

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11570 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceThematic Area on Human Interface and the Management of Information, HIMI 2019, held as part of the 21st International Conference on Human-Computer Interaction, HCI International 2019
CountryUnited States
CityOrlando
Period19/7/2619/7/31

Keywords

  • Context awareness
  • Immersive shopping
  • Intelligent system

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Show Something: Intelligent Shopping Assistant Supporting Quick Scene Understanding and Immersive Preview'. Together they form a unique fingerprint.

  • Cite this

    Dou, H., Li, Z., Cai, M., Cheng, K., Masuko, S., & Tanaka, J. (2019). Show Something: Intelligent Shopping Assistant Supporting Quick Scene Understanding and Immersive Preview. In H. Mori, & S. Yamamoto (Eds.), Human Interface and the Management of Information. Information in Intelligent Systems - Thematic Area, HIMI 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Proceedings (pp. 205-218). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11570 LNCS). Springer-Verlag. https://doi.org/10.1007/978-3-030-22649-7_17