Show Something

Intelligent Shopping Assistant Supporting Quick Scene Understanding and Immersive Preview

Hao Dou, Zhinan Li, Minghao Cai, Kelvin Cheng, Soh Masuko, Jiro Tanaka

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper, we introduce an intelligent shopping assistant system supported by quick scene understanding and augmented-reality 3D preview. By understanding the scene that users are looking at, and using the detected scene information, our system recommends related products that are not in the current scene and which could potentially interest users. With the help of existing speech recognition techniques, our system extracts users’ voice command and keywords, and provides responses in real-time, which allows users to search and filter specific products just by using voice. After finding the potential target products, our system provides users with an augmented-reality preview experience. It automatically brings products to the suitable space in front of the users by using life-size three-dimensional virtual products and spatial understanding. Users can also use two-hand gestural manipulation to operate the virtual products. Through our system, users can obtain products that are strongly related to the current environment, and intuitively preview the products in the current scene by automatic placement and two-hand manipulation to make shopping decisions.

Original languageEnglish
Title of host publicationHuman Interface and the Management of Information. Information in Intelligent Systems - Thematic Area, HIMI 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Proceedings
EditorsHirohiko Mori, Sakae Yamamoto
PublisherSpringer-Verlag
Pages205-218
Number of pages14
ISBN (Print)9783030226480
DOIs
Publication statusPublished - 2019 Jan 1
EventThematic Area on Human Interface and the Management of Information, HIMI 2019, held as part of the 21st International Conference on Human-Computer Interaction, HCI International 2019 - Orlando, United States
Duration: 2019 Jul 262019 Jul 31

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11570 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceThematic Area on Human Interface and the Management of Information, HIMI 2019, held as part of the 21st International Conference on Human-Computer Interaction, HCI International 2019
CountryUnited States
CityOrlando
Period19/7/2619/7/31

Fingerprint

Augmented reality
Speech recognition
Information systems
Augmented Reality
Manipulation
Product Systems
Speech Recognition
Placement
Information Systems
Filter
Real-time
Three-dimensional
Target

Keywords

  • Context awareness
  • Immersive shopping
  • Intelligent system

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Dou, H., Li, Z., Cai, M., Cheng, K., Masuko, S., & Tanaka, J. (2019). Show Something: Intelligent Shopping Assistant Supporting Quick Scene Understanding and Immersive Preview. In H. Mori, & S. Yamamoto (Eds.), Human Interface and the Management of Information. Information in Intelligent Systems - Thematic Area, HIMI 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Proceedings (pp. 205-218). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11570 LNCS). Springer-Verlag. https://doi.org/10.1007/978-3-030-22649-7_17

Show Something : Intelligent Shopping Assistant Supporting Quick Scene Understanding and Immersive Preview. / Dou, Hao; Li, Zhinan; Cai, Minghao; Cheng, Kelvin; Masuko, Soh; Tanaka, Jiro.

Human Interface and the Management of Information. Information in Intelligent Systems - Thematic Area, HIMI 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Proceedings. ed. / Hirohiko Mori; Sakae Yamamoto. Springer-Verlag, 2019. p. 205-218 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11570 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Dou, H, Li, Z, Cai, M, Cheng, K, Masuko, S & Tanaka, J 2019, Show Something: Intelligent Shopping Assistant Supporting Quick Scene Understanding and Immersive Preview. in H Mori & S Yamamoto (eds), Human Interface and the Management of Information. Information in Intelligent Systems - Thematic Area, HIMI 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11570 LNCS, Springer-Verlag, pp. 205-218, Thematic Area on Human Interface and the Management of Information, HIMI 2019, held as part of the 21st International Conference on Human-Computer Interaction, HCI International 2019, Orlando, United States, 19/7/26. https://doi.org/10.1007/978-3-030-22649-7_17
Dou H, Li Z, Cai M, Cheng K, Masuko S, Tanaka J. Show Something: Intelligent Shopping Assistant Supporting Quick Scene Understanding and Immersive Preview. In Mori H, Yamamoto S, editors, Human Interface and the Management of Information. Information in Intelligent Systems - Thematic Area, HIMI 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Proceedings. Springer-Verlag. 2019. p. 205-218. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-22649-7_17
Dou, Hao ; Li, Zhinan ; Cai, Minghao ; Cheng, Kelvin ; Masuko, Soh ; Tanaka, Jiro. / Show Something : Intelligent Shopping Assistant Supporting Quick Scene Understanding and Immersive Preview. Human Interface and the Management of Information. Information in Intelligent Systems - Thematic Area, HIMI 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Proceedings. editor / Hirohiko Mori ; Sakae Yamamoto. Springer-Verlag, 2019. pp. 205-218 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{30e34a06c69e47659ffd9ccadde1fc54,
title = "Show Something: Intelligent Shopping Assistant Supporting Quick Scene Understanding and Immersive Preview",
abstract = "In this paper, we introduce an intelligent shopping assistant system supported by quick scene understanding and augmented-reality 3D preview. By understanding the scene that users are looking at, and using the detected scene information, our system recommends related products that are not in the current scene and which could potentially interest users. With the help of existing speech recognition techniques, our system extracts users’ voice command and keywords, and provides responses in real-time, which allows users to search and filter specific products just by using voice. After finding the potential target products, our system provides users with an augmented-reality preview experience. It automatically brings products to the suitable space in front of the users by using life-size three-dimensional virtual products and spatial understanding. Users can also use two-hand gestural manipulation to operate the virtual products. Through our system, users can obtain products that are strongly related to the current environment, and intuitively preview the products in the current scene by automatic placement and two-hand manipulation to make shopping decisions.",
keywords = "Context awareness, Immersive shopping, Intelligent system",
author = "Hao Dou and Zhinan Li and Minghao Cai and Kelvin Cheng and Soh Masuko and Jiro Tanaka",
year = "2019",
month = "1",
day = "1",
doi = "10.1007/978-3-030-22649-7_17",
language = "English",
isbn = "9783030226480",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer-Verlag",
pages = "205--218",
editor = "Hirohiko Mori and Sakae Yamamoto",
booktitle = "Human Interface and the Management of Information. Information in Intelligent Systems - Thematic Area, HIMI 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Proceedings",

}

TY - GEN

T1 - Show Something

T2 - Intelligent Shopping Assistant Supporting Quick Scene Understanding and Immersive Preview

AU - Dou, Hao

AU - Li, Zhinan

AU - Cai, Minghao

AU - Cheng, Kelvin

AU - Masuko, Soh

AU - Tanaka, Jiro

PY - 2019/1/1

Y1 - 2019/1/1

N2 - In this paper, we introduce an intelligent shopping assistant system supported by quick scene understanding and augmented-reality 3D preview. By understanding the scene that users are looking at, and using the detected scene information, our system recommends related products that are not in the current scene and which could potentially interest users. With the help of existing speech recognition techniques, our system extracts users’ voice command and keywords, and provides responses in real-time, which allows users to search and filter specific products just by using voice. After finding the potential target products, our system provides users with an augmented-reality preview experience. It automatically brings products to the suitable space in front of the users by using life-size three-dimensional virtual products and spatial understanding. Users can also use two-hand gestural manipulation to operate the virtual products. Through our system, users can obtain products that are strongly related to the current environment, and intuitively preview the products in the current scene by automatic placement and two-hand manipulation to make shopping decisions.

AB - In this paper, we introduce an intelligent shopping assistant system supported by quick scene understanding and augmented-reality 3D preview. By understanding the scene that users are looking at, and using the detected scene information, our system recommends related products that are not in the current scene and which could potentially interest users. With the help of existing speech recognition techniques, our system extracts users’ voice command and keywords, and provides responses in real-time, which allows users to search and filter specific products just by using voice. After finding the potential target products, our system provides users with an augmented-reality preview experience. It automatically brings products to the suitable space in front of the users by using life-size three-dimensional virtual products and spatial understanding. Users can also use two-hand gestural manipulation to operate the virtual products. Through our system, users can obtain products that are strongly related to the current environment, and intuitively preview the products in the current scene by automatic placement and two-hand manipulation to make shopping decisions.

KW - Context awareness

KW - Immersive shopping

KW - Intelligent system

UR - http://www.scopus.com/inward/record.url?scp=85069642873&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85069642873&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-22649-7_17

DO - 10.1007/978-3-030-22649-7_17

M3 - Conference contribution

SN - 9783030226480

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 205

EP - 218

BT - Human Interface and the Management of Information. Information in Intelligent Systems - Thematic Area, HIMI 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Proceedings

A2 - Mori, Hirohiko

A2 - Yamamoto, Sakae

PB - Springer-Verlag

ER -