Understanding the Behavior Transparency of Voice Assistant Applications Using the ChatterBox Framework

Atsuko Natatsuka, Ryo Iijima, Takuya Watanabe, Mitsuaki Akiyama, Tetsuya Sakai, Tatsuya Mori

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

A voice assistant (VA) is a platform that provides users with a wide range of services via interaction with a voice application using verbal commands. Since the VA application is deployed in the cloud, its behavior is not transparent to the user, which raises privacy concerns. In this study, we developed a framework called ChatterBox, which attempts to analyze VA applications via extensive continuous interaction, to understand their behavior. ChatterBox is capable of parsing and generating dialogues by utilizing natural language processing approach. It can also parse application-level messages to understand how a VA app acquires personal information. ChatterBox supports English and Japanese, which are completely different languages, and can extract more than twice as many dialogues from VA applications compared to SkillExplorer, a state-of-the-art VA dialogue analysis system. Based on analyses of English and Japanese VA applications using ChatterBox, we revealed that 5-15% of VA applications collect personal information or recorded user identifiers in a non-transparent manner, and 76-94% applications collected personal information without providing appropriate privacy policies. In light of these findings, we discuss the implementation of a highly transparent VA application platform.

Original languageEnglish
Title of host publicationProceedings of 25th International Symposium on Researchin Attacks, Intrusions and Defenses, RAID 2022
PublisherAssociation for Computing Machinery
Pages143-159
Number of pages17
ISBN (Electronic)9781450397049
DOIs
Publication statusPublished - 2022 Oct 26
Event25th International Symposium on Researchin Attacks, Intrusions and Defenses, RAID 2022 - Limassol, Cyprus
Duration: 2022 Oct 262022 Oct 28

Publication series

NameACM International Conference Proceeding Series

Conference

Conference25th International Symposium on Researchin Attacks, Intrusions and Defenses, RAID 2022
Country/TerritoryCyprus
CityLimassol
Period22/10/2622/10/28

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Networks and Communications
  • Computer Vision and Pattern Recognition
  • Software

Fingerprint

Dive into the research topics of 'Understanding the Behavior Transparency of Voice Assistant Applications Using the ChatterBox Framework'. Together they form a unique fingerprint.

Cite this