Proactive Detection of Query-based Adversarial Scenarios in NLP Systems

Mohammad Maghsoudimehrabani, Amin Azmoodeh, Ali Dehghantanha, Behrouz Zolfaghari, Gautam Srivastava

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Adversarial attacks can mislead a Deep Learning (DL) algorithm into generating erroneous predictions via feeding maliciously-disturbed inputs called adversarial examples. DL-based Natural Language Processing (NLP) algorithms are severely threatened by adversarial attacks. In real-world, black-box adversarial attacks, the adversary needs to submit many highly-similar queries before drafting an adversarial example. Due to this long process, in-progress attack detection can play a significant role in adversarial defense in DL-based NLP algorithms. Although there are several approaches for detecting adversarial attacks in NLP, these approaches are reactive in the sense that they can detect adversarial examples only when they are fabricated and fed into the algorithm. In this study, we take one step towards proactive detection of adversarial attacks in NLP systems by proposing a robust, history-based model named Stateful Query Analysis (SQA) to identify suspiciously-similar sequences of queries capable of generating textual adversarial examples to which we refer by adversarial scenarios. The model exhibits a detection rate of over 99.9% in our extensive experimental tests against several state-of-The-Art black-box adversarial attack methods.

Original languageEnglish
Title of host publicationAISec 2022 - Proceedings of the 15th ACM Workshop on Artificial Intelligence and Security, co-located with CCS 2022
PublisherAssociation for Computing Machinery, Inc
Pages103-113
Number of pages11
ISBN (Electronic)9781450398800
DOIs
Publication statusPublished - 2022 Nov 11
Externally publishedYes
Event15th ACM Workshop on Artificial Intelligence and Security, AISec 2022 - Co-located with CCS 2022 - Los Angeles, United States
Duration: 2022 Nov 11 → …

Publication series

NameAISec 2022 - Proceedings of the 15th ACM Workshop on Artificial Intelligence and Security, co-located with CCS 2022

Conference

Conference15th ACM Workshop on Artificial Intelligence and Security, AISec 2022 - Co-located with CCS 2022
Country/TerritoryUnited States
CityLos Angeles
Period22/11/11 → …

Keywords

  • adversarial attack detection
  • natural language processing
  • textual adversarial example

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Software

Fingerprint

Dive into the research topics of 'Proactive Detection of Query-based Adversarial Scenarios in NLP Systems'. Together they form a unique fingerprint.

Cite this