Venue & metadata

• Journal/Proceedings: Procedia Computer Science
• Volume: 192
• Pages: 3570 – 3579
• Note: Cited by: 27; All Open Access, Gold Open Access
• Author keywords: Data poisoning attacks; Deep learning vulnerabilities; Natural language processing; Poisoned word embeddings; Reliable machine learning

Abstract

Natural Language Processing (NLP) is being recently explored also to its application in supporting malicious activities and objects detection. Furthermore, NLP and Deep Learning have become targets of malicious attacks too. Very recent researches evidenced that adversarial attacks are able to affect also NLP tasks, in addition to the more popular adversarial attacks on deep learning systems for image processing tasks. More precisely, while small perturbations applied to the data set adopted for training typical NLP tasks (e.g., Part-of-Speech Tagging, Named Entity Recognition, etc..) could be easily recognized, models poisoning, performed by the means of altered data models, typically provided in the transfer learning phase to a deep neural networks (e.g., poisoning attacks by word embeddings), are harder to be detected. In this work, we preliminary explore the effectiveness of a poisoned word embeddings attack aimed at a deep neural network trained to accomplish a Named Entity Recognition (NER) task. By adopting the NER case study, we aimed to analyze the severity of such a kind of attack to accuracy in recognizing the right classes for the given entities. Finally, this study represents a preliminary step to assess the impact and the vulnerabilities of some NLP systems we adopt in our research activities, and further investigating some potential mitigation strategies, in order to make these systems more resilient to data and models poisoning attacks. © 2021 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0) Peer-review under responsibility of the scientific committee of KES International.

Keywords

Computational linguistics GS Deep neural networks GS Embeddings GS Object detection GS Speech recognition GS Activity detection GS Data poisoning attack GS Deep learning vulnerability GS Embeddings GS ITS applications GS Malicious activities GS Named entity recognition GS Poisoned word embedding GS Poisoning attacks GS Reliable machine learning GS Natural language processing systems GS

Links & artifacts

DOI Publisher Copy BibTeX Download .bib

{"key"=>"Marulli20213570", "type"=>"Conference paper", "bibtex"=>"@conference{Marulli20213570,\n  author = {Marulli, Fiammetta and Verde, Laura and Campanile, Lelio},\n  title = {Exploring data and model poisoning attacks to deep learning-based NLP systems},\n  year = {2021},\n  journal = {Procedia Computer Science},\n  volume = {192},\n  pages = {3570 – 3579},\n  doi = {10.1016/j.procs.2021.09.130},\n  type = {Conference paper}\n}\n", "author"=>"Marulli, Fiammetta and Verde, Laura and Campanile, Lelio", "author_array"=>[{"first"=>"Fiammetta", "last"=>"Marulli", "prefix"=>"", "suffix"=>""}, {"first"=>"Laura", "last"=>"Verde", "prefix"=>"", "suffix"=>""}, {"first"=>"Lelio", "last"=>"Campanile", "prefix"=>"", "suffix"=>""}], "author_0_first"=>"Fiammetta", "author_0_last"=>"Marulli", "author_0_prefix"=>"", "author_0_suffix"=>"", "author_1_first"=>"Laura", "author_1_last"=>"Verde", "author_1_prefix"=>"", "author_1_suffix"=>"", "author_2_first"=>"Lelio", "author_2_last"=>"Campanile", "author_2_prefix"=>"", "author_2_suffix"=>"", "title"=>"Exploring data and model poisoning attacks to deep learning-based NLP systems", "year"=>"2021", "journal"=>"Procedia Computer Science", "volume"=>"192", "pages"=>"3570 – 3579", "doi"=>"10.1016/j.procs.2021.09.130", "url"=>"https://www.scopus.com/inward/record.uri?eid=2-s2.0-85116922484&doi=10.1016%2fj.procs.2021.09.130&partnerID=40&md5=bdd9c6e94db10582de39cd67012dfa4c", "abstract"=>"Natural Language Processing (NLP) is being recently explored also to its application in supporting malicious activities and objects detection. Furthermore, NLP and Deep Learning have become targets of malicious attacks too. Very recent researches evidenced that adversarial attacks are able to affect also NLP tasks, in addition to the more popular adversarial attacks on deep learning systems for image processing tasks. More precisely, while small perturbations applied to the data set adopted for training typical NLP tasks (e.g., Part-of-Speech Tagging, Named Entity Recognition, etc..) could be easily recognized, models poisoning, performed by the means of altered data models, typically provided in the transfer learning phase to a deep neural networks (e.g., poisoning attacks by word embeddings), are harder to be detected. In this work, we preliminary explore the effectiveness of a poisoned word embeddings attack aimed at a deep neural network trained to accomplish a Named Entity Recognition (NER) task. By adopting the NER case study, we aimed to analyze the severity of such a kind of attack to accuracy in recognizing the right classes for the given entities. Finally, this study represents a preliminary step to assess the impact and the vulnerabilities of some NLP systems we adopt in our research activities, and further investigating some potential mitigation strategies, in order to make these systems more resilient to data and models poisoning attacks. © 2021 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0) Peer-review under responsibility of the scientific committee of KES International.", "author_keywords"=>"Data poisoning attacks; Deep learning vulnerabilities; Natural language processing; Poisoned word embeddings; Reliable machine learning", "keywords"=>"Computational linguistics; Deep neural networks; Embeddings; Object detection; Speech recognition; Activity detection; Data poisoning attack; Deep learning vulnerability; Embeddings; ITS applications; Malicious activities; Named entity recognition; Poisoned word embedding; Poisoning attacks; Reliable machine learning; Natural language processing systems", "publication_stage"=>"Final", "source"=>"Scopus", "note"=>"Cited by: 27; All Open Access, Gold Open Access"}
    

Suggested citation

Marulli, F., Verde, L., & Campanile, L. (2021). Exploring data and model poisoning attacks to deep learning-based NLP systems [Conference paper]. Procedia Computer Science, 192, 3570–3579. https://doi.org/10.1016/j.procs.2021.09.130