Venue & metadata

• Journal/Proceedings: Procedia Computer Science
• Volume: 225
• Pages: 1241 – 1250
• Note: Cited by: 1; All Open Access, Gold Open Access
• Author keywords: Emotional State Inference; Human Digital Twin; Human Machine Interaction; Process Mining; Speech Analysis

Abstract

Human-Machine Interfaces (HMIs) are getting more and more important in a hyper-connected society. Traditional HMIs are built considering cognitive features while emotional ones are often neglected, bringing sometimes such interfaces to misuse. As a part of a long run research, oriented to the definition of an HMI engineering approach, this paper concretely proposes a method to build an emotional-aware explicit model of the user starting from the behaviour of the human with a virtual agent. The paper also proposes an instance of this model inference process in voice assistants in an automatic depression context, which can constitute the core phase to realize a Human Digital Twin of a patient. The case study generated a model composed of Fluid Stochastic Petri Net sub-models, achieved after the data analysis by a Support Vector Machine. © 2023 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0)

Keywords

Behavioral research GS Emotion Recognition GS Man machine systems GS Petri nets GS Stochastic models GS Stochastic systems GS Virtual reality GS Emotional models GS Emotional state GS Emotional state inference GS Human digital twin GS Human machine interaction GS Human Machine Interface GS Human-machine GS Longest run GS Process mining GS Speech interaction GS Support vector machines GS

Links & artifacts

DOI Publisher Copy BibTeX Download .bib

{"key"=>"Campanile20231241", "type"=>"Conference paper", "bibtex"=>"@conference{Campanile20231241,\n  author = {Campanile, Lelio and de Fazio, Roberta and Di Giovanni, Michele and Marrone, Stefano and Marulli, Fiammetta and Verde, Laura},\n  title = {Inferring Emotional Models from Human-Machine Speech Interactions},\n  year = {2023},\n  journal = {Procedia Computer Science},\n  volume = {225},\n  pages = {1241 – 1250},\n  doi = {10.1016/j.procs.2023.10.112},\n  type = {Conference paper}\n}\n", "author"=>"Campanile, Lelio and de Fazio, Roberta and Di Giovanni, Michele and Marrone, Stefano and Marulli, Fiammetta and Verde, Laura", "author_array"=>[{"first"=>"Lelio", "last"=>"Campanile", "prefix"=>"", "suffix"=>""}, {"first"=>"Roberta", "last"=>"Fazio", "prefix"=>"de", "suffix"=>""}, {"first"=>"Michele", "last"=>"Di Giovanni", "prefix"=>"", "suffix"=>""}, {"first"=>"Stefano", "last"=>"Marrone", "prefix"=>"", "suffix"=>""}, {"first"=>"Fiammetta", "last"=>"Marulli", "prefix"=>"", "suffix"=>""}, {"first"=>"Laura", "last"=>"Verde", "prefix"=>"", "suffix"=>""}], "author_0_first"=>"Lelio", "author_0_last"=>"Campanile", "author_0_prefix"=>"", "author_0_suffix"=>"", "author_1_first"=>"Roberta", "author_1_last"=>"Fazio", "author_1_prefix"=>"de", "author_1_suffix"=>"", "author_2_first"=>"Michele", "author_2_last"=>"Di Giovanni", "author_2_prefix"=>"", "author_2_suffix"=>"", "author_3_first"=>"Stefano", "author_3_last"=>"Marrone", "author_3_prefix"=>"", "author_3_suffix"=>"", "author_4_first"=>"Fiammetta", "author_4_last"=>"Marulli", "author_4_prefix"=>"", "author_4_suffix"=>"", "author_5_first"=>"Laura", "author_5_last"=>"Verde", "author_5_prefix"=>"", "author_5_suffix"=>"", "title"=>"Inferring Emotional Models from Human-Machine Speech Interactions", "year"=>"2023", "journal"=>"Procedia Computer Science", "volume"=>"225", "pages"=>"1241 – 1250", "doi"=>"10.1016/j.procs.2023.10.112", "url"=>"https://www.scopus.com/inward/record.uri?eid=2-s2.0-85183548328&doi=10.1016%2fj.procs.2023.10.112&partnerID=40&md5=40fa5982063fbd25e375c1ae365a69e4", "abstract"=>"Human-Machine Interfaces (HMIs) are getting more and more important in a hyper-connected society. Traditional HMIs are built considering cognitive features while emotional ones are often neglected, bringing sometimes such interfaces to misuse. As a part of a long run research, oriented to the definition of an HMI engineering approach, this paper concretely proposes a method to build an emotional-aware explicit model of the user starting from the behaviour of the human with a virtual agent. The paper also proposes an instance of this model inference process in voice assistants in an automatic depression context, which can constitute the core phase to realize a Human Digital Twin of a patient. The case study generated a model composed of Fluid Stochastic Petri Net sub-models, achieved after the data analysis by a Support Vector Machine. © 2023 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0)", "author_keywords"=>"Emotional State Inference; Human Digital Twin; Human Machine Interaction; Process Mining; Speech Analysis", "keywords"=>"Behavioral research; Emotion Recognition; Man machine systems; Petri nets; Stochastic models; Stochastic systems; Virtual reality; Emotional models; Emotional state; Emotional state inference; Human digital twin; Human machine interaction; Human Machine Interface; Human-machine; Longest run; Process mining; Speech interaction; Support vector machines", "publication_stage"=>"Final", "source"=>"Scopus", "note"=>"Cited by: 1; All Open Access, Gold Open Access"}
    

Suggested citation

Campanile, L., de Fazio, R., Di Giovanni, M., Marrone, S., Marulli, F., & Verde, L. (2023). Inferring Emotional Models from Human-Machine Speech Interactions [Conference paper]. Procedia Computer Science, 225, 1241–1250. https://doi.org/10.1016/j.procs.2023.10.112