Towards Expressive Perception and Generation in Human-Computer Conversational Interaction

Yaohua Bu · Runnan Li · Zhenwei You

Jul 2024 · Scientific Research Publishing, Inc. USA

About this ebook

The construction of a natural interactive human-computer interaction has become an integral component of intelligent system development, constituting a core subject within the field of human-computer interaction. Grounded in the utilization of human perception theories, this study proposes a robust method for speech emotion recognition and a model for inferring user emotional state changes, thereby achieving a hu-man-computer interaction experience characterized by both listening and articulating, and conveying sentiments effectively. Simultaneously, focusing on the audio-visual modalities within human-computer interaction, this research explores methods for con-structing interactive feedback in auditory and visual modalities within real hu-man-computer dialog scenarios. This aims to synchronize textual, verbal, and visual expressions in human-computer interaction, enhancing its naturalness, improving user experience, and augmenting satisfaction and pleasure during interaction. The research contributions are as follows:

Introducing a method that robustly identifies user emotional states in real hu-man-computer speech dialogue scenarios through the utilization of local and global attention mechanisms. This method generates robust and distinguishable representa-tions of speech emotion, thereby enhancing the robustness and accuracy of speech emotion recognition systems in authentic dialogue scenarios.

About the author

Dr. Yaohua Bu, PhD in Design, Master of Science in Computer Science and Technology, is currently a lecturer in the Design Department of the School of Digital Media and De-sign Arts at Beijing University of Posts and Telecommunications. Her research interests include multimedia human-computer interaction and virtual human interaction design. She has published more than 10 academic papers at top international conferences such as ACM CHI, ACM MM, and AAAI.

Dr. Runnan Li, Associate Professor in Beijing University of Posts and Telecommunica-tions, member of the CCF Human-Computer Interaction Committee, member of the CCF Multimedia Technology Committee, has conducted extensive research in the field of human-computer interaction and has led the development and implementation of various HCI products in the industry, gaining deep insights into HCI applications.

Dr. Zhenwei You, Professor, member of the German Design Council, member of Japan Designers’ Association, member of Design Ergonomics Branch of Chinese Ergonomics Society, member of Beijing Mechanical Engineering Society, is currently the head of the design department in the School of Digital Media and Design Arts of Beijing University of Posts and Telecommunications, and the editor-in-chief of the American Journal Art and Design Review.