Brainwave-Driven Emotional Speech Analysis using Adversarial Networks and Combined Deep RL Approaches for Improved Recognition Performance

Recognition of emotions in speech is a complex problem because of the inconsistency in vocal expressions and the absence of the trusted internal emotional expressions. This paper presents Brainwave-Driven Emotional Speech Analysis, a composite system combining adversarial synthetic EEG generation and deep reinforcement learning-based classifier in order to improve the accuracy of emotion recognition. It begins by extracting high-level acoustic features of input speech and then uses a Progressive Growing Generative Adversarial Network (PG-GAN) to produce synthetic EEG signal energies that are related to emotions. The resulting EEG signal is denoised with a Savitzky Golay denoising filter followed by additional processing by a combined deep learning network comprising of LSTM, GRU and Deep Q-network models to provide strong emotion classification in positive, negative and neutral affective states. Experimental analysis proves better recognition results than traditional audio only models, with emphasis on the merits of the use of synthesized neural patterns as auxiliary emotional signals. The given solution offers a new avenue of improving affective computing systems through the application of adversarial brainwave synthesis to supplement scarce real EEG data and enhance the processing of emotional context in speech-based applications.