Modality encoded latent dataset for emotion recognition
Küçük Resim Yok
Tarih
2023
Yazarlar
Dergi Başlığı
Dergi ISSN
Cilt Başlığı
Yayıncı
Elsevier Sci Ltd
Erişim Hakkı
info:eu-repo/semantics/closedAccess
Özet
Variational autoencoder (VAE) is an unsupervised learning that represents high dimensional input data into normally distributed latent space. Multi-channel physiological signals, namely EEG and peripherals are mostly preferred for affective computing. The DEAP dataset is converted into multimodal latent dataset for emotion recognition in this study. 40-ch recordings of 32 participants are encoded to different modalities of peripherals and 32-ch EEG. First, short-time Fourier transform (STFT) is used to extract time-frequency (TF) distribution for training VAE. Thus, the localized components in the each channel of the modalities is converted to 100 -dimensional space using VAE. The proposed method is applied to each participant's recordings to obtain new latent encoded dataset. Within and between subject classification results using latent dataset are compared to the original data for peripheral, 32ch EEG and peripheral with EEG modalities. Naive Bayes (NB) classifier is used to evaluate the encoding performance of the 100-dimensional modalities, and compared to original results. The error rates of leave-one participant-out cross-validation (LOPO CV) 0.3322 and 0.3327 are yielded for high/low arousal and valence states while the originals are 0.349 and 0.382.
Açıklama
Anahtar Kelimeler
Variational autoencoder, Emotion recognition, Multimodal data fusion, Latent space
Kaynak
Biomedical Signal Processing and Control
WoS Q Değeri
Q2
Scopus Q Değeri
Q1
Cilt
79
