Optimizable Ensemble Regression for Arousal and Valence Predictions from Visual Features

The cognitive state of a person can be categorized using the Circumplex model of emotional states. The Circumplex model is a continuous model of two dimensions: arousal and valence, where arousal measures the energy level and valence measures the positivity level of a person's emotion. The arousal and valence values can be estimated via machine learning regression. We exploit the Remote Collaborative and Affective Interactions ‎‎(RECOLA) dataset which includes audio, video, and physiological recordings of online interactions between human participants. We previously succeeded to predict arousal and valence values using the physiological [1,2] and video [2,3] recordings of RECOLA. Features are attributes that describe the data. They can be predesigned or learned. Learned features are attributes that are automatically learned and utilized by deep machine learning solutions. On the other hand, predesigned features are attributes that are calculated before the machine learning process, and provided as input to the machine learner later on. Our previous work on the video recordings of RECOLA focused on learned features. In this paper, we expand our work to analyze and assess the predesigned visual features, extracted from the video recordings of RECOLA, for predicting the arousal and valence values of cognitive/emotional states. We process the visual features of RECOLA by applying time delay and sequencing, arousal and valence annotation labelling, and data shuffling and splitting. We then train and test machine learning regressors to predict the arousal and valence values. Our preliminary results outperform those from the literature. We have achieved a testing root mean squared error (RMSE), Pearson’s correlation coefficient (PCC), and concordance correlation coefficient (CCC) of ‎0.1033, 0.8498, ‎‎and 0.8001‎ on arousal predictions, respectively. We have achieved a testing RMSE, PCC, and CCC of ‎0.07016‎, 0.8473, and 0.8053 on valence predictions, respectively. These performances are obtained using an optimizable ensemble regressor.