Measuring affective state: Subject-dependent and - independent prediction based on longitudinal multimodal sensing
conference paper
Current sensors offering passive and continuous monitoring of behavioral patterns potentially enable real-time affective state monitoring. Previous research on affective state prediction with multimodal sensing in daily life has shown only small-to-moderate effects. One reason for this limited success might be the large variability across individuals. Current research is often of short duration, preventing proper within-individual modeling. With an extensive longitudinal data collection of nine months, this research focuses on individual-level predictions of valence and arousal in daily life. Sixteen PhD candidates from the Netherlands provided data about their affective states (self-reported valence and arousal), physiology (Oura rings) and behavioral patterns (AWARE framework for mobile phone data). Supporting our hypothesis, subject-dependent random forest (RF) models significantly outperformed subject-independent leave-one-subject-out (LOSO) models in predicting self-reported valence and arousal. The subject-dependent models achieved an average Spearman’s rho correlation of 0.28 [0.14-0.60] for valence and 0.36 [0.16-0.69] for arousal. In many cases, participants’ a priori indicated informative sources matched with the feature importance. Making use of participants’ self-knowledge might thus help to reduce the amount of data to be collected. For future work, longer-term changes in affective state and combinations of features for estimating real behavioral patterns should be explored.
Topics
TNO Identifier
1001274
Publisher
IEEE
Source title
IEEE Transactions on Affective Computing
Files
To receive the publication files, please send an e-mail request to TNO Repository.