Automatic audio-visual fusion for aggression detection using meta-information
conference paper
We propose a new method for audio-visual sensor fusion and apply it to automatic aggression detection. While a variety of definitions of aggression exist, in this paper we see it as any kind of behavior that has a disturbing effect on others. We have collected multi- and unimodal assessments by humans, who have given aggression scores on a 3 point scale. There are no trivial fusion algorithms to predict the multimodal labels from the unimodal labels. We propose an intermediate step to discover the structure in the fusion process. We call these metafeatures and we find a set of five which have an impact on the fusion process. We use simple state of the art low level audio and video features to predict the level of aggression in audio and video, and we also predict the three most feasible metafeatures. We show the significant positive impact of adding the meta-features on predicting the multimodal label as compared to standard fusion techniques like feature and decision level fusion. © 2012 IEEE.
TNO Identifier
465811
Publisher
IEEE
Article nr.
6327978
Source title
2012 IEEE 9th International Conference on Advanced Video and Signal-Based Surveillance, AVSS 2012, 18-21 September 2012, Beijing, China
Place of publication
Piscataway, NJ
Pages
19-24
Files
To receive the publication files, please send an e-mail request to TNO Repository.