Addressing multimodality in overt aggression detection

Addressing multimodality in overt aggression detection

Lefter, I.
Rothkrantz, L.J.M.
Burghouts, G.
Yang, Z.
Wiggers, P.

2011

Automatic detection of aggressive situations has a high societal and scientific relevance. It has been argued that using data from multimodal sensors as for example video and sound as opposed to unimodal is bound to increase the accuracy of detections. We approach the problem of multimodal aggression detection from the viewpoint of a human observer and try to reproduce his predictions automatically. Typically, a single ground truth for all available modalities is used when training recognizers. We explore the benefits of adding an extra level of annotations, namely audio-only and video-only. We analyze these annotations and compare them to the multimodal case in order to have more insight into how humans reason using multimodal data. We train classifiers and compare the results when using unimodal and multimodal labels as ground truth. Both in the case of audio and video recognizer the performance increases when using the unimodal labels. © 2011 Springer-Verlag.

Physics & Electronics
II - Intelligent Imaging
TS - Technical Sciences
Image processing
Behavior detection
Sensors
Multimodal
Agression detection
Audio
Video

http://resolver.tudelft.nl/uuid:88c4652b-efa9-4fbe-8952-f6516e5d7191

https://doi.org/10.1007/978-3-642-23538-2_4

435968

Springer, Berlin

9783642235375

0302-9743

14th International Conference on Text, Speech and Dialogue, TSD 2011, 1-5 September 2011, Pilsen, 6836 LNAI, 25-32

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

conference paper