Gesture recognition with a low power FMCW radar and a deep convolutional neural network

Gesture recognition with radar enables remote control of consumer devices such as audio equipment, television sets and gaming consoles. In this paper, experimental results of hand gesture recognition with a low power FMCW radar and a deep convolutional neural network (CNN) are presented. The FMCW radar operates in the 24 GHz ISM frequency band...

Dataflow-based modeling and performance analysis for online gesture recognition

Cyber-Physical Systems (CPS) are tightly coupled with the environment, and therefore it is important that interactions with the surroundings like Human-Computer-Interactions are performed very responsive. Since CPS are often embedded without traditional input devices, like in medical or automotive contexts, gesture recognition approaches are...

Recognizing Stress Using Semantics and Modulation of Speech and Gestures

This paper investigates how speech and gestures convey stress, and how they can be used for automatic stress recognition. As a first step, we look into how humans use speech and gestures to convey stress. In particular, for both speech and gestures, we distinguish between stress conveyed by the intended semantic message (e.g. spoken words for...

An audio-visual dataset of human-human interactions in stressful situations

Stressful situations are likely to occur at human operated service desks, as well as at human-computer interfaces used in public domain. Automatic surveillance can help notifying when extra assistance is needed. Human communication is inherently multimodal e.g. speech, gestures, facial expressions. It is expected that automatic surveillance...

Towards a format-agnostic approach for production, delivery and rendering of immersive media

The media industry is currently being pulled in the often-opposing directions of increased realism (high resolution, stereoscopic, large screen) and personalization (selection and control of content, availability on many devices). We investigate the feasibility of an end-to-end format-agnostic approach to support both these trends. In this paper...

Subjective usability of speech, touch and gesture in a heterogeneous multi-display environment

Several interaction techniques have been proposed to enable transfer of information between different displays in heterogeneous multi-display environments. However, it is not clear whether subjective user preference for these different techniques depends on the nature of the displays between which information is transferred. We explore...

Format-agnostic Approach for Production, Delivery and Rendering of Immersive Media

The media industry is currently being pulled in the often-opposing directions of increased realism (high resolution, stereoscopic, large screen) and personalisation (selection and control of content, availability on many devices). A capture, production, delivery and rendering system capable of supporting both these trends is being developed by...

Seeing Signs: On the appearance of manual movements in gestures

This dissertation presents the results of a series of studies on the appearance of manual movements in gestures. The main goal of this research is to increase our understanding of how humans perceive signs and other gestures. Generated insights from human perception may aid the development of technology for recognizing gestures and sign language...

Automatic discrimination between laughter and speech

Emotions can be recognized by audible paralinguistic cues in speech. By detecting these paralinguistic cues that can consist of laughter, a trembling voice, coughs, changes in the intonation contour etc., information about the speaker’s state and emotion can be revealed. This paper describes the development of a gender-independent laugh detector...

Automatic detection of laughter

In the context of detecting ‘paralinguistic events’ with the aim to make classification of the speaker’s emotional state possible, a detector was developed for one of the most obvious ‘paralinguistic events’, namely laughter. Gaussian Mixture Models were trained with Perceptual Linear Prediction features, pitch&energy, pitch&voicing and...

Perceptual evaluation of different image fusion schemes

Human scene recognition performance was tested with images of night-time outdoor scenes. The scenes were registered both with a dual band (visual and near infrared) image intensified low-light CCD camera (DII) and with a thermal middle wavelength band (3–5 mm) infrared (IR) camera. Fused imagery was produced through a grayscale pyramid image...