Towards interactive multisensory data representations

Despite the availability of a broad range of newly developed multisensory displays and interaction techniques, multisensory interactive data representations are still not widely used. We argue that for complex information analysis multisensory data representations and multimodal interactivity are essential. By leveraging the benefits of the...

Benefits of multisensory presentation on perception, memory and navigation [De voordelen van multisensorische presentatie op perceptie, geheugen en navigatie]

The overarching purpose of this thesis was to investigate the beneficial effects of congruent visual, auditory, and tactile presentation on navigation performance. Overall, the results show that multisensory presentation is advantageous. An important aspect of this finding is that multisensory performance was always equal to or higher than the...

Does bimodal stimulus presentation increase ERP components usable in BCIs?

Event-related potential (ERP)-based brain–computer interfaces (BCIs) employ differences in brain responses to attended and ignored stimuli. Typically, visual stimuli are used. Tactile stimuli have recently been suggested as a gaze-independent alternative. Bimodal stimuli could evoke additional brain activity due to multisensory integration which...

The ten rules of touch: Guidelines for social agents and robots that can touch

Touching is essential in interpersonal and affective communication, yet most social agents lack the capability to touch the user. In this paper we show the credibility of three premises that make the case that providing touch capability to social robots will increase their effectiveness in communicating emotions, building trust and achieving...

Integration of visual and inertial cues in perceived heading of self-motion

In the present study, we investigated whether the perception of heading of linear self-motion can be explained by Maximum Likelihood Integration (MLI) of visual and non-visual sensory cues. MLI predicts smaller variance for multisensory judgments compared to unisensory judgments. Nine participants were exposed to visual, inertial, or visual...

Attention and the multiple stages of multisensory integration: A review of audiovisual studies

Multisensory integration and crossmodal attention have a large impact on how we perceive the world. Therefore, it is important to know under what circumstances these processes take place and how they affect our performance. So far, no consensus has been reached on whether multisensory integration and crossmodal attention operate independently...

Counting visual and tactile events: The effect of attention on multisensory integration

Irrelevant events in one sensory modality can influence the number of events that are perceived in another modality. Previously, the underlying process of sensory integration was studied in conditions in which participants knew a priori which sensory modality was relevant and which was not. Consequently, (bottom-up) sensory interference and (top...

Poke and pop: Tactile-visual synchrony increases visual saliency

The majority of studies investigating interactions between vision and touch have typically explored single events, presenting one object at a time. The present study investigates how tactile-visual interactions affect competition between multiple visual objects in more dynamic cluttered environments. Participants searched for a horizontal or...

Multisensory temporal numerosity judgment

In temporal numerosity judgment, observers systematically underestimate the number of pulses. The strongest underestimations occur when stimuli are presented with a short interstimulus interval (ISI) and are stronger for vision than for audition and touch. We investigated if multisensory presentation leads to a reduction of underestimation....

Human factors issues of tactice displays for military environments

The overall goal of this chapter is to give the reader insights into the human factors issues related to the use of tactile displays. Torso-mounted displays, which are particularly suited for direction and orientation cues, are emphasized. First, perceptual issues relevant to tactile stimulation are discussed. These include issues regarding...

Audiovisual events capture attention: Evidence from temporal order judgments

Is an irrelevant audiovisual event able to guide attention automatically? In Experiments 1 and 2, participants were asked to make a temporal order judgment (TOJ) about which of two dots (left or right) appeared first. In Experiment 3, participants were asked to make a simultaneity judgment (SJ) instead. Such tasks have been shown to be affected...

Pip and pop: Non-spatial auditory signals improve spatial visual search

Searching for an object within a cluttered, continuously changing environment can be a very time-consuming process. The authors show that a simple auditory pip drastically decreases search times for a synchronized visual object that is normally very difficult to find. This effect occurs even though the pip contains no information on the location...

Portable real-time color night vision

We developed a simple and fast lookup-table based method to derive and apply natural daylight colors to multi-band night-time images. The method deploys an optimal color transformation derived from a set of samples taken from a daytime color reference image. The colors in the resulting colorized multiband night-time images closely resemble the...

Method for applying daytime colors to nighttime imagery in realtime

We present a fast and efficient method to derive and apply natural colors to nighttime imagery from multiband sensors. The color mapping is derived from the combination of a multiband image and a corresponding natural color reference image. The mapping optimizes the match between the multiband image and the reference image, and yields a...

Towards cognitive image fusion

The increasing availability and deployment of imaging sensors operating in multiple spectral bands has led to a large research effort in image fusion, resulting in a plethora of pixel-level image fusion algorithms. However, the cognitive aspects of multisensor image fusion have not received much attention in the development of these methods. In...

Tactile displays for navigation and orientation: Perception and behaviour

Perceiving and understanding information of, for example, a visual navigation display may be difficult for people with a visual challenge or in situations where the user's visual sense and cognitive resources are heavily loaded. Developing information presentation schemes that reduce the threat of overloading eyes and mind becomes increasingly...

Recalibration of auditory phoneme perception by lipread and lexical information

In a natural environment, many events have perceptual consequences in more than one modality. For example, when we hear someone speaking, there is not only auditory information about what is said, but also visual information about where the sound comes from, and phonetic information that can be lipread. It is known that our perceptual system...

Detection of dim point targets in cluttered maritime backgrounds through multisensor image fusion

Multispectral IR imaging techniques are frequently deployed in maritime operations, for instance to detect floating mines or to find small dinghies and swimmers during search and rescue operations. However, maritime backgrounds usually contain a large amount of clutter that severely hampers the detection of dim point targets. Here we present a...