In a natural environment, many events have perceptual consequences in more than one modality. For example, when we hear someone speaking, there is not only auditory information about what is said, but also visual information about where the sound comes from, and phonetic information that can be lipread. It is known that our perceptual system integrates these various kinds of information into a coherent representation of the world. In the current project, we try to obtain a detailed insight in the mechanisms at the basis of crossmodal perception. More in particular, we investigate how spatial information from the different senses (i.e., hearing, vision, and touch) is integrated. It is known that when visual and auditory stimuli such as tone bursts and light flashes are presented synchronously but in different locations, the apparent location of the auditory stimulus is often shifted in the direction of the visual stimulus. This is known as the ventriloquist effect. Apart from this immediate bias, prolonged exposure to synchronous but spatially discordant sound-light pairs may result in recalibration of auditory space as demonstrated by the occurrence of aftereffects. In the current project, we address five inter-related topics concerning these two phenomena. These are: 1) whether higher-order knowledge affects the ventriloquist effect; 2) how crossmodally induced aftereffects built up, how they generalize across space, frequencies, and spectral composition, and how they dissipate; 3) whether ventriloquism affects where auditory attention is directed; 4) whether ventriloquism affects components in the EEG usually considered to reflect 'auditory' processes (i.e., a Mismatch Negativity, MMN, with ventriloquized sounds); and 5) whether spatially conflicting audio-tactile stimulus-pairs induce immediate bias and aftereffects. Together, these projects will give a detailed insight into how humans integrate spatial information from different senses, and they may provide a paradigm case for understanding the integrative functions of the human brain.
Interzintuiglijke integratie van ruimtelijke informatie