Multisensory integration effects caused by cross-modal mental imagery
The existence of perceptually robust multisensory interactions, such as the ventriloquism and McGurk effects, has been well established in behavioral studies, and neuroimaging studies have further shown that multisensory processing of stimuli takes place even in primary sensory cortical areas. There is also evidence suggesting that mental imagery, such as an imagined sound of a hammer seen to hit an anvil in a silent movie, modulates processing in sensory cortical areas. What has remained less explored is the extent that imaginary visual stimuli influence processing of real auditory stimuli and vice versa.
In their recently published study, Berger and Ehrsson (2013) conducted a series of behavioral experiments where they tested whether imagined stimuli cause well-known multisensory illusory percepts similarly as real stimuli. They first tested the effects of an imagined sound of collision on cross-bounce illusion, followed by testing the effects of an imagined visual stimulus on ventriloquism effect, and as a third test, the effects of an imagined seen articulation on the so-called McGurk effect. In all three experiments, the authors were able to demonstrate that imagined stimulus causes similar multisensory illusions as real cross-modal stimuli; imagining the sound of a collision gave rise to the cross-bounce illusion, imagining a visual stimulus shifted the perceived location of an auditory stimulus, and auditory imagery of speech stimuli led to a promotion of an illusory speech percept, i.e., in a modified McGurk illusion.
These highly exciting results nicely expand previous findings on multisensory interactions, and provide further evidence for the view that sensory cortices play a pivotal role in generation of mental imagery – even to the extent that visual imagery modulate processing of auditory stimuli and vice versa. It is easy to see that these behavioral results also provide an excellent starting point for further neuroimaging studies investigating the multisensory effects of mental imagery in sensory cortical areas of the brain.
Reference: Berger CC, Ehrsson HH. Mental imagery changes multisensory perception. Current Biology (2013), e-publication ahead of print.