Multimodal integration in statistical learning

Evidence from the McGurk illusion

Aaron Mitchell, Morten Hyllekvist Christiansen, Dan Weiss

Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

Resumé

Recent advances in the field of statistical learning have established that learners are able to track regularities of multimodal stimuli, yet it is unknown whether the statistical computations are performed on integrated representations or on separate, unimodal representations. In the present study, we investigated the ability of adults to integrate audio and visual input during statistical learning. We presented learners with a speech stream synchronized with a video of a speaker’s face. In the critical condition, the visual (e.g., /gi/) and auditory (e.g., /mi/) signals were occasionally incongruent, which we predicted would produce the McGurk illusion, resulting in the perception of an audiovisual syllable (e.g., /ni/). In this way, we used the McGurk illusion to manipulate the underlying statistical structure of the speech streams, such that perception of these illusory syllables facilitated participants’ ability to segment the speech stream. Our results therefore demonstrate that participants can integrate audio and visual input to perceive the McGurk illusion during statistical learning. We interpret our findings as support for modality-interactive accounts of statistical learning.
OriginalsprogEngelsk
Artikelnummer407
TidsskriftFrontiers in Psychology
Vol/bind5
Antal sider6
ISSN1664-1078
DOI
StatusUdgivet - 16. maj 2014

Emneord

  • multisensory statistical learning, statistical learning mechanisms, multisensory perception, language acquisition, McGurk illusion, multisensory integration, audiovisual speech perception

Citer dette

Mitchell, Aaron ; Christiansen, Morten Hyllekvist ; Weiss, Dan. / Multimodal integration in statistical learning : Evidence from the McGurk illusion. I: Frontiers in Psychology. 2014 ; Bind 5.
@article{ae04fcdbbd8943248a806f88d98b1959,
title = "Multimodal integration in statistical learning: Evidence from the McGurk illusion",
abstract = "Recent advances in the field of statistical learning have established that learners are able to track regularities of multimodal stimuli, yet it is unknown whether the statistical computations are performed on integrated representations or on separate, unimodal representations. In the present study, we investigated the ability of adults to integrate audio and visual input during statistical learning. We presented learners with a speech stream synchronized with a video of a speaker’s face. In the critical condition, the visual (e.g., /gi/) and auditory (e.g., /mi/) signals were occasionally incongruent, which we predicted would produce the McGurk illusion, resulting in the perception of an audiovisual syllable (e.g., /ni/). In this way, we used the McGurk illusion to manipulate the underlying statistical structure of the speech streams, such that perception of these illusory syllables facilitated participants’ ability to segment the speech stream. Our results therefore demonstrate that participants can integrate audio and visual input to perceive the McGurk illusion during statistical learning. We interpret our findings as support for modality-interactive accounts of statistical learning.",
keywords = "multisensory statistical learning, statistical learning mechanisms, multisensory perception, language acquisition, McGurk illusion, multisensory integration, audiovisual speech perception",
author = "Aaron Mitchell and Christiansen, {Morten Hyllekvist} and Dan Weiss",
year = "2014",
month = "5",
day = "16",
doi = "10.3389/fpsyg.2014.00407",
language = "English",
volume = "5",
journal = "Frontiers in Psychology",
issn = "1664-1078",
publisher = "Frontiers Media S.A.",

}

Multimodal integration in statistical learning : Evidence from the McGurk illusion. / Mitchell, Aaron; Christiansen, Morten Hyllekvist; Weiss, Dan.

I: Frontiers in Psychology, Bind 5, 407, 16.05.2014.

Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

TY - JOUR

T1 - Multimodal integration in statistical learning

T2 - Evidence from the McGurk illusion

AU - Mitchell, Aaron

AU - Christiansen, Morten Hyllekvist

AU - Weiss, Dan

PY - 2014/5/16

Y1 - 2014/5/16

N2 - Recent advances in the field of statistical learning have established that learners are able to track regularities of multimodal stimuli, yet it is unknown whether the statistical computations are performed on integrated representations or on separate, unimodal representations. In the present study, we investigated the ability of adults to integrate audio and visual input during statistical learning. We presented learners with a speech stream synchronized with a video of a speaker’s face. In the critical condition, the visual (e.g., /gi/) and auditory (e.g., /mi/) signals were occasionally incongruent, which we predicted would produce the McGurk illusion, resulting in the perception of an audiovisual syllable (e.g., /ni/). In this way, we used the McGurk illusion to manipulate the underlying statistical structure of the speech streams, such that perception of these illusory syllables facilitated participants’ ability to segment the speech stream. Our results therefore demonstrate that participants can integrate audio and visual input to perceive the McGurk illusion during statistical learning. We interpret our findings as support for modality-interactive accounts of statistical learning.

AB - Recent advances in the field of statistical learning have established that learners are able to track regularities of multimodal stimuli, yet it is unknown whether the statistical computations are performed on integrated representations or on separate, unimodal representations. In the present study, we investigated the ability of adults to integrate audio and visual input during statistical learning. We presented learners with a speech stream synchronized with a video of a speaker’s face. In the critical condition, the visual (e.g., /gi/) and auditory (e.g., /mi/) signals were occasionally incongruent, which we predicted would produce the McGurk illusion, resulting in the perception of an audiovisual syllable (e.g., /ni/). In this way, we used the McGurk illusion to manipulate the underlying statistical structure of the speech streams, such that perception of these illusory syllables facilitated participants’ ability to segment the speech stream. Our results therefore demonstrate that participants can integrate audio and visual input to perceive the McGurk illusion during statistical learning. We interpret our findings as support for modality-interactive accounts of statistical learning.

KW - multisensory statistical learning, statistical learning mechanisms, multisensory perception, language acquisition, McGurk illusion, multisensory integration, audiovisual speech perception

U2 - 10.3389/fpsyg.2014.00407

DO - 10.3389/fpsyg.2014.00407

M3 - Journal article

VL - 5

JO - Frontiers in Psychology

JF - Frontiers in Psychology

SN - 1664-1078

M1 - 407

ER -