Concurrent intramodal learning enhances multisensory responses of symmetric crossmodal learning in robotic audio-visual tracking

Danish Shaikh*, Leon Bodenhagen, Poramate Manoonpong

*Corresponding author for this work

Research output: Contribution to journalJournal articleResearchpeer-review

98 Downloads (Pure)


Tracking an audio-visual target involves integrating spatial cues about target position from both modalities. Such sensory cue integration is a developmental process in the brain involving learning, with neuroplasticity as its underlying mechanism. We present a Hebbian learning-based adaptive neural circuit for multi-modal cue integration. The circuit temporally correlates stimulus cues within each modality via intramodal learning as well as symmetrically across modalities via crossmodal learning to independently update modality-specific neural weights on a sample-by-sample basis. It is realised as a robotic agent that must orient towards a moving audio-visual target. It continuously learns the best possible weights required for a weighted combination of auditory and visual spatial target directional cues that is directly mapped to robot wheel velocities to elicit an orientation response. Visual directional cues are noise-free and continuous but arising from a relatively narrow receptive field while auditory directional cues are noisy and intermittent but arising from a relatively wider receptive field. Comparative trials in simulation demonstrate that concurrent intramodal learning improves both the overall accuracy and precision of the orientation responses of symmetric crossmodal learning. We also demonstrate that symmetric crossmodal learning improves multisensory responses as compared to asymmetric crossmodal learning. The neural circuit also exhibits multisensory effects such as sub-additivity, additivity and super-additivity.
Original languageEnglish
JournalCognitive Systems Research
Pages (from-to)138-153
Publication statusPublished - May 2019


  • crossmodal learning
  • multisensory integration
  • audio-visual tracking
  • biorobotics


Dive into the research topics of 'Concurrent intramodal learning enhances multisensory responses of symmetric crossmodal learning in robotic audio-visual tracking'. Together they form a unique fingerprint.

Cite this