{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,22]],"date-time":"2026-04-22T17:41:54Z","timestamp":1776879714853,"version":"3.51.2"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1012246","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2024,7,17]],"date-time":"2024-07-17T00:00:00Z","timestamp":1721174400000}}],"reference-count":29,"publisher":"Public Library of Science (PLoS)","issue":"7","license":[{"start":{"date-parts":[[2024,7,5]],"date-time":"2024-07-05T00:00:00Z","timestamp":1720137600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100018694","name":"HORIZON EUROPE Marie Sklodowska-Curie Actions","doi-asserted-by":"publisher","award":["945298-ParisRegionFP"],"award-info":[{"award-number":["945298-ParisRegionFP"]}],"id":[{"id":"10.13039\/100018694","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100010663","name":"H2020 European Research Council","doi-asserted-by":"publisher","award":["715980"],"award-info":[{"award-number":["715980"]}],"id":[{"id":"10.13039\/100010663","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>Animals continuously detect information via multiple sensory channels, like vision and hearing, and integrate these signals to realise faster and more accurate decisions; a fundamental neural computation known as multisensory integration. A widespread view of this process is that multimodal neurons linearly fuse information across sensory channels. However, does linear fusion generalise beyond the classical tasks used to explore multisensory integration? Here, we develop novel multisensory tasks, which focus on the underlying statistical relationships between channels, and deploy models at three levels of abstraction: from probabilistic ideal observers to artificial and spiking neural networks. Using these models, we demonstrate that when the information provided by different channels is not independent, linear fusion performs sub-optimally and even fails in extreme cases. This leads us to propose a simple nonlinear algorithm for multisensory integration which is compatible with our current knowledge of multimodal circuits, excels in naturalistic settings and is optimal for a wide class of multisensory tasks. Thus, our work emphasises the role of nonlinear fusion in multisensory integration, and provides testable hypotheses for the field to explore at multiple levels: from single neurons to behaviour.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1012246","type":"journal-article","created":{"date-parts":[[2024,7,5]],"date-time":"2024-07-05T13:47:03Z","timestamp":1720187223000},"page":"e1012246","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":9,"title":["Nonlinear fusion is optimal for a wide class of multisensory tasks"],"prefix":"10.1371","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2428-4605","authenticated-orcid":true,"given":"Marcus","family":"Ghosh","sequence":"first","affiliation":[]},{"given":"Gabriel","family":"B\u00e9na","sequence":"additional","affiliation":[]},{"given":"Volker","family":"Bormuth","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1007-6474","authenticated-orcid":true,"given":"Dan F. M.","family":"Goodman","sequence":"additional","affiliation":[]}],"member":"340","published-online":{"date-parts":[[2024,7,5]]},"reference":[{"key":"pcbi.1012246.ref001","article-title":"Possible principles underlying the transformation of sensory messages","author":"HB Barlow","year":"1961","journal-title":"Sensory communication"},{"key":"pcbi.1012246.ref002","article-title":"Sensory cue integration","author":"J Trommershauser","year":"2011","journal-title":"Oxford University Press"},{"key":"pcbi.1012246.ref003","article-title":"From understanding computation to understanding neural circuitry","author":"D Marr","year":"1976","journal-title":"MIT"},{"key":"pcbi.1012246.ref004","article-title":"Vision: A computational investigation into the human representation and processing of visual information","author":"D Marr","year":"1982","journal-title":"MIT Press"},{"key":"pcbi.1012246.ref005","article-title":"A tutorial on cue combination and signal detection theory: Using changes in sensitivity to evaluate how observers integrate sensory information","author":"PR Jones","year":"2016","journal-title":"Journal of Mathematical Psychology"},{"key":"pcbi.1012246.ref006","article-title":"A theory of interactive parallel processing: New capacity measures and predictions for a response time inequality series","author":"JT Townsend","year":"2004","journal-title":"American Psychological Association"},{"key":"pcbi.1012246.ref007","article-title":"Noise and correlations in parallel perceptual decision making","author":"TU Otto","year":"2012","journal-title":"Current Biology"},{"key":"pcbi.1012246.ref008","article-title":"Neural correlates of reliability-based cue weighting during multisensory integration","author":"CR Fetsch","year":"2012","journal-title":"Nature Neuroscience"},{"key":"pcbi.1012246.ref009","article-title":"Optimal multisensory decision-making in a reaction-time task","author":"J Drugowitsch","year":"2014","journal-title":"eLife"},{"key":"pcbi.1012246.ref010","doi-asserted-by":"crossref","DOI":"10.1016\/j.neuron.2019.08.038","article-title":"Neural correlates of optimal multisensory decision making under time-varying reliabilities with an invariant linear probabilistic population code","author":"H Hou","year":"2019","journal-title":"Neuron"},{"key":"pcbi.1012246.ref011","article-title":"Mouse frontal cortex mediates additive multisensory decisions","author":"P Coen","year":"2023","journal-title":"Neuron"},{"key":"pcbi.1012246.ref012","article-title":"Where are multisensory signals combined for perceptual decision-making?","author":"JK Bizley","year":"2016","journal-title":"Current Opinion in Neurobiology"},{"key":"pcbi.1012246.ref013","article-title":"Attention bottlenecks for multimodal fusion","author":"A Nagrani","year":"2021","journal-title":"Advances in Neural Information Processing Systems"},{"key":"pcbi.1012246.ref014","article-title":"Evaluating the operations underlying multisensory integration in the cat superior colliculus","author":"TR Stanford","year":"2005","journal-title":"Journal of Neuroscience"},{"key":"pcbi.1012246.ref015","article-title":"Multisensory integration: current issues from the perspective of the single neuron","author":"BE Stein","year":"2008","journal-title":"Nature Reviews Neuroscience"},{"key":"pcbi.1012246.ref016","article-title":"Superspike: Supervised learning in multilayer spiking neural networks","author":"F Zenke","year":"2018","journal-title":"Neural Computation"},{"key":"pcbi.1012246.ref017","article-title":"Bridging the gap between theories of sensory cue integration and the physiology of multisensory neurons","author":"CR Fetsch","year":"2013","journal-title":"Nature Reviews Neuroscience"},{"key":"pcbi.1012246.ref018","article-title":"A normalization model of multisensory integration","author":"T Ohshiro","year":"2011","journal-title":"Nature Neuroscience"},{"key":"pcbi.1012246.ref019","article-title":"Modelling firing regularity in the ventral cochlear nucleus: Mechanisms, and effects of stimulus level and synaptopathy","author":"DFM Goodman","year":"2018","journal-title":"Hearing Research"},{"key":"pcbi.1012246.ref020","article-title":"Dynamics of the firing probability of noisy integrate-and-fire neurons","author":"N Fourcaud","year":"2002","journal-title":"Neural Computation"},{"key":"pcbi.1012246.ref021","article-title":"Systematic perturbation of an artificial neural network: A step towards quantifying causal contributions in the brain","author":"K Fakhar","year":"2022","journal-title":"PLOS Computational Biology"},{"key":"pcbi.1012246.ref022","article-title":"Correlation detection as a general mechanism for multisensory integration","author":"CV Parise","year":"2016","journal-title":"Nature Communications"},{"key":"pcbi.1012246.ref023","article-title":"A multilevel multimodal circuit enhances action selection in Drosophila","author":"T Ohyama","year":"2015","journal-title":"Nature"},{"key":"pcbi.1012246.ref024","article-title":"Causal inference in multisensory perception","author":"KP K\u00f6rding","year":"2007","journal-title":"PLoS ONE"},{"key":"pcbi.1012246.ref025","article-title":"Complementary congruent and opposite neurons achieve concurrent multisensory integration and segregation","author":"WH Zhang","year":"2019","journal-title":"eLife"},{"key":"pcbi.1012246.ref026","article-title":"How multisensory neurons solve causal inference","author":"R Rideaux","year":"2021","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"pcbi.1012246.ref027","article-title":"Adam: A method for stochastic optimization","author":"DP Kingma","year":"2014","journal-title":"arXiv"},{"key":"pcbi.1012246.ref028","article-title":"Scikit-learn: Machine learning in python","author":"F Pedregosa","year":"2012","journal-title":"arXiv"},{"key":"pcbi.1012246.ref029","article-title":"Causal localization of neural function: the Shapley value method","author":"A Keinan","year":"2004","journal-title":"Neurocomputing"}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1012246","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2024,7,17]],"date-time":"2024-07-17T00:00:00Z","timestamp":1721174400000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1012246","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,17]],"date-time":"2024-07-17T13:39:48Z","timestamp":1721223588000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1012246"}},"subtitle":[],"editor":[{"given":"Tianming","family":"Yang","sequence":"first","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2024,7,5]]},"references-count":29,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2024,7,5]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1012246","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2023.07.24.550311","asserted-by":"object"}]},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,5]]}}}