{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,12]],"date-time":"2026-04-12T08:08:06Z","timestamp":1775981286664,"version":"3.50.1"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1010250","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2022,6,30]],"date-time":"2022-06-30T00:00:00Z","timestamp":1656547200000}}],"reference-count":56,"publisher":"Public Library of Science (PLoS)","issue":"6","license":[{"start":{"date-parts":[[2022,6,17]],"date-time":"2022-06-17T00:00:00Z","timestamp":1655424000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"crossref","award":["178316478-A1"],"award-info":[{"award-number":["178316478-A1"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"crossref","award":["TRR169\/A2"],"award-info":[{"award-number":["TRR169\/A2"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Bernstein Network","award":["SMARTSTART"],"award-info":[{"award-number":["SMARTSTART"]}]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"crossref","award":["178316478-A1"],"award-info":[{"award-number":["178316478-A1"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"crossref","award":["TRR169\/A2"],"award-info":[{"award-number":["TRR169\/A2"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"crossref","award":["178316478-A1; TRR169\/A2; SFB 1461\/A4; SPP 2041\/HI 1286\/7-1, HI 1286\/6-1"],"award-info":[{"award-number":["178316478-A1; TRR169\/A2; SFB 1461\/A4; SPP 2041\/HI 1286\/7-1, HI 1286\/6-1"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"crossref"}]},{"name":"The Human Brain Project","award":["SGA2, SGA3"],"award-info":[{"award-number":["SGA2, SGA3"]}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>Lesion inference analysis is a fundamental approach for characterizing the causal contributions of neural elements to brain function. This approach has gained new prominence through the arrival of modern perturbation techniques with unprecedented levels of spatiotemporal precision. While inferences drawn from brain perturbations are conceptually powerful, they face methodological difficulties. Particularly, they are challenged to disentangle the true causal contributions of the involved elements, since often functions arise from coalitions of distributed, interacting elements, and localized perturbations have unknown global consequences. To elucidate these limitations, we systematically and exhaustively lesioned a small artificial neural network (ANN) playing a classic arcade game. We determined the functional contributions of all nodes and links, contrasting results from sequential single-element perturbations with simultaneous perturbations of multiple elements. We found that lesioning individual elements, one at a time, produced biased results. By contrast, multi-site lesion analysis captured crucial details that were missed by single-site lesions. We conclude that even small and seemingly simple ANNs show surprising complexity that needs to be addressed by multi-lesioning for a coherent causal characterization.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1010250","type":"journal-article","created":{"date-parts":[[2022,6,17]],"date-time":"2022-06-17T13:40:02Z","timestamp":1655473202000},"page":"e1010250","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":18,"title":["Systematic perturbation of an artificial neural network: A step towards quantifying causal contributions in the brain"],"prefix":"10.1371","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0615-1777","authenticated-orcid":true,"given":"Kayson","family":"Fakhar","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2129-8910","authenticated-orcid":true,"given":"Claus C.","family":"Hilgetag","sequence":"additional","affiliation":[]}],"member":"340","published-online":{"date-parts":[[2022,6,17]]},"reference":[{"key":"pcbi.1010250.ref001","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1016\/j.tics.2015.01.007","article-title":"The unsolved problems of neuroscience","volume":"19","author":"R. Adolphs","year":"2015","journal-title":"Trends Cogn Sci"},{"key":"pcbi.1010250.ref002","volume-title":"Brain, Vision, Memory: Tales in the History of Neuroscience","author":"CG Gross","year":"1999"},{"key":"pcbi.1010250.ref003","doi-asserted-by":"crossref","first-page":"2522","DOI":"10.1093\/brain\/awu164","article-title":"Human brain lesion-deficit inference remapped","volume":"137","author":"YH Mah","year":"2014","journal-title":"Brain"},{"key":"pcbi.1010250.ref004","doi-asserted-by":"crossref","first-page":"1151","DOI":"10.1016\/j.neuron.2016.05.014","article-title":"Human Lesion Studies in the 21st Century","volume":"90","author":"R. Adolphs","year":"2016","journal-title":"Neuron"},{"key":"pcbi.1010250.ref005","doi-asserted-by":"crossref","first-page":"3293","DOI":"10.1523\/JNEUROSCI.4399-14.2015","article-title":"Granger Causality Analysis in Neuroscience and Neuroimaging","volume":"35","author":"AK Seth","year":"2015","journal-title":"Journal of Neuroscience"},{"key":"pcbi.1010250.ref006","volume-title":"Making Things Happen: A Theory of Causal Explanation","author":"J. Woodward","year":"2005"},{"key":"pcbi.1010250.ref007","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1214\/09-SS057","article-title":"Causal inference in statistics: A Primer","volume":"3","author":"J Pearl","year":"2009","journal-title":"Stat Surv"},{"key":"pcbi.1010250.ref008","doi-asserted-by":"crossref","first-page":"4812","DOI":"10.1093\/cercor\/bhv173","article-title":"From phineas gage and monsieur leborgne to H.M.: Revisiting disconnection syndromes","volume":"25","author":"M Thiebaut De Schotten","year":"2015","journal-title":"Cereb Cortex"},{"key":"pcbi.1010250.ref009","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1016\/j.neuron.2011.06.004","article-title":"Optogenetics in Neural Systems","author":"O Yizhar","year":"2011","journal-title":"Neuron"},{"key":"pcbi.1010250.ref010","doi-asserted-by":"crossref","first-page":"683","DOI":"10.1016\/j.neuron.2016.01.040","article-title":"DREADDs for Neuroscientists","author":"BL Roth","year":"2016","journal-title":"Neuron"},{"key":"pcbi.1010250.ref011","first-page":"239","volume-title":"The Proceedings of the social statistics section of the American Statistical Association","author":"DB Rubin","year":"1975"},{"key":"pcbi.1010250.ref012","doi-asserted-by":"crossref","first-page":"1010","DOI":"10.1016\/j.cortex.2010.11.007","article-title":"The quest for the \u201ccritical lesion site\u201d in cognitive deficits: Problems and perspectives","volume":"47","author":"P. Bartolomeo","year":"2011","journal-title":"Cortex"},{"key":"pcbi.1010250.ref013","doi-asserted-by":"crossref","first-page":"134","DOI":"10.1016\/j.neuropsychologia.2017.09.007","article-title":"The dimensionalities of lesion-deficit mapping","volume":"115","author":"T Xu","year":"2018","journal-title":"Neuropsychologia"},{"key":"pcbi.1010250.ref014","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1098\/rstb.2000.0555","article-title":"On imputing function to structure from the behavioural effects of brain lesions","volume":"355","author":"MP Young","year":"2000","journal-title":"Philos Trans R Soc Lond B Biol Sci"},{"key":"pcbi.1010250.ref015","doi-asserted-by":"crossref","first-page":"1342","DOI":"10.1016\/j.neuroimage.2012.06.007","article-title":"Modeling the outcome of structural disconnection on resting-state functional connectivity","volume":"62","author":"J Cabral","year":"2012","journal-title":"Neuroimage"},{"key":"pcbi.1010250.ref016","doi-asserted-by":"crossref","first-page":"2408","DOI":"10.1093\/brain\/awu101","article-title":"Diaschisis: past, present, future","volume":"137","author":"E Carrera","year":"2014","journal-title":"Brain"},{"key":"pcbi.1010250.ref017","doi-asserted-by":"crossref","first-page":"e1005268","DOI":"10.1371\/journal.pcbi.1005268","article-title":"Could a Neuroscientist Understand a Microprocessor?","volume":"13","author":"E Jonas","year":"2017","journal-title":"PLoS Comput Biol."},{"key":"pcbi.1010250.ref018","doi-asserted-by":"crossref","first-page":"1544","DOI":"10.1126\/science.153.3743.1544","article-title":"Interaction of cortex and superior colliculus in mediation of visually guided behavior in the cat","volume":"153","author":"JM Sprague","year":"1966","journal-title":"Science"},{"key":"pcbi.1010250.ref019","first-page":"10","article-title":"Perturbation-driven paradoxical facilitation of visuo-spatial function: Revisiting the \u2018Sprague effect.\u2019","volume":"2","author":"A Valero-Cabr\u00e9","year":"2019","journal-title":"Cortex"},{"key":"pcbi.1010250.ref020","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511978098","author":"N Kapur","year":"2011","journal-title":"The paradoxical brain"},{"key":"pcbi.1010250.ref021","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1016\/j.neucom.2004.01.046","article-title":"Causal localization of neural function: the Shapley value method","volume":"58\u201360","author":"A Keinan","year":"2004","journal-title":"Neurocomputing"},{"key":"pcbi.1010250.ref022","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1162\/artl.2006.12.3.333","article-title":"Axiomatic scalable neurocontroller analysis via the Shapley value","volume":"12","author":"A Keinan","year":"2006","journal-title":"Artif Life"},{"key":"pcbi.1010250.ref023","first-page":"307","volume-title":"Contributions to the theory of games","author":"LS Shapley","year":"1953"},{"key":"pcbi.1010250.ref024","doi-asserted-by":"crossref","first-page":"3454","DOI":"10.1002\/hbm.23601","article-title":"Game theoretical mapping of causal interactions underlying visuo-spatial attention in the human brain based on stroke lesions","volume":"3471","author":"MN Toba","year":"2017","journal-title":"Hum Brain Mapp"},{"key":"pcbi.1010250.ref025","doi-asserted-by":"crossref","first-page":"1887","DOI":"10.1162\/0899766041336387","article-title":"Fair attribution of functional contribution in artificial and biological networks","volume":"16","author":"A Keinan","year":"2004","journal-title":"Neural Comput"},{"key":"pcbi.1010250.ref026","first-page":"1","author":"G Brockman","year":"2016","journal-title":"OpenAI Gym"},{"key":"pcbi.1010250.ref027","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"V Mnih","year":"2015","journal-title":"Nature"},{"key":"pcbi.1010250.ref028","first-page":"355","volume":"6","author":"M Hausknecht","year":"2014","journal-title":"A Neuroevolution Approach to General Atari Game Playing"},{"key":"pcbi.1010250.ref029","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1162\/106365602320169811","article-title":"Evolving Neural Networks through Augmenting Topologies","volume":"10","author":"KO Stanley","year":"2002","journal-title":"Evol Comput."},{"key":"pcbi.1010250.ref030","first-page":"11","article-title":"Brain disconnections link structural connectivity with function and behaviour","author":"M Thiebaut de Schotten","year":"2020","journal-title":"Nat Commun"},{"key":"pcbi.1010250.ref031","doi-asserted-by":"crossref","first-page":"3063","DOI":"10.1093\/brain\/aww194","article-title":"Brain networks under attack: robustness properties and the impact of lesions","volume":"139","author":"H Aerts","year":"2016","journal-title":"Brain"},{"key":"pcbi.1010250.ref032","doi-asserted-by":"crossref","first-page":"e1000408","DOI":"10.1371\/journal.pcbi.1000408","article-title":"Modeling the impact of lesions in the human brain","volume":"5","author":"J Alstott","year":"2009","journal-title":"PLoS Comput Biol"},{"key":"pcbi.1010250.ref033","doi-asserted-by":"crossref","first-page":"13982","DOI":"10.1073\/pnas.1305062110","article-title":"Causal effect of disconnection lesions on interhemispheric functional connectivity in rhesus monkeys","volume":"110","author":"JX O\u2019Reilly","year":"2013","journal-title":"Proc Natl Acad Sci U S A"},{"key":"pcbi.1010250.ref034","doi-asserted-by":"crossref","first-page":"20210110","DOI":"10.1098\/rspa.2021.0110","article-title":"Bits and pieces: understanding information decomposition from part-whole relationships and formal logic","volume":"477","author":"AJ Gutknecht","year":"2021","journal-title":"Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences"},{"key":"pcbi.1010250.ref035","article-title":"Spatial neglect and paradoxical lesion effects in the cat- A model based on midbrain connectivity","author":"CC Hilgetag","year":"2000","journal-title":"Neurocomputing"},{"key":"pcbi.1010250.ref036","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12868-016-0275-6","article-title":"Technical considerations of a game-theoretical approach for lesion symptom mapping","volume":"17","author":"M Zavaglia","year":"2016","journal-title":"BMC Neurosci"},{"key":"pcbi.1010250.ref037","doi-asserted-by":"crossref","first-page":"fcab204","DOI":"10.1093\/braincomms\/fcab204","article-title":"Game-theoretical mapping of fundamental brain functions based on lesion deficits in acute stroke","volume":"3","author":"C Malherbe","year":"2021","journal-title":"Brain Commun"},{"key":"pcbi.1010250.ref038","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1142\/S0219525908001465","article-title":"INFORMATION FLOWS IN CAUSAL NETWORKS","author":"N Ay","year":"2008","journal-title":"Advances in Complex Systems"},{"key":"pcbi.1010250.ref039","doi-asserted-by":"crossref","first-page":"2123","DOI":"10.1098\/rsta.2003.1253","article-title":"Controlled analysis of neurocontrollers with informational lesioning","author":"A Keinan","year":"2003","journal-title":"Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences"},{"key":"pcbi.1010250.ref040","doi-asserted-by":"crossref","first-page":"1673","DOI":"10.1016\/j.artint.2008.05.003","article-title":"A linear approximation method for the Shapley value","volume":"172","author":"SS Fatima","year":"2008","journal-title":"Artif Intell"},{"key":"pcbi.1010250.ref041","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s13278-017-0480-z","article-title":"A new approximation method for the Shapley value applied to the WTC 9\/11 terrorist attack","volume":"8","author":"T van Campen","year":"2018","journal-title":"Social Network Analysis and Mining"},{"key":"pcbi.1010250.ref042","article-title":"Neuron Shapley: Discovering the responsible neurons","author":"A Ghorbani","year":"2020","journal-title":"arXiv"},{"key":"pcbi.1010250.ref043","article-title":"Revisiting \u2018brain modes\u2019 in a new computational era: approaches for the characterization of brain-behavioural associations","author":"MN Toba","year":"2019","journal-title":"Brain"},{"key":"pcbi.1010250.ref044","doi-asserted-by":"crossref","first-page":"e39","DOI":"10.1093\/brain\/awaa093","article-title":"Reply: Inhibition between human brain areas or methodological artefact?","volume":"143","author":"MN Toba","year":"2020","journal-title":"Brain"},{"key":"pcbi.1010250.ref045","doi-asserted-by":"crossref","DOI":"10.1016\/S0079-6123(08)63071-X","article-title":"Inter-hemispheric competition of sub-cortical structures is a crucial mechanism in paradoxical lesion effects and spatial neglect","author":"CC Hilgetag","year":"1999","journal-title":"Prog Brain Res"},{"key":"pcbi.1010250.ref046","doi-asserted-by":"crossref","first-page":"212","DOI":"10.1038\/s41586-019-0919-7","article-title":"Discrete attractor dynamics underlies persistent activity in the frontal cortex","volume":"566","author":"HK Inagaki","year":"2019","journal-title":"Nature"},{"key":"pcbi.1010250.ref047","doi-asserted-by":"crossref","first-page":"132","DOI":"10.1038\/nrn729","article-title":"Evolutionary autonomous agents: A neuroscience perspective","volume":"3","author":"E. Ruppin","year":"2002","journal-title":"Nat Rev Neurosci"},{"key":"pcbi.1010250.ref048","doi-asserted-by":"crossref","first-page":"470","DOI":"10.1177\/1059712310379923","article-title":"Evolving plastic neural networks with novelty search","volume":"18","author":"S Risi","year":"2010","journal-title":"Adapt Behav"},{"key":"pcbi.1010250.ref049","author":"LE Su","year":"2020","journal-title":"Learning function from structure in neuromorphic networks"},{"key":"pcbi.1010250.ref050","first-page":"1","author":"F Damicelli","year":"2021","journal-title":"Brain Connectivity meets Reservoir Computing"},{"key":"pcbi.1010250.ref051","doi-asserted-by":"crossref","first-page":"608","DOI":"10.1016\/j.neunet.2021.07.011","article-title":"Bio-instantiated recurrent neural networks: Integrating neurobiology-based network topology in artificial networks","volume":"142","author":"A Goulas","year":"2021","journal-title":"Neural Netw"},{"key":"pcbi.1010250.ref052","article-title":"Bringing Anatomical Information into Neuronal Network Models","author":"SJ van Albada","year":"2020","journal-title":"arXiv"},{"key":"pcbi.1010250.ref053","first-page":"201905544","article-title":"Recurrence is required to capture the representational dynamics of the human visual system","author":"TC Kietzmann","year":"2019","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"pcbi.1010250.ref054","doi-asserted-by":"crossref","first-page":"1202","DOI":"10.1126\/science.1225266","article-title":"A Large-Scale Model of the Functioning Brain","volume":"338","author":"C Eliasmith","year":"2012","journal-title":"Science"},{"key":"pcbi.1010250.ref055","unstructured":"Silva AMMKCGMCF da. NEAT-Python. Available: https:\/\/github.com\/CodeReclaimers\/neat-python"},{"key":"pcbi.1010250.ref056","unstructured":"Chollet F. keras. 2015."}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1010250","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2022,6,30]],"date-time":"2022-06-30T00:00:00Z","timestamp":1656547200000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1010250","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,23]],"date-time":"2023-11-23T05:31:56Z","timestamp":1700717516000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1010250"}},"subtitle":[],"editor":[{"given":"Daniele","family":"Marinazzo","sequence":"first","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2022,6,17]]},"references-count":56,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2022,6,17]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1010250","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2021.11.04.467251","asserted-by":"object"}]},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,17]]}}}