{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T02:56:03Z","timestamp":1787021763276,"version":"build-2736575974"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1011074","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2024,4,2]],"date-time":"2024-04-02T00:00:00Z","timestamp":1712016000000}}],"reference-count":34,"publisher":"Public Library of Science (PLoS)","issue":"3","license":[{"start":{"date-parts":[[2024,3,13]],"date-time":"2024-03-13T00:00:00Z","timestamp":1710288000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001691","name":"Japan Society for the Promotion of Science","doi-asserted-by":"publisher","award":["JP16H06316, JP16H06463, JP21H04813, JP23H04233, JP23KK0132"],"award-info":[{"award-number":["JP16H06316, JP16H06463, JP21H04813, JP23H04233, JP23KK0132"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100009619","name":"Japan Agency for Medical Research and Development","doi-asserted-by":"publisher","award":["JP21wm0425011"],"award-info":[{"award-number":["JP21wm0425011"]}],"id":[{"id":"10.13039\/100009619","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100020963","name":"Moonshot Research and Development Program","doi-asserted-by":"publisher","award":["JPMJMS2299, JPMJMS229B"],"award-info":[{"award-number":["JPMJMS2299, JPMJMS229B"]}],"id":[{"id":"10.13039\/501100020963","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100009438","name":"National Center of Neurology and Psychiatry","doi-asserted-by":"publisher","award":["30-9"],"award-info":[{"award-number":["30-9"]}],"id":[{"id":"10.13039\/501100009438","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100007449","name":"Takeda Science Foundation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100007449","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100016289","name":"Taiju Life Social Welfare Foundation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100016289","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001691","name":"Japan Society for the Promotion of Science","doi-asserted-by":"publisher","award":["JP19K16886, JP23K14673, JP23H04138"],"award-info":[{"award-number":["JP19K16886, JP23K14673, JP23H04138"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>Deep learning is a powerful tool for neural decoding, broadly applied to systems neuroscience and clinical studies. Interpretable and transparent models that can explain neural decoding for intended behaviors are crucial to identifying essential features of deep learning decoders in brain activity. In this study, we examine the performance of deep learning to classify mouse behavioral states from mesoscopic cortex-wide calcium imaging data. Our convolutional neural network (CNN)-based end-to-end decoder combined with recurrent neural network (RNN) classifies the behavioral states with high accuracy and robustness to individual differences on temporal scales of sub-seconds. Using the CNN-RNN decoder, we identify that the forelimb and hindlimb areas in the somatosensory cortex significantly contribute to behavioral classification. Our findings imply that the end-to-end approach has the potential to be an interpretable deep learning method with unbiased visualization of critical brain regions.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1011074","type":"journal-article","created":{"date-parts":[[2024,3,13]],"date-time":"2024-03-13T13:45:45Z","timestamp":1710337545000},"page":"e1011074","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":12,"title":["End-to-end deep learning approach to mouse behavior classification from cortex-wide calcium imaging"],"prefix":"10.1371","volume":"20","author":[{"given":"Takehiro","family":"Ajioka","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7171-230X","authenticated-orcid":true,"given":"Nobuhiro","family":"Nakai","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Okito","family":"Yamashita","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7153-266X","authenticated-orcid":true,"given":"Toru","family":"Takumi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2024,3,13]]},"reference":[{"issue":"3","key":"pcbi.1011074.ref001","doi-asserted-by":"crossref","first-page":"031001","DOI":"10.1088\/1741-2552\/ab0ab5","article-title":"Deep learning for electroencephalogram (EEG) classification tasks: a review.","volume":"16","author":"A Craik","year":"2019","journal-title":"Journal of neural engineering"},{"issue":"7553","key":"pcbi.1011074.ref002","first-page":"436","volume":"521","author":"Y LeCun","year":"2015","journal-title":"Deep learning. nature"},{"issue":"2","key":"pcbi.1011074.ref003","doi-asserted-by":"crossref","first-page":"1577","DOI":"10.1093\/bib\/bbaa355","article-title":"Deep learning approaches for neural decoding across architectures and recording modalities","volume":"22","author":"JA Livezey","year":"2021","journal-title":"Briefings in bioinformatics"},{"issue":"7398","key":"pcbi.1011074.ref004","doi-asserted-by":"crossref","first-page":"372","DOI":"10.1038\/nature11076","article-title":"Reach and grasp by people with tetraplegia using a neurally controlled robotic arm","volume":"485","author":"LR Hochberg","year":"2012","journal-title":"Nature"},{"key":"pcbi.1011074.ref005","doi-asserted-by":"crossref","first-page":"555","DOI":"10.3389\/fnins.2018.00555","article-title":"Rapid decoding of hand gestures in electrocorticography using recurrent neural networks","volume":"12","author":"G Pan","year":"2018","journal-title":"Frontiers in neuroscience"},{"issue":"11","key":"pcbi.1011074.ref006","doi-asserted-by":"crossref","first-page":"1669","DOI":"10.1038\/s41591-018-0171-y","article-title":"Meeting brain\u2013computer interface user performance expectations using a deep neural network decoding framework","volume":"24","author":"MA Schwemmer","year":"2018","journal-title":"Nature medicine"},{"key":"pcbi.1011074.ref007","doi-asserted-by":"crossref","first-page":"763","DOI":"10.3389\/fnins.2018.00763","article-title":"A characterization of brain-computer interface performance trade-offs using support vector machines and deep neural networks to decode movement intent.","author":"ND Skomrock","year":"2018","journal-title":"Frontiers in neuroscience."},{"issue":"1","key":"pcbi.1011074.ref008","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1038\/s41592-019-0625-2","article-title":"Simultaneous mesoscopic and two-photon imaging of neuronal activity in cortical circuits","volume":"17","author":"D Barson","year":"2020","journal-title":"Nature methods"},{"issue":"19","key":"pcbi.1011074.ref009","doi-asserted-by":"crossref","first-page":"4160","DOI":"10.1523\/JNEUROSCI.3003-20.2021","article-title":"Characterizing cortex-wide dynamics with wide-field calcium imaging","volume":"41","author":"C Ren","year":"2021","journal-title":"Journal of Neuroscience"},{"key":"pcbi.1011074.ref010","first-page":"2021","article-title":"Behavioral clusters revealed by end-to-end decoding from microendoscopic imaging.","volume":"16","author":"CJ Chang","year":"2021","journal-title":"BioRxiv."},{"key":"pcbi.1011074.ref011","doi-asserted-by":"crossref","first-page":"19","DOI":"10.3389\/fncir.2020.00019","article-title":"A probabilistic framework for decoding behavior from in vivo calcium imaging data","volume":"14","author":"G Etter","year":"2020","journal-title":"Frontiers in neural circuits"},{"issue":"32","key":"pcbi.1011074.ref012","doi-asserted-by":"crossref","first-page":"e2106830119","DOI":"10.1073\/pnas.2106830119","article-title":"Multiple types of navigational information are independently encoded in the population activities of the dentate gyrus neurons","volume":"119","author":"T Murano","year":"2022","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"pcbi.1011074.ref013","doi-asserted-by":"crossref","first-page":"88","DOI":"10.3389\/fncel.2019.00088","article-title":"Prediction of forelimb reach results from motor cortex activities based on calcium imaging and deep learning","volume":"13","author":"C Li","year":"2019","journal-title":"Frontiers in cellular neuroscience"},{"key":"pcbi.1011074.ref014","article-title":"Virtual reality-based real-time imaging reveals abnormal cortical dynamics during behavioral transitions in a mouse model of autism","author":"N Nakai","year":"2023","journal-title":"Cell Reports"},{"issue":"10","key":"pcbi.1011074.ref015","doi-asserted-by":"crossref","first-page":"e0206049","DOI":"10.1371\/journal.pone.0206049","article-title":"A novel attention-based hybrid CNN-RNN architecture for sEMG-based gesture recognition.","volume":"13","author":"Y Hu","year":"2018","journal-title":"PloS one."},{"issue":"3","key":"pcbi.1011074.ref016","doi-asserted-by":"crossref","first-page":"036009","DOI":"10.1088\/1741-2552\/aa9dbe","article-title":"Decoding of finger trajectory from ECoG using deep learning","volume":"15","author":"Z Xie","year":"2018","journal-title":"Journal of neural engineering"},{"issue":"37","key":"pcbi.1011074.ref017","doi-asserted-by":"crossref","DOI":"10.1126\/scirobotics.aay7120","article-title":"XAI\u2014Explainable artificial intelligence","volume":"4","author":"D Gunning","year":"2019","journal-title":"Science robotics"},{"key":"pcbi.1011074.ref018","doi-asserted-by":"crossref","unstructured":"Zeiler MD, Fergus R. Visualizing and understanding convolutional networks. InComputer Vision\u2013ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6\u201312, 2014, Proceedings, Part I 13 2014 (pp. 818\u2013833). Springer International Publishing.","DOI":"10.1007\/978-3-319-10590-1_53"},{"key":"pcbi.1011074.ref019","doi-asserted-by":"crossref","first-page":"e66551","DOI":"10.7554\/eLife.66551","article-title":"Interpreting wide-band neural activity using convolutional neural networks.","volume":"10","author":"M Frey","year":"2021","journal-title":"Elife"},{"key":"pcbi.1011074.ref020","article-title":"A unified approach to interpreting model predictions","volume":"30","author":"SM Lundberg","year":"2017","journal-title":"Advances in neural information processing systems"},{"key":"pcbi.1011074.ref021","doi-asserted-by":"crossref","first-page":"101039","DOI":"10.1016\/j.ecoinf.2019.101039","article-title":"Shapley additive explanations for NO2 forecasting.","volume":"56","author":"MV Garc\u00eda","year":"2020","journal-title":"Ecological Informatics."},{"key":"pcbi.1011074.ref022","first-page":"6105","article-title":"Efficientnet: Rethinking model scaling for convolutional neural networks.","author":"M Tan","year":"2019","journal-title":"InInternational conference on machine learning"},{"issue":"8","key":"pcbi.1011074.ref023","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory.","volume":"9","author":"S Hochreiter","year":"1997","journal-title":"Neural computation"},{"key":"pcbi.1011074.ref024","article-title":"Learning phrase representations using RNN encoder-decoder for statistical machine translation.","author":"K Cho","year":"2014","journal-title":"arXiv preprint arXiv:1406.1078."},{"key":"pcbi.1011074.ref025","article-title":"Adam: A method for stochastic optimization.","author":"DP Kingma","year":"2014","journal-title":"arXiv preprint arXiv:1412.6980"},{"issue":"1","key":"pcbi.1011074.ref026","doi-asserted-by":"crossref","first-page":"4372","DOI":"10.1038\/s41598-020-60932-4","article-title":"An empirical comparison of neural networks and machine learning algorithms for EEG gait decoding.","volume":"10","author":"S Nakagome","year":"2020","journal-title":"Scientific reports."},{"issue":"4","key":"pcbi.1011074.ref027","doi-asserted-by":"crossref","first-page":"703","DOI":"10.1016\/j.neuron.2020.05.022","article-title":"A distributed neural code in the dentate gyrus and in CA1","volume":"107","author":"F Stefanini","year":"2020","journal-title":"Neuron"},{"issue":"2","key":"pcbi.1011074.ref028","doi-asserted-by":"crossref","first-page":"e1006822","DOI":"10.1371\/journal.pcbi.1006822","article-title":"Efficient neural decoding of self-location with a deep recurrent network","volume":"15","author":"A Tampuu","year":"2019","journal-title":"PLoS computational biology"},{"issue":"4","key":"pcbi.1011074.ref029","doi-asserted-by":"crossref","first-page":"575","DOI":"10.1038\/s41593-020-0608-8","article-title":"Machine translation of cortical activity to text with an encoder\u2013decoder framework","volume":"23","author":"JG Makin","year":"2020","journal-title":"Nature neuroscience"},{"issue":"7858","key":"pcbi.1011074.ref030","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1038\/s41586-021-03506-2","article-title":"High-performance brain-to-text communication via handwriting","volume":"593","author":"FR Willett","year":"2021","journal-title":"Nature"},{"issue":"7","key":"pcbi.1011074.ref031","doi-asserted-by":"crossref","DOI":"10.1126\/sciadv.aaw5388","article-title":"The somatosensory cortex receives information about motor output","volume":"5","author":"T Umeda","year":"2019","journal-title":"Science advances"},{"issue":"6544","key":"pcbi.1011074.ref032","doi-asserted-by":"crossref","first-page":"831","DOI":"10.1126\/science.abd0380","article-title":"A brain-computer interface that evokes tactile sensations improves robotic arm control","volume":"372","author":"SN Flesher","year":"2021","journal-title":"Science"},{"key":"pcbi.1011074.ref033","doi-asserted-by":"crossref","first-page":"101730","DOI":"10.1016\/j.bspc.2019.101730","article-title":"Decoding human brain activity with deep learning","volume":"56","author":"X Zheng","year":"2020","journal-title":"Biomedical Signal Processing and Control"},{"key":"pcbi.1011074.ref034","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"Imagenet large scale visual recognition challenge","volume":"115","author":"O Russakovsky","year":"2015","journal-title":"International journal of computer vision"}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1011074","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2024,4,2]],"date-time":"2024-04-02T00:00:00Z","timestamp":1712016000000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1011074","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,2]],"date-time":"2024-04-02T14:45:28Z","timestamp":1712069128000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1011074"}},"subtitle":[],"editor":[{"given":"Joshua","family":"Glaser","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2024,3,13]]},"references-count":34,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,3,13]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1011074","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2023.04.05.535664","asserted-by":"object"}]},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,13]]}}}