{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T13:44:13Z","timestamp":1780407853049,"version":"3.54.1"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1010792","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2023,1,23]],"date-time":"2023-01-23T00:00:00Z","timestamp":1674432000000}}],"reference-count":41,"publisher":"Public Library of Science (PLoS)","issue":"1","license":[{"start":{"date-parts":[[2023,1,10]],"date-time":"2023-01-10T00:00:00Z","timestamp":1673308800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"HSE Basic Research Program"}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>\n                    Modern well-performing approaches to neural decoding are based on machine learning models such as decision tree ensembles and deep neural networks. The wide range of algorithms that can be utilized to learn from neural spike trains, which are essentially time-series data, results in the need for diverse and challenging benchmarks for neural decoding, similar to the ones in the fields of computer vision and natural language processing. In this work, we propose a spike train classification benchmark, based on open-access neural activity datasets and consisting of several learning tasks such as stimulus type classification, animal\u2019s behavioral state prediction, and neuron type identification. We demonstrate that an approach based on hand-crafted time-series feature engineering establishes a strong baseline performing on par with state-of-the-art deep learning-based models for neural decoding. We release the\n                    <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/github.com\/lzrvch\/spikebench\" xlink:type=\"simple\">code allowing to reproduce the reported results<\/jats:ext-link>\n                    .\n                  <\/jats:p>","DOI":"10.1371\/journal.pcbi.1010792","type":"journal-article","created":{"date-parts":[[2023,1,10]],"date-time":"2023-01-10T14:11:26Z","timestamp":1673359886000},"page":"e1010792","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":7,"title":["Spikebench: An open benchmark for spike train time-series classification"],"prefix":"10.1371","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9521-7832","authenticated-orcid":true,"given":"Ivan","family":"Lazarevich","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8639-1238","authenticated-orcid":true,"given":"Ilya","family":"Prokin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Boris","family":"Gutkin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Victor","family":"Kazantsev","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2023,1,10]]},"reference":[{"key":"pcbi.1010792.ref001","first-page":"061507","article-title":"Suite2p: beyond 10,000 neurons with standard two-photon microscopy","author":"M Pachitariu","year":"2016","journal-title":"Biorxiv"},{"key":"pcbi.1010792.ref002","doi-asserted-by":"crossref","unstructured":"Tsai D, John E, Chari T, Yuste R, Shepard K. High-channel-count, high-density microelectrode array for closed-loop investigation of neuronal networks. In: Engineering in Medicine and Biology Society (EMBC), 2015 37th Annual International Conference of the IEEE. IEEE; 2015. p. 7510\u20137513.","DOI":"10.1109\/EMBC.2015.7320129"},{"key":"pcbi.1010792.ref003","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1016\/j.conb.2018.01.009","article-title":"Challenges and opportunities for large-scale electrophysiology with Neuropixels probes","volume":"50","author":"NA Steinmetz","year":"2018","journal-title":"Current opinion in neurobiology"},{"issue":"S1","key":"pcbi.1010792.ref004","doi-asserted-by":"crossref","first-page":"S6","DOI":"10.1186\/1471-2202-10-S1-S6","article-title":"Crcns. org: a repository of high-quality data sets and tools for computational neuroscience","volume":"10","author":"JL Teeters","year":"2009","journal-title":"BMC Neuroscience"},{"key":"pcbi.1010792.ref005","unstructured":"Glaser JI, Chowdhury RH, Perich MG, Miller LE, Kording KP. Machine learning for neural decoding. arXiv preprint arXiv:170800909. 2017;."},{"issue":"3","key":"pcbi.1010792.ref006","doi-asserted-by":"crossref","first-page":"606","DOI":"10.1007\/s10618-016-0483-9","article-title":"The great time series classification bake off: a review and experimental evaluation of recent algorithmic advances","volume":"31","author":"A Bagnall","year":"2017","journal-title":"Data Mining and Knowledge Discovery"},{"issue":"4","key":"pcbi.1010792.ref007","doi-asserted-by":"crossref","first-page":"917","DOI":"10.1007\/s10618-019-00619-1","article-title":"Deep learning for time series classification: a review","volume":"33","author":"HI Fawaz","year":"2019","journal-title":"Data Mining and Knowledge Discovery"},{"issue":"2","key":"pcbi.1010792.ref008","doi-asserted-by":"crossref","first-page":"1577","DOI":"10.1093\/bib\/bbaa355","article-title":"Deep learning approaches for neural decoding across architectures and recording modalities","volume":"22","author":"JA Livezey","year":"2021","journal-title":"Briefings in bioinformatics"},{"key":"pcbi.1010792.ref009","doi-asserted-by":"crossref","DOI":"10.3389\/fncom.2018.00056","article-title":"Modern Machine Learning as a Benchmark for Fitting Neural Responses","volume":"12","author":"AS Benjamin","year":"2018","journal-title":"Frontiers in computational neuroscience"},{"key":"pcbi.1010792.ref010","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1016\/j.neunet.2018.02.013","article-title":"Multineuron spike train analysis with R-convolution linear combination kernel","volume":"102","author":"T Tezuka","year":"2018","journal-title":"Neural Networks"},{"issue":"6","key":"pcbi.1010792.ref011","doi-asserted-by":"crossref","first-page":"2321","DOI":"10.1523\/JNEUROSCI.2853-10.2011","article-title":"Spike-train communities: finding groups of similar spike trains","volume":"31","author":"MD Humphries","year":"2011","journal-title":"Journal of Neuroscience"},{"issue":"4","key":"pcbi.1010792.ref012","doi-asserted-by":"crossref","first-page":"751","DOI":"10.1162\/089976601300014321","article-title":"A novel spike distance","volume":"13","author":"Mv Rossum","year":"2001","journal-title":"Neural computation"},{"issue":"2","key":"pcbi.1010792.ref013","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1088\/0954-898X_8_2_003","article-title":"Metric-space analysis of spike trains: theory, algorithms and application","volume":"8","author":"JD Victor","year":"1997","journal-title":"Network: computation in neural systems"},{"key":"pcbi.1010792.ref014","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1016\/j.softx.2016.07.006","article-title":"PySpike\u2014A Python library for analyzing spike train synchrony","volume":"5","author":"M Mulansky","year":"2016","journal-title":"SoftwareX"},{"key":"pcbi.1010792.ref015","doi-asserted-by":"crossref","unstructured":"Tezuka T. Spike Train Pattern Discovery Using Interval Structure Alignment. In: International Conference on Neural Information Processing. Springer; 2015. p. 241\u2013249.","DOI":"10.1007\/978-3-319-26535-3_28"},{"key":"pcbi.1010792.ref016","first-page":"407635","article-title":"Non-parametric physiological classification of retinal ganglion cells","author":"J Jouty","year":"2018","journal-title":"bioRxiv"},{"issue":"1","key":"pcbi.1010792.ref017","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13064-014-0028-0","article-title":"Quantitative differences in developmental profiles of spontaneous activity in cortical and hippocampal cultures","volume":"10","author":"P Charlesworth","year":"2015","journal-title":"Neural development"},{"key":"pcbi.1010792.ref018","doi-asserted-by":"crossref","first-page":"12474","DOI":"10.1038\/srep12474","article-title":"Computational classification approach to profile neuron subtypes from brain activity mapping data","volume":"5","author":"M Li","year":"2015","journal-title":"Scientific reports"},{"key":"pcbi.1010792.ref019","first-page":"376863","article-title":"High-density extracellular probes reveal dendritic backpropagation and facilitate neuron classification","author":"X Jia","year":"2018","journal-title":"bioRxiv"},{"issue":"5","key":"pcbi.1010792.ref020","doi-asserted-by":"crossref","first-page":"527","DOI":"10.1016\/j.cels.2017.10.001","article-title":"hctsa: A computational framework for automated time-series phenotyping using massive feature extraction","volume":"5","author":"BD Fulcher","year":"2017","journal-title":"Cell systems"},{"key":"pcbi.1010792.ref021","article-title":"Time Series FeatuRe Extraction on basis of Scalable Hypothesis tests (tsfresh\u2013A Python package)","author":"M Christ","year":"2018","journal-title":"Neurocomputing"},{"key":"pcbi.1010792.ref022","first-page":"1189","article-title":"Greedy function approximation: a gradient boosting machine","author":"JH Friedman","year":"2001","journal-title":"Annals of statistics"},{"key":"pcbi.1010792.ref023","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1016\/j.neunet.2019.04.014","article-title":"Multivariate LSTM-FCNs for time series classification","volume":"116","author":"F Karim","year":"2019","journal-title":"Neural Networks"},{"issue":"1","key":"pcbi.1010792.ref024","doi-asserted-by":"crossref","first-page":"162","DOI":"10.21629\/JSEE.2017.01.18","article-title":"Convolutional neural networks for time series classification","volume":"28","author":"B Zhao","year":"2017","journal-title":"Journal of Systems Engineering and Electronics"},{"issue":"6","key":"pcbi.1010792.ref025","doi-asserted-by":"crossref","first-page":"1936","DOI":"10.1007\/s10618-020-00710-y","article-title":"Inceptiontime: Finding alexnet for time series classification","volume":"34","author":"HI Fawaz","year":"2020","journal-title":"Data Mining and Knowledge Discovery"},{"issue":"6","key":"pcbi.1010792.ref026","doi-asserted-by":"crossref","first-page":"1293","DOI":"10.1109\/JAS.2019.1911747","article-title":"The UCR time series archive","volume":"6","author":"HA Dau","year":"2019","journal-title":"IEEE\/CAA Journal of Automatica Sinica"},{"key":"pcbi.1010792.ref027","doi-asserted-by":"crossref","unstructured":"Zerveas G, Jayaraman S, Patel D, Bhamidipaty A, Eickhoff C. A Transformer-based Framework for Multivariate Time Series Representation Learning. arXiv preprint arXiv:201002803. 2020;.","DOI":"10.1145\/3447548.3467401"},{"key":"pcbi.1010792.ref028","unstructured":"Tang W, Long G, Liu L, Zhou T, Jiang J, Blumenstein M. Rethinking 1d-cnn for time series classification: A stronger baseline. arXiv preprint arXiv:200210061. 2020;."},{"issue":"11","key":"pcbi.1010792.ref029","doi-asserted-by":"crossref","first-page":"e1005148","DOI":"10.1371\/journal.pcbi.1005148","article-title":"Error-robust modes of the retinal population code","volume":"12","author":"JS Prentice","year":"2016","journal-title":"PLoS computational biology"},{"key":"pcbi.1010792.ref030","article-title":"Multi-electrode retinal ganglion cell population spiking data","author":"AR Loback","year":"2016","journal-title":"Dryad Dataset"},{"key":"pcbi.1010792.ref031","article-title":"Multi-unit spiking activity recorded from rat frontal cortex (brain regions mPFC, OFC, ACC, and M2) during wake-sleep episode wherein at least 7 minutes of wake are followed by 20 minutes of sleep","author":"B Watson","year":"2016","journal-title":"CRCNS.org"},{"issue":"4","key":"pcbi.1010792.ref032","doi-asserted-by":"crossref","first-page":"839","DOI":"10.1016\/j.neuron.2016.03.036","article-title":"Network homeostasis and state dynamics of neocortical sleep","volume":"90","author":"BO Watson","year":"2016","journal-title":"Neuron"},{"key":"pcbi.1010792.ref033","article-title":"Dataset: Allen Institute for Brain Science","author":"A Budzillo","year":"2020","journal-title":"Allen Cell Types Database\u2014Mouse Patch-seq"},{"issue":"1","key":"pcbi.1010792.ref034","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1002\/dneu.20853","article-title":"Three groups of interneurons account for nearly 100% of neocortical GABAergic neurons","volume":"71","author":"B Rudy","year":"2011","journal-title":"Developmental neurobiology"},{"key":"pcbi.1010792.ref035","volume-title":"Self-supervised visual feature learning with deep neural networks: A survey","author":"L Jing","year":"2020"},{"key":"pcbi.1010792.ref036","doi-asserted-by":"crossref","unstructured":"Jeni LA, Cohn JF, De La Torre F. Facing imbalanced data\u2013recommendations for the use of performance metrics. In: 2013 Humaine association conference on affective computing and intelligent interaction. IEEE; 2013. p. 245\u2013251.","DOI":"10.1109\/ACII.2013.47"},{"issue":"3","key":"pcbi.1010792.ref037","doi-asserted-by":"crossref","first-page":"849","DOI":"10.1016\/S0031-3203(02)00257-1","article-title":"Strategies for learning in class imbalance problems","volume":"36","author":"R Barandela","year":"2003","journal-title":"Pattern Recognition"},{"issue":"9","key":"pcbi.1010792.ref038","doi-asserted-by":"crossref","first-page":"1263","DOI":"10.1109\/TKDE.2008.239","article-title":"Learning from imbalanced data","volume":"21","author":"H He","year":"2009","journal-title":"IEEE Transactions on knowledge and data engineering"},{"issue":"3","key":"pcbi.1010792.ref039","doi-asserted-by":"crossref","first-page":"e0118432","DOI":"10.1371\/journal.pone.0118432","article-title":"The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets","volume":"10","author":"T Saito","year":"2015","journal-title":"PloS one"},{"key":"pcbi.1010792.ref040","unstructured":"Oguiza I. tsai\u2014A state-of-the-art deep learning library for time series and sequential data; 2020. Github. Available from: https:\/\/github.com\/timeseriesAI\/tsai."},{"issue":"29","key":"pcbi.1010792.ref041","doi-asserted-by":"crossref","first-page":"861","DOI":"10.21105\/joss.00861","article-title":"UMAP: Uniform Manifold Approximation and Projection","volume":"3","author":"L McInnes","year":"2018","journal-title":"The Journal of Open Source Software"}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1010792","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2023,1,23]],"date-time":"2023-01-23T00:00:00Z","timestamp":1674432000000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1010792","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,23]],"date-time":"2023-01-23T13:49:01Z","timestamp":1674481741000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1010792"}},"subtitle":[],"editor":[{"given":"Samuel J.","family":"Gershman","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2023,1,10]]},"references-count":41,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,1,10]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1010792","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2021.03.24.436765","asserted-by":"object"}]},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,10]]}}}