{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,17]],"date-time":"2026-01-17T09:22:12Z","timestamp":1768641732641,"version":"3.49.0"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1010401","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2022,8,18]],"date-time":"2022-08-18T00:00:00Z","timestamp":1660780800000}}],"reference-count":48,"publisher":"Public Library of Science (PLoS)","issue":"8","license":[{"start":{"date-parts":[[2022,8,8]],"date-time":"2022-08-08T00:00:00Z","timestamp":1659916800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"UCSD ECE Department\u2019s Medical Devices and Systems Initiative"},{"name":"UCSD Centers for Human Brain Activity Mapping and Brain Activity Mapping"},{"name":"Frontiers of Innovation Scholars Program"},{"name":"Qualcomm Institute\u2019s Calit2 Strategic Research Opportunities Program"},{"name":"Hellman Fellowship"},{"name":"Altman Clinical and Translational Research Institute"},{"name":"UCSD Office of Research Affairs Center Launch Program"},{"name":"Institute of Engineering in Medicine Graduate Student Fellowship"}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>In analyzing the neural correlates of naturalistic and unstructured behaviors, features of neural activity that are ignored in a trial-based experimental paradigm can be more fully studied and investigated. Here, we analyze neural activity from two patients using electrocorticography (ECoG) and stereo-electroencephalography (sEEG) recordings, and reveal that multiple neural signal characteristics exist that discriminate between unstructured and naturalistic behavioral states such as \u201cengaging in dialogue\u201d and \u201cusing electronics\u201d. Using the high gamma amplitude as an estimate of neuronal firing rate, we demonstrate that behavioral states in a naturalistic setting are discriminable based on long-term mean shifts, variance shifts, and differences in the specific neural activity\u2019s covariance structure. Both the rapid and slow changes in high gamma band activity separate unstructured behavioral states. We also use Gaussian process factor analysis (GPFA) to show the existence of salient spatiotemporal features with variable smoothness in time. Further, we demonstrate that both temporally smooth and stochastic spatiotemporal activity can be used to differentiate unstructured behavioral states. This is the first attempt to elucidate how different neural signal features contain information about behavioral states collected outside the conventional experimental paradigm.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1010401","type":"journal-article","created":{"date-parts":[[2022,8,8]],"date-time":"2022-08-08T17:36:06Z","timestamp":1659980166000},"page":"e1010401","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":5,"title":["Spatiotemporal dynamics of human high gamma discriminate naturalistic behavioral states"],"prefix":"10.1371","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5246-5318","authenticated-orcid":true,"given":"Abdulwahab","family":"Alasfour","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5464-769X","authenticated-orcid":true,"given":"Paolo","family":"Gabriel","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4257-7628","authenticated-orcid":true,"given":"Xi","family":"Jiang","sequence":"additional","affiliation":[]},{"given":"Isaac","family":"Shamie","sequence":"additional","affiliation":[]},{"given":"Lucia","family":"Melloni","sequence":"additional","affiliation":[]},{"given":"Thomas","family":"Thesen","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6199-1870","authenticated-orcid":true,"given":"Patricia","family":"Dugan","sequence":"additional","affiliation":[]},{"given":"Daniel","family":"Friedman","sequence":"additional","affiliation":[]},{"given":"Werner","family":"Doyle","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0044-4632","authenticated-orcid":true,"given":"Orin","family":"Devinsky","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4681-7337","authenticated-orcid":true,"given":"David","family":"Gonda","sequence":"additional","affiliation":[]},{"given":"Shifteh","family":"Sattar","sequence":"additional","affiliation":[]},{"given":"Sonya","family":"Wang","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5926-6872","authenticated-orcid":true,"given":"Eric","family":"Halgren","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0619-2686","authenticated-orcid":true,"given":"Vikash","family":"Gilja","sequence":"additional","affiliation":[]}],"member":"340","published-online":{"date-parts":[[2022,8,8]]},"reference":[{"issue":"9","key":"pcbi.1010401.ref001","doi-asserted-by":"crossref","first-page":"2424","DOI":"10.1523\/JNEUROSCI.3886-06.2007","article-title":"Spectral changes in cortical surface potentials during motor movement","volume":"27","author":"KJ Miller","year":"2007","journal-title":"Journal of Neuroscience"},{"key":"pcbi.1010401.ref002","first-page":"711","article-title":"Broadband changes in the cortical surface potential track activation of functionally diverse neuronal populations","volume":"85","author":"KJ Miller","year":"2014","journal-title":"New Horizons for Neural Oscillations"},{"issue":"7441","key":"pcbi.1010401.ref003","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1038\/nature11911","article-title":"Functional organization of human sensorimotor cortex for speech articulation","volume":"495","author":"KE Bouchard","year":"2013","journal-title":"Nature"},{"issue":"3","key":"pcbi.1010401.ref004","doi-asserted-by":"crossref","DOI":"10.1088\/1741-2552\/ab0c59","article-title":"Speech synthesis from ECoG using densely connected 3D convolutional neural networks","volume":"16","author":"M Angrick","year":"2019","journal-title":"Journal of Neural Engineering"},{"issue":"5","key":"pcbi.1010401.ref005","doi-asserted-by":"crossref","DOI":"10.1088\/1741-2560\/7\/5\/056007","article-title":"Decoding spoken words using local field potentials recorded from the cortical surface","volume":"7","author":"S Kellis","year":"2010","journal-title":"Journal of Neural Engineering"},{"issue":"7753","key":"pcbi.1010401.ref006","doi-asserted-by":"crossref","first-page":"493","DOI":"10.1038\/s41586-019-1119-1","article-title":"Speech synthesis from neural decoding of spoken sentences","volume":"568","author":"GK Anumanchipalli","year":"2019","journal-title":"Nature"},{"issue":"6437","key":"pcbi.1010401.ref007","doi-asserted-by":"crossref","DOI":"10.1126\/science.aav3932","article-title":"Thirst regulates motivated behavior through modulation of brainwide neural population dynamics","volume":"364","author":"WE Allen","year":"2019","journal-title":"Science"},{"issue":"7504","key":"pcbi.1010401.ref008","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1038\/nature13235","article-title":"Emergence of reproducible spatiotemporal activity during motor learning","volume":"510","author":"AJ Peters","year":"2014","journal-title":"Nature"},{"issue":"6","key":"pcbi.1010401.ref009","doi-asserted-by":"crossref","first-page":"1099","DOI":"10.1016\/j.neuron.2018.05.015","article-title":"Unsupervised Discovery of Demixed, Low-Dimensional Neural Dynamics across Multiple Timescales through Tensor Component Analysis","volume":"98","author":"AH Williams","year":"2018","journal-title":"Neuron"},{"issue":"12","key":"pcbi.1010401.ref010","doi-asserted-by":"crossref","first-page":"2040","DOI":"10.1038\/s41593-019-0533-x","article-title":"Unsupervised identification of the internal states that shape natural behavior","volume":"22","author":"AJ Calhoun","year":"2019","journal-title":"Nature Neuroscience"},{"issue":"35","key":"pcbi.1010401.ref011","doi-asserted-by":"crossref","first-page":"7551","DOI":"10.1523\/JNEUROSCI.1920-17.2018","article-title":"Beyond trial-based paradigms: Continuous behavior, ongoing neural activity, and natural stimuli","volume":"38","author":"A Huk","year":"2018","journal-title":"Journal of Neuroscience"},{"issue":"5","key":"pcbi.1010401.ref012","doi-asserted-by":"crossref","DOI":"10.1088\/1741-2552\/ab2214","article-title":"Dynamic network modeling and dimensionality reduction for human ECoG activity","volume":"16","author":"Y Yang","year":"2019","journal-title":"Journal of Neural Engineering"},{"issue":"4","key":"pcbi.1010401.ref013","doi-asserted-by":"crossref","first-page":"422","DOI":"10.1093\/cercor\/13.4.422","article-title":"Very Slow Activity Fluctuations in Monkey Visual Cortex: Implications for Functional Brain Imaging","volume":"13","author":"DA Leopold","year":"2003","journal-title":"Cerebral Cortex"},{"issue":"9","key":"pcbi.1010401.ref014","doi-asserted-by":"crossref","first-page":"1100","DOI":"10.1038\/nn.2177","article-title":"Interhemispheric correlations of slow spontaneous neuronal fluctuations revealed in human sensory cortex","volume":"11","author":"Y Nir","year":"2008","journal-title":"Nature Neuroscience"},{"issue":"2","key":"pcbi.1010401.ref015","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1016\/j.neuron.2012.08.011","article-title":"Slow Cortical Dynamics and the Accumulation of Information over Long Timescales","volume":"76","author":"CJ Honey","year":"2012","journal-title":"Neuron"},{"issue":"9","key":"pcbi.1010401.ref016","doi-asserted-by":"crossref","first-page":"2019","DOI":"10.1152\/jn.00268.2013","article-title":"A place for time: the spatiotemporal structure of neural dynamics during natural audition","volume":"110","author":"GJ Stephens","year":"2013","journal-title":"Journal of Neurophysiology"},{"issue":"1","key":"pcbi.1010401.ref017","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1146\/annurev-neuro-100219-105424","article-title":"Neuromodulation of Brain State and Behavior","volume":"43","author":"DA McCormick","year":"2020","journal-title":"Annual Review of Neuroscience"},{"issue":"6437","key":"pcbi.1010401.ref018","doi-asserted-by":"crossref","first-page":"eaav7893","DOI":"10.1126\/science.aav7893","article-title":"Spontaneous behaviors drive multidimensional, brainwide activity","volume":"364","author":"C Stringer","year":"2019","journal-title":"Science"},{"key":"pcbi.1010401.ref019","doi-asserted-by":"crossref","DOI":"10.1038\/ncomms3528","article-title":"Numerical processing in the human parietal cortex during experimental and natural conditions","volume":"4","author":"M Dastjerdi","year":"2013","journal-title":"Nature Communications"},{"issue":"APR2016","key":"pcbi.1010401.ref020","first-page":"1","article-title":"Unsupervised decoding of long-term, naturalistic human neural recordings with automated video and audio annotations","volume":"10","author":"NXR Wang","year":"2016","journal-title":"Frontiers in Human Neuroscience"},{"issue":"6","key":"pcbi.1010401.ref021","doi-asserted-by":"crossref","DOI":"10.1088\/1741-2552\/ab355c","article-title":"Neural correlates of unstructured motor behaviors","volume":"16","author":"PG Gabriel","year":"2019","journal-title":"Journal of Neural Engineering"},{"key":"pcbi.1010401.ref022","article-title":"Behavioral and neural variability of naturalistic arm movements","author":"SM Peterson","year":"2020","journal-title":"bioRxiv"},{"issue":"10","key":"pcbi.1010401.ref023","doi-asserted-by":"crossref","first-page":"954","DOI":"10.1038\/nbt.4200","article-title":"Mood variations decoded from multi-site intracranial human brain activity","volume":"36","author":"OG Sani","year":"2018","journal-title":"Nature Biotechnology"},{"issue":"1","key":"pcbi.1010401.ref024","doi-asserted-by":"crossref","DOI":"10.1088\/1741-2552\/aaee9c","article-title":"Coarse behavioral context decoding","volume":"16","author":"A Alasfour","year":"2019","journal-title":"Journal of Neural Engineering"},{"key":"pcbi.1010401.ref025","doi-asserted-by":"crossref","unstructured":"Gabriel P, Doyle WK, Devinsky O, Friedman D, Thesen T, Gilja V. Neural correlates to automatic behavior estimations from RGB-D video in epilepsy unit. Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS. 2016;2016-Octob:3402\u20133405.","DOI":"10.1109\/EMBC.2016.7591458"},{"issue":"1","key":"pcbi.1010401.ref026","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1523\/JNEUROSCI.0532-13.2014","article-title":"Intracranial cortical responses during visual-tactile integration in humans","volume":"34","author":"BT Quinn","year":"2014","journal-title":"Journal of Neuroscience"},{"issue":"9","key":"pcbi.1010401.ref027","doi-asserted-by":"crossref","first-page":"4430 LP","DOI":"10.1073\/pnas.0913697107","article-title":"Cortical activity during motor execution, motor imagery, and imagery-based online feedback","volume":"107","author":"KJ Miller","year":"2010","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"1","key":"pcbi.1010401.ref028","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pbio.1001251","article-title":"Reconstructing speech from human auditory cortex","volume":"10","author":"BN Pasley","year":"2012","journal-title":"PLoS Biology"},{"issue":"2","key":"pcbi.1010401.ref029","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1227\/NEU.0000000000001034","article-title":"Technique, Results, and Complications Related to Robot-Assisted Stereoelectroencephalography","volume":"78","author":"J Gonz\u00e1lez-Mart\u00ednez","year":"2016","journal-title":"Neurosurgery"},{"issue":"5736","key":"pcbi.1010401.ref030","doi-asserted-by":"crossref","first-page":"951","DOI":"10.1126\/science.1110913","article-title":"Neuroscience: Coupling between neuronal firing, field potentials, and fMRI in human auditory cortex","volume":"309","author":"R Mukamel","year":"2005","journal-title":"Science"},{"issue":"43","key":"pcbi.1010401.ref031","doi-asserted-by":"crossref","first-page":"13613","DOI":"10.1523\/JNEUROSCI.2041-09.2009","article-title":"Broadband Shifts in Local Field Potential Power Spectra Are Correlated with Single-Neuron Spiking in Humans","volume":"29","author":"JR Manning","year":"2009","journal-title":"Journal of Neuroscience"},{"issue":"4","key":"pcbi.1010401.ref032","doi-asserted-by":"crossref","first-page":"e1000610","DOI":"10.1371\/journal.pbio.1000610","article-title":"Different Origins of Gamma Rhythm and High-Gamma Activity in Macaque Visual Cortex","volume":"9","author":"S Ray","year":"2011","journal-title":"PLOS Biology"},{"issue":"3","key":"pcbi.1010401.ref033","doi-asserted-by":"crossref","first-page":"524","DOI":"10.1016\/S1388-2457(98)00064-9","article-title":"Increased gamma-range activity in human sensorimotor cortex during performance of visuomotor tasks","volume":"110","author":"F Aoki","year":"1999","journal-title":"Clinical Neurophysiology"},{"key":"pcbi.1010401.ref034","doi-asserted-by":"crossref","first-page":"2301","DOI":"10.1093\/brain\/121.12.2301","article-title":"Functional mapping of human sensorimotor cortex with electrocorticographic spectral analysis","volume":"121","author":"N Crone","year":"1998","journal-title":"Brain"},{"key":"pcbi.1010401.ref035","doi-asserted-by":"crossref","unstructured":"Zanos S, Miller KJ, Ojemann JG. Electrocorticographic spectral changes associated with ipsilateral individual finger and whole hand movement. Annual International Conference of the IEEE Engineering in Medicine and Biology Society IEEE Engineering in Medicine and Biology Society Annual International Conference. 2008;2008:5939\u20135942.","DOI":"10.1109\/IEMBS.2008.4650569"},{"issue":"12","key":"pcbi.1010401.ref036","doi-asserted-by":"crossref","first-page":"2430","DOI":"10.1523\/JNEUROSCI.1368-19.2020","article-title":"Electrocorticogram (ECoG) Is Highly Informative in Primate Visual Cortex","volume":"40","author":"ST Kanth","year":"2020","journal-title":"Journal of Neuroscience"},{"key":"pcbi.1010401.ref037","doi-asserted-by":"crossref","first-page":"126","DOI":"10.1016\/j.jneumeth.2015.01.010","article-title":"Exceeding chance level by chance: The caveat of theoretical chance levels in brain signal classification and statistical assessment of decoding accuracy","volume":"250","author":"E Combrisson","year":"2015","journal-title":"Journal of Neuroscience Methods"},{"issue":"4","key":"pcbi.1010401.ref038","doi-asserted-by":"crossref","first-page":"920","DOI":"10.1109\/TBME.2011.2172210","article-title":"Multiclass Brain\u2013Computer Interface Classification by Riemannian Geometry","volume":"59","author":"A Barachant","year":"2012","journal-title":"IEEE Transactions on Biomedical Engineering"},{"issue":"1","key":"pcbi.1010401.ref039","doi-asserted-by":"crossref","first-page":"614","DOI":"10.1152\/jn.90941.2008","article-title":"Gaussian-Process Factor Analysis for Low-Dimensional Single-Trial Analysis of Neural Population Activity","volume":"102","author":"BM Yu","year":"2009","journal-title":"Journal of Neurophysiology"},{"key":"pcbi.1010401.ref040","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.jneumeth.2015.10.010","article-title":"Extracting spatial-temporal coherent patterns in large-scale neural recordings using dynamic mode decomposition","volume":"258","author":"BW Brunton","year":"2016","journal-title":"Journal of Neuroscience Methods"},{"key":"pcbi.1010401.ref041","doi-asserted-by":"crossref","first-page":"e38471","DOI":"10.7554\/eLife.38471","article-title":"Unsupervised discovery of temporal sequences in high-dimensional datasets, with applications to neuroscience","volume":"8","author":"EL Mackevicius","year":"2019","journal-title":"eLife"},{"key":"pcbi.1010401.ref042","article-title":"Generalized neural decoders for transfer learning across participants and recording modalities","author":"SM Peterson","year":"2021","journal-title":"Journal of Neural Engineering"},{"issue":"10","key":"pcbi.1010401.ref043","doi-asserted-by":"crossref","first-page":"805","DOI":"10.1038\/s41592-018-0109-9","article-title":"Inferring single-trial neural population dynamics using sequential auto-encoders","volume":"15","author":"C Pandarinath","year":"2018","journal-title":"Nature Methods"},{"issue":"6437","key":"pcbi.1010401.ref044","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1126\/science.aax1512","article-title":"Parsing signal and noise in the brain","volume":"364","author":"AC Huk","year":"2019","journal-title":"Science"},{"issue":"9","key":"pcbi.1010401.ref045","doi-asserted-by":"crossref","first-page":"991","DOI":"10.1038\/nn0908-991","article-title":"Finding coherence in spontaneous oscillations","volume":"11","author":"PJ Drew","year":"2008","journal-title":"Nature Neuroscience"},{"key":"pcbi.1010401.ref046","article-title":"Unsupervised discovery of behaviorally relevant brain states in rats playing hide-and-seek","author":"B Bagi","year":"2022","journal-title":"Current Biology"},{"issue":"1","key":"pcbi.1010401.ref047","doi-asserted-by":"crossref","first-page":"184","DOI":"10.1038\/s41597-022-01280-y","article-title":"AJILE12: Long-term naturalistic human intracranial neural recordings and pose","volume":"9","author":"SM Peterson","year":"2022","journal-title":"Scientific Data"},{"key":"pcbi.1010401.ref048","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1016\/j.conb.2021.07.014","article-title":"Unsupervised quantification of naturalistic animal behaviors for gaining insight into the brain","volume":"70","author":"MH McCullough","year":"2021","journal-title":"Current Opinion in Neurobiology"}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1010401","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2022,8,18]],"date-time":"2022-08-18T00:00:00Z","timestamp":1660780800000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1010401","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,8,18]],"date-time":"2022-08-18T17:59:33Z","timestamp":1660845573000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1010401"}},"subtitle":[],"editor":[{"given":"Marieke Karlijn","family":"van Vugt","sequence":"first","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2022,8,8]]},"references-count":48,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2022,8,8]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1010401","relation":{"new_version":[{"id-type":"doi","id":"10.1371\/journal.pcbi.1010401","asserted-by":"object"}]},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,8,8]]}}}