{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,28]],"date-time":"2026-02-28T18:07:06Z","timestamp":1772302026927,"version":"3.50.1"},"reference-count":38,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2021,9,24]],"date-time":"2021-09-24T00:00:00Z","timestamp":1632441600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004663","name":"Ministry of Science and Technology, Taiwan","doi-asserted-by":"publisher","award":["110-2636-E-006-010 (Young Scholar Fellowship Program), 110-2321-B-010-006, 109-2314-B-303-016, and 109-2221-E-010-004-MY2"],"award-info":[{"award-number":["110-2636-E-006-010 (Young Scholar Fellowship Program), 110-2321-B-010-006, 109-2314-B-303-016, and 109-2221-E-010-004-MY2"]}],"id":[{"id":"10.13039\/501100004663","id-type":"DOI","asserted-by":"publisher"}]},{"name":"National Key R&amp;D Program of China","award":["2018YFA0701400"],"award-info":[{"award-number":["2018YFA0701400"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61673346"],"award-info":[{"award-number":["61673346"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities","doi-asserted-by":"publisher","award":["2019XZZX001-01-21"],"award-info":[{"award-number":["2019XZZX001-01-21"]}],"id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Intracortical brain\u2013computer interfaces (iBCIs) translate neural activity into control commands, thereby allowing paralyzed persons to control devices via their brain signals. Recurrent neural networks (RNNs) are widely used as neural decoders because they can learn neural response dynamics from continuous neural activity. Nevertheless, excessively long or short input neural activity for an RNN may decrease its decoding performance. Based on the temporal attention module exploiting relations in features over time, we propose a temporal attention-aware timestep selection (TTS) method that improves the interpretability of the salience of each timestep in an input neural activity. Furthermore, TTS determines the appropriate input neural activity length for accurate neural decoding. Experimental results show that the proposed TTS efficiently selects 28 essential timesteps for RNN-based neural decoders, outperforming state-of-the-art neural decoders on two nonhuman primate datasets (R2=0.76\u00b10.05 for monkey Indy and CC=0.91\u00b10.01 for monkey N). In addition, it reduces the computation time for offline training (reducing 5\u201312%) and online prediction (reducing 16\u201318%). When visualizing the attention mechanism in TTS, the preparatory neural activity is consecutively highlighted during arm movement, and the most recent neural activity is highlighted during the resting state in nonhuman primates. Selecting only a few essential timesteps for an RNN-based neural decoder provides sufficient decoding performance and requires only a short computation time.<\/jats:p>","DOI":"10.3390\/s21196372","type":"journal-article","created":{"date-parts":[[2021,9,27]],"date-time":"2021-09-27T22:16:38Z","timestamp":1632780998000},"page":"6372","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Selection of Essential Neural Activity Timesteps for Intracortical Brain\u2013Computer Interface Based on Recurrent Neural Network"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5893-6299","authenticated-orcid":false,"given":"Shih-Hung","family":"Yang","sequence":"first","affiliation":[{"name":"Department of Mechanical Engineering, National Cheng Kung University, Tainan City 701, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jyun-We","family":"Huang","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, National Cheng Kung University, Tainan City 701, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chun-Jui","family":"Huang","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, National Cheng Kung University, Tainan City 701, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Po-Hsiung","family":"Chiu","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, National Cheng Kung University, Tainan City 701, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6413-0179","authenticated-orcid":false,"given":"Hsin-Yi","family":"Lai","sequence":"additional","affiliation":[{"name":"Key Laboratory of Medical Neurobiology of Zhejiang Province, Department of Neurology of the Second Affiliated Hospital, Interdisciplinary Institute of Neuroscience and Technology, Zhejiang University School of Medicine, Zhejiang University, Hangzhou 310027, China"},{"name":"Key Laboratory for Biomedical Engineering of Ministry of Education, College of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou 310027, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"You-Yin","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Biomedical Engineering, National Yang Ming Chiao Tung University, Taipei 112, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,9,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"380","DOI":"10.1109\/TNSRE.2019.2962708","article-title":"Sparse Ensemble Machine Learning to improve robustness of long-term decoding in iBMIs","volume":"28","author":"Shaikh","year":"2019","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1109\/TNSRE.2020.3034234","article-title":"Feature-Selection-Based Transfer Learning for Intracortical Brain\u2013Machine Interface Decoding","volume":"29","author":"Zhang","year":"2020","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1142","DOI":"10.1038\/nm.3953","article-title":"Clinical translation of a high-performance neural prosthesis","volume":"21","author":"Gilja","year":"2015","journal-title":"Nat. Med."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1669","DOI":"10.1038\/s41591-018-0171-y","article-title":"Meeting brain\u2013computer interface user performance expectations using a deep neural network decoding framework","volume":"24","author":"Schwemmer","year":"2018","journal-title":"Nat. Med."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"9390","DOI":"10.1523\/JNEUROSCI.1669-18.2018","article-title":"Latent factors and dynamics in motor cortex and their application to brain\u2013machine interfaces","volume":"38","author":"Pandarinath","year":"2018","journal-title":"J. Neurosci."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1085","DOI":"10.1162\/neco_a_01189","article-title":"Decoding movements from cortical ensemble activity using a long short-term memory recurrent network","volume":"31","author":"Tseng","year":"2019","journal-title":"Neural Comput."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Tampuu, A., Matiisen, T., \u00d3lafsd\u00f3ttir, H.F., Barry, C., and Vicente, R. (2019). Efficient neural decoding of self-location with a deep recurrent network. PLoS Comput. Biol., 15.","DOI":"10.1371\/journal.pcbi.1006822"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"026011","DOI":"10.1088\/1741-2552\/abde8a","article-title":"Robust and accurate decoding of hand kinematics from entire spiking activity using deep learning","volume":"18","author":"Ahmadi","year":"2021","journal-title":"J. Neural Eng."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Wang, Y., Truccolo, W., and Borton, D.A. (2018, January 17\u201321). Decoding Hindlimb Kinematics from Primate Motor Cortex using Long Short-Term Memory Recurrent Neural Networks. Proceedings of the 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, (EMBC), Honolulu, HI, USA.","DOI":"10.1109\/EMBC.2018.8512609"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"587","DOI":"10.3389\/fnins.2016.00587","article-title":"An improved unscented kalman filter based decoder for cortical brain-machine interfaces","volume":"10","author":"Li","year":"2016","journal-title":"Front. Neurosci."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zhang, P., Chao, L., Chen, Y., Ma, X., Wang, W., He, J., Huang, J., and Li, Q. (2020). Reinforcement Learning Based Fast Self-Recalibrating Decoder for Intracortical Brain\u2013Machine Interface. Sensors, 20.","DOI":"10.3390\/s20195528"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"246","DOI":"10.1016\/j.neuron.2019.10.020","article-title":"Discovering precise temporal patterns in large-scale neural recordings through robust and interpretable time warping","volume":"105","author":"Williams","year":"2020","journal-title":"Neuron"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1152\/jn.00329.2018","article-title":"A muscle-activity-dependent gain between motor cortex and EMG","volume":"121","author":"Naufel","year":"2019","journal-title":"J. Neurophysiol."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Mwata-Velu, T.y., Ruiz-Pinales, J., Rostro-Gonzalez, H., Ibarra-Manzano, M.A., Cruz-Duarte, J.M., and Avina-Cervantes, J.G. (2021). Motor Imagery Classification Based on a Recurrent-Convolutional Architecture to Control a Hexapod Robot. Mathematics, 9.","DOI":"10.3390\/math9060606"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2222","DOI":"10.1109\/TNNLS.2016.2582924","article-title":"LSTM: A search space odyssey","volume":"28","author":"Greff","year":"2016","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Ahmadi, N., Constandinou, T.G., and Bouganis, C.-S. (2019, January 2\u20136). Decoding Hand Kinematics from Local Field Potentials using Long Short-Term Memory (LSTM) Network. Proceedings of the 9th International IEEE\/EMBS Conference on Neural Engineering (NER), Shanghai, China.","DOI":"10.1109\/NER.2019.8717045"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Park, J., and Kim, S.-P. (2019, January 18\u201320). Estimation of Speed and Direction of Arm Movements from M1 Activity Using a Nonlinear Neural Decoder. Proceedings of the 7th International Winter Conference on Brain-Computer Interface (BCI), Gangwon, Korea.","DOI":"10.1109\/IWW-BCI.2019.8737305"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Pei, W., Baltrusaitis, T., Tax, D.M., and Morency, L.-P. (2017, January 21\u201326). Temporal Attention-Gated Model for Robust Sequence Classification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.94"},{"key":"ref_19","first-page":"1243","article-title":"Learning to combine foveal glimpses with a third-order boltzmann machine","volume":"23","author":"Larochelle","year":"2010","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Hu, J., Shen, L., and Sun, G. (2018, January 18\u201322). Squeeze-and-Excitation Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00745"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Yang, P., Hu, V.T., Mettes, P., and Snoek, C.G. (2020, January 23\u201328). Localizing the Common Action Among a Few Videos. Proceedings of the European Conference on Computer Vision, Glasgow, UK.","DOI":"10.1007\/978-3-030-58571-6_30"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Liu, D., Jiang, T., and Wang, Y. (2019, January 15\u201320). Completeness Modeling and Context Separation for Weakly Supervised Temporal Action Localization. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00139"},{"key":"ref_23","unstructured":"O\u2019Doherty, J.E., Cardoso, M., Makin, J., and Sabes, P. (2020, September 01). Nonhuman Primate Reaching with Multichannel Sensorimotor Cortex Electrophysiology. Available online: https:\/\/zenodo.org\/record\/583331."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/sdata.2018.55","article-title":"Massively parallel recordings in macaque motor cortex during an instructed delayed reach-to-grasp task","volume":"5","author":"Brochier","year":"2018","journal-title":"Sci. Data"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"026010","DOI":"10.1088\/1741-2552\/aa9e95","article-title":"Superior arm-movement decoding from cortex with a new, unsupervised-learning algorithm","volume":"15","author":"Makin","year":"2018","journal-title":"J. Neural Eng."},{"key":"ref_26","unstructured":"(2021, September 08). Temporal Attention-Aware Timestep Selection. Available online: https:\/\/github.com\/nclab-me-ncku\/Temporal_Attention_LSTM."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1207\/s15516709cog1402_1","article-title":"Finding structure in time","volume":"14","author":"Elman","year":"1990","journal-title":"Cogn. Sci."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_29","unstructured":"Chung, J., Gulcehre, C., Cho, K., and Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv."},{"key":"ref_30","first-page":"20150202","article-title":"Principal component analysis: A review and recent developments","volume":"374","author":"Jolliffe","year":"2016","journal-title":"Philos. Trans. Royal Soc. A Math. Phys. Eng. Sci."},{"key":"ref_31","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_32","first-page":"21","article-title":"Power comparisons of shapiro-wilk, kolmogorov-smirnov, lilliefors and anderson-darling tests","volume":"2","author":"Razali","year":"2011","journal-title":"J. Stat. Modeling Anal."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1126\/science.286.5437.105","article-title":"Enhanced cortical dopamine output and antipsychotic-like effects of raclopride by \u03b12 adrenoceptor blockade","volume":"286","author":"Hertel","year":"1999","journal-title":"Science"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1523\/ENEURO.0506-19.2020","article-title":"Machine learning for neural decoding","volume":"7","author":"Glaser","year":"2020","journal-title":"Eneuro"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1960","DOI":"10.1109\/TBME.2005.856245","article-title":"Impedance characterization of microarray recording electrodes in vitro","volume":"52","author":"Merrill","year":"2005","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1394","DOI":"10.1016\/j.neuron.2014.04.045","article-title":"Optimal control of transient dynamics in balanced networks supports generation of complex movements","volume":"82","author":"Hennequin","year":"2014","journal-title":"Neuron"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1025","DOI":"10.1038\/nn.4042","article-title":"A neural network that finds a naturalistic solution for the production of muscle activity","volume":"18","author":"Sussillo","year":"2015","journal-title":"Nature Neurosci."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Kaufman, M.T., Seely, J.S., Sussillo, D., Ryu, S.I., Shenoy, K.V., and Churchland, M.M. (2016). The largest response component in the motor cortex reflects movement timing but not movement type. Eneuro, 3.","DOI":"10.1523\/ENEURO.0085-16.2016"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/19\/6372\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:04:24Z","timestamp":1760166264000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/19\/6372"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,24]]},"references-count":38,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2021,10]]}},"alternative-id":["s21196372"],"URL":"https:\/\/doi.org\/10.3390\/s21196372","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,24]]}}}